| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This reverts commit 3c4ceb16. We don't need this anymore as bulk_create
gets automatic batching now on sqlite3 so it is safe to use.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We have been better about doing this to most of our models, but the ones
here didn't have a created field. Add it where appropriate and set a
reasonably old default value.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
For now, it is not included in the default selection, but we have a few
existing mirrors that do support it.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This was added in Django 1.5 and allows saving only a subset of a
model's fields. It makes sense in a few cases to utilize it.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Make it long enough to support a full-form IPv6 address with a subnet.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Just as we do for the normal status HTML view.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Less noticeable in production as the templates don't show
'@@@INVALID@@@' there, but we were trying to access attributes that
don't actually exist on certain mirror objects.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We now always look for this information at the URL level, not the mirror
level. This simplifies quite a bit of code in and around the mirror
views.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Rather than have the weird indirection we need now to find the right
country for URLs, just always store it on the URL.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Bug #16519 in Django deprecates mimetype, so update our code
accordingly.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This seems to generate much more performant queries at the database
level than what we were previously doing, and also doesn't show
duplicate rows.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We need to ensure we don't duplicate URLs in the status view, so add a
distinct() call back in to the queryset when it was inadvertently
dropped in commit a2cfa7edbb. This negates a lot of the performance
gains we had, unfortunately, so it looks like a nested subquery might be
more efficient. Disappointing the planner can't do this for us.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Rather than the old idiom of dict((k, v) for <> in <>).
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Now that we have as many mirror URLs as we do, we can do a better job
fetching and aggregating this data. The prior method resulted in a
rather unwieldy query being pushed down to the database with a
horrendously long GROUP BY clause. Instead of trying to group by
everything at once so we can retrieve mirror URL info at the same time,
separate the two queries- one for getting URL performance data, one for
the qualitative data.
The impetus behind fixing this is the PostgreSQL slow query log in
production; this currently shows up the most of any queries we run in
the system.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Otherwise we are doing one query per mirror, which at this point is over
100 separate queries.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
* Clamp y-axis minimum to 0.
* Don't plot `is_success == false` values.
* Ensure URLs are sorted predictably.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We might have to tweak the interpolation method once we see this with
real data, but for now it looks really pretty locally.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Still have some hardcoded stuff to rip out and replace to make this a
bit more dynamic on things like sizing, but for now, this is a great
start.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
When we don't need them all, no need to fetch them all. Let the database
do the work for us, hopefully.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
When asking for status for a single mirror, we can include logs from the
past 24 hours in addition to the normal information we provide. This is
slated for usage by a frontend graph still to come, similar to those on
the NTP pool website.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We don't need the full mirror log objects; we just need a very small
subset of values from them here to do the required math and object
building.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Fixes FS#31503.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This makes it easier to do manual manipulation/insertion/etc. at the
database level, as well as just making things act more sane from an
overall software stack perspective.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Now that Django actually provides a concise way to use a RequestContext
object without instantiating it, we can use that rather than the old
function-based generic view that worked well to do the same.
Additionally, these function-based generic views will be gone in Django
1.5, so might as well make the move now.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This was around from the time when we handled timezones sanely and
Django did not; now that we are on 1.4 we no longer need our own code to
handle this.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This was a dumb oversight on my part in commit 0f3c894e7a0.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
It isn't worth it, as we run into the 999 max SQL statement variables
issue when using it on any significant amount of mirrors. Since this is
just a development database setup, and it isn't a command we need to run
especially fast, we can ditch it.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Because this function isn't shipped by default, it makes more sense to
just omit it completely from the query we do to build the tables on this
page when in development. Substitute 0.0 for the value so the rest of
the calculations and display work as expected.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Managed to see this bubble up today when running the mirrorcheck command
on a less than ideal connection that was experiencing timeouts at the
wrong time.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We have a lot of these in the freeform text area in the mirror notes;
attempt to make this data usable as necessary if we want to do some sort
of mirror notification automation in the future.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This should make it easier to catch errors in our Tier 1 mirrors.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
The main changes in this patch implement rsync:// protocol checking
support by calling the rsync binary, requested in FS#29878. We track and
log much of the same things as we already do for FTP and HTTP URLs-
check time, last sync, total check duration, etc.
Also added in this patch is a configurable timeout value which defaults
to the previous hardcoded value of 10 seconds; this can be passed as an
option to the mirrorcheck command.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Use TextField rather than a limited-length CharField; leave it up to the
code filling this field to determine an appropriate length.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
If results weren't available for certain URLs, they won't show up anyway
in this list, and if we start to check rsync URLs, then we want their
values to come back in this status list.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
As of Python 2.6, this is a builtin module that has all the same
functions and capabilities of the Django simplejson module. Additionally
simplejson is deprecated in the upcoming Django 1.5 release.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This will only list FTP mirrors for a given country if there are no HTTP
mirrors available, since FTP must die.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Add a helper method that checks if we know about the protocol; if so, we
can spit out a URL for it. This allows (if you are insane) generation of
an rsync mirrorlist, for instance.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
98% of the time, we won't need to update the existing values as it will
be the same as the prior run of this command. Do a quick check of the
old and new values and don't send anything to the database if there is
no need for an update.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
It would be straightforward if we just needed to remove the field class
definition, but we also need to update all migrations that referenced it
so we don't have problems starting and migrating from scratch.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
