| Age | Commit message (Collapse) | Author |
|---|
|
The XML generation underlying our package feeds was doing 1600+ calls to
the write() method on the outfile. For some reason, the Python standard
library insists on calling flush() after every write, which really makes
performance take a nosedive. Wrap the write calls and do them in batches
to remove some of the overhead and make feed generation a bit snappier.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This makes entering new releases a bit less cumbersome as we don't
really need to enter either the file size or the torrent infohash.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We were missing two duration-related attributes here, causing some 500
errors to happen if we had cached status_data around that didn't agree
with our current list of checked mirrors. Don't blow up on the JSON data
fetch by ensuring we provide a value, even if it is out of date.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This will work with both the newer and older versions.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This gives us some large memory savings in python due to the internal
storage of Unicode strings vs. byte strings, as well as saving us
processing time up front for filelist data we are never going to have to
actually use.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
For packages with filelists with > 80,000 items, we were starting to see
some serious memory issues in reporead. This was both on the statement
generation side in Python as well as on the database side. Break the
updates into chunks of 10,000 when we encounter packages with tons of
files to control things in a bit.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This will allow those that care about mirrors to zoom into URL-level
details for each mirror and examine the individual check results.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
It doesn't do much good to mark a function as cacheable if we call it
every single time with different arguments due to using the current date
and time. Fix it by passing the offset in instead.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
The return value from get_recent_updates() was too big for memcached due
to all the attached objects, so the cache never actually worked. This
sucks, because we ended up doing all the work in this function and most
of the time we didn't use it because template fragment caching kicked
in.
Remove the cache_function decorator from this method, and instead
implement delayed calling of the function so we don't compute values we
aren't going to use. Template fragment caching will help us in most
cases.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This looks really crazy on our current Tier 1 mirrors, as some of them
have tons of downstream mirrors. Instead, do what we did on the package
details page and allow wrapping of the comma-separated list.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Regardless of whether the mirror URL is active or not, we often have
data we can show the end user, especially if mirror admins care to see
the data we've been gathering.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We were spending a lot of time getting the developer name for a given
key on this page, which involved sending one query per key ID. Use the
cache to our advantage here and save ourselves the "expensive" lookups.
This eliminates ~100 queries per page load.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
It turns out this is a HUGE part of our slow mirror status template
rendering, due to the internal workings. Everything is converted to a
Python decimal object which is way slower than just staying in native
floating point. Given we are always dealing with floats when we need to
do our formatting, a home-rolled template tag can accomplish this much
faster.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
I was never using this and it adds some pretty substantial overhead to
each render of the admin page, so toss it. Add the simple date filter to
the sidebar instead.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This allows the tag to be used in a few more places we weren't already
able to use it, and hopefully speeds up rendering a tad on the package
differences page.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
|
|
|
|
|
|
From our friends over at web.archive.org again, thanks!
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
For something like gambas3 which has a makedepend on postgresql, we end
up getting every single split package listed in the required by list for
postgresql. This is a bit crazy and unnecessary, so slim it down a bit
when possible by using a slightly crazy groupby function and some smarts
in our get_requiredby function.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Use get_or_create, even though it leaves a bad taste in my mouth. The
first user created won't have a profile becuase Django doesn't create
one for users created at the command line, causing an exception when
the user goes to edit it.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We've moved onto bytes only now.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Now that we have a BinaryField option in Django 1.6, put it to use.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
PendingDeprecationWarning: commit_on_success is deprecated in favor of atomic.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
PendingDeprecationWarning: 'The `cycle` template tag is changing to
escape its arguments; the non-autoescaping version is deprecated. Load
it from the `future` tag library to start using the new behavior.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
PendingDeprecationWarning: Creating a ModelForm without either the 'fields' attribute or the 'exclude' attribute is deprecated - form needs updating
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
PendingDeprecationWarning: `queryset` method should be renamed `get_queryset`
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Not a common case, but one we can and should support and hasn't been
noticed up until this point. That pesky colon! Fixes FS#37477.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Also avoid internal redirect to the Installation Guide article.
Fixes FS#36111.
Dan: remove "Official" from install guide link text as well to match
wiki page name.
|
|
|
