| Age | Commit message (Collapse) | Author |
|---|
|
# Conflicts:
# templates/public/download.html
|
|
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Angel Velasquez <angvp@archlinux.org>
|
|
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Mirror URL page, other random stuff
|
|
This gives us some large memory savings in python due to the internal
storage of Unicode strings vs. byte strings, as well as saving us
processing time up front for filelist data we are never going to have to
actually use.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
For packages with filelists with > 80,000 items, we were starting to see
some serious memory issues in reporead. This was both on the statement
generation side in Python as well as on the database side. Break the
updates into chunks of 10,000 when we encounter packages with tons of
files to control things in a bit.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Django 1.6 upgrade and other changes
Conflicts:
templates/news/list.html
templates/public/index.html
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
PendingDeprecationWarning: commit_on_success is deprecated in favor of atomic.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Not a common case, but one we can and should support and hasn't been
noticed up until this point. That pesky colon! Fixes FS#37477.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
|
|
Conflicts:
devel/views.py
feeds.py
public/views.py
settings.py
sitestatic/archweb.js
templates/base.html
templates/devel/profile.html
templates/mirrors/status.html
templates/news/view.html
templates/packages/flaghelp.html
templates/packages/opensearch.xml
templates/public/download.html
templates/public/feeds.html
templates/public/index.html
templates/registration/login.html
templates/releng/results.html
templates/todolists/public_list.html
|
|
This was added in 2010 in commit e95c4563e32 as a short-term fix. The
short-term is up.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
|
|
Now that Django 1.5 is out and realized SQLite3 only allows for 999
parameters per SQL call, we don't need to manually batch things up
anymore and can let the underlying bulk_create code do it for us.
This basically reverts commit 88ee61a39ac3.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
A few minor things are fixed here. One is PostgreSQL, and more
specifically pgbouncer, don't like it when the connection is closed
after psycopg2 has started an implicit transaction even for read-only
queries. Ensure we call commit as our last database action in all cases.
The other is related- Django in management commands doesn't ever call
close on any database connection you may have been using, so PostgreSQL
gets mad about this fact and logs a message saying such. Close the
connection explicitly when we are done with it to play nice.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This will be used to eventually implement the UI side of FS#13441, but
to do that, we first need the data.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Rather than the old idiom of dict((k, v) for <> in <>).
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
In the architecture agnostic case, this error is much more likely to
happen, so printing it like an error message is deceiving.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This is the cause of these warnings showing up in the PostgreSQL log:
LOG: unexpected EOF on client connection with an open transaction
All management commands are guilty of this as they do not clean up and
close the connection when they exit, unlike the standard web request
cycle. Other commands should probably be updated as well, but for now,
this is the biggest culprit.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
FS#30323. This will take some time to propagate to all existing
packages, but all new and updated packages will start getting filelists
in the right order.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We don't have these in the database yet, but future verisons of repo-add
will put this information in the sync databases.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Very little dealt directly with this field.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We don't do this anywhere else, so we shouldn't do this here either.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This was around from the time when we handled timezones sanely and
Django did not; now that we are on 1.4 we no longer need our own code to
handle this.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Given the 999 SQL statement variable limit, we can easily hit it when
updating a package with thousands of files or a few hundred depends.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
At least on Linux, we hit a huge bottleneck waiting for the FULL commit
to happen for each added package during reporead operations. It makes
much more sense to back this off to FULL level instead, which trades
some possible loss of durability for speedier operation. Additionally,
no one would possibly be running their production version of this site
on sqlite3, right?
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Add the slash only if we have a directory name, and not otherwise.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We should assume the filelists are up to date in this case, not out of
date.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This adds a Manager and log_update method to help log all updates made
to the packages table during reporead runs.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We do this for every other related package attribute, so do it here too.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Depends, conflicts, provides, etc. can all be done via bulk_create.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Conflicts:
templates/releng/result_section.html
|
|
This is most of the transition to Django 1.4 `USE_TZ = True`. We need to
ensure we don't mix aware and non-aware datetime objects when dealing
with datetimes in the code. Add a utc_now() helper method that we can
use most places, and ensure there is always a timezone attached when
necessary.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
As per TODO comments in the existing code.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
The bytes saved on the shorter name aren't worth it.
Also ensure 'desc' is always initialized to None in case packages do not
provide one.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This preserves the flag date if only a simple pkgrel bump occurred,
which makes sense more often than not for rebuilds.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We can use the easier transaction.commit_on_success() decorator if we be
sure to explicitly mark the transaction dirty. This fixes the issue
where a raised exception in this code called neither commit nor
rollback.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Rather than delegating to Django and batch deletion by ID, force issuing
of a single delete query to clear out all existing file objects when
necessary. This should speed up the deletion and update of packages with
a lot of files by a non-trivial amount.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
