summaryrefslogtreecommitdiff
path: root/devel
AgeCommit message (Collapse)Author
2012-07-08Work around bulk_create limitations in sqlite3 in reporeadDan McGee
Given the 999 SQL statement variable limit, we can easily hit it when updating a package with thousands of files or a few hundred depends. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-07-08reporead: disable FULL synchronous writes for sqlite3Dan McGee
At least on Linux, we hit a huge bottleneck waiting for the FULL commit to happen for each added package during reporead operations. It makes much more sense to back this off to FULL level instead, which trades some possible loss of durability for speedier operation. Additionally, no one would possibly be running their production version of this site on sqlite3, right? Signed-off-by: Dan McGee <dan@archlinux.org>
2012-07-05reporead: don't append slash to empty (root) directoryDan McGee
Add the slash only if we have a directory name, and not otherwise. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-07-05reporead: handle files in root directory properlyDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-07-05reporead: properly handle cases where last_update == files_last_updateDan McGee
We should assume the filelists are up to date in this case, not out of date. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-07-01Log package updates during reporead invocationDan McGee
This adds a Manager and log_update method to help log all updates made to the packages table during reporead runs. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-06-20Add a last_modified field to user profilesDan McGee
A behind the scenes field that might be slightly useful. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-06-07Use 3 decimal places for showing compression ratioDan McGee
Otherwise there are too many grouped under each value. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-05-19reporead: fix copy/paste issueDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-05-18Switch to usage of new Depend objectDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-05-12Change to new time access methods in pgpdump codeDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-05-02rematch_developers: do mass updates instead of single savesDan McGee
When updating a lot of objects, it makes much more sense to perform targeted update queries rather than one-row-at-a-time saves. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-05-02Ensure order_by default value is cleared when using distinct()Dan McGee
Otherwise the queryset returns nonsensical results. I find the design of this less than obvious but so be it; we can ensure the results work regardless of a default ordering on the model. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-29Add a "last action" column to developer clocks pageDan McGee
This allows people to easily see if a developer has done anything recently that we can easily grab a date for. Obviously this doesn't include all sources of activity, so the list of things checked is clearly stated at the top. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-27Dev dashboard performance improvementDan McGee
Rather than one query per cell in the arches and repos statistics tables, we can group these together up front using Django annotations. This means we only need one query per table. In my local instance with all of the staging repos imported, this reduces the total query count on this page from 56 to 26, a rather marked improvement. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-27Incomplete-only todolists optimizationDan McGee
We can push this down to the database if we know in advance we only need the incomplete lists. This helps our call on the developer dashboard quite a bit; the time of the single query in question drops from >1300ms to around 40ms. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-26Add migration to auto-populate developer countries, display flagsDan McGee
Add a quick and dirty migration to derive country info from the developer-provided timezone, and display the flag next to the location if we have it available on the clocks and developer profiles pages. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-26Add country attribute to developer profileDan McGee
We can then show little flag icons later on in relevant places based off of this, such as on the developer profile, clocks page, etc. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-20UserProfile model and fields shuffleDan McGee
Move this model into the devel/ application, and move the PGPKeyField which is used only by these models into the application as well. This involves updating some old migrations along the way to ensure we don't reference a field class that no longer exists. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-08Change old packages report from two years to one yearDan McGee
Now that all packages are getting more attention, we can shorten the time period on the report to a shorter date range. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-07Add a mismatched signatures developer reportDan McGee
This finds odd signatures in our repositories, which includes signature times not matching with build dates, different signer and packager, etc. We enhance our user lookup helper class to look up users by PGP key. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-26Rename 'packagedepend_set' attribute to 'depends'Dan McGee
We do this for every other related package attribute, so do it here too. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-23reporead: use bulk_create() for more propertiesDan McGee
Depends, conflicts, provides, etc. can all be done via bulk_create. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-23Merge branch 'django14'Dan McGee
Conflicts: templates/releng/result_section.html
2012-03-23Make all datetime objects fully timezone awareDan McGee
This is most of the transition to Django 1.4 `USE_TZ = True`. We need to ensure we don't mix aware and non-aware datetime objects when dealing with datetimes in the code. Add a utc_now() helper method that we can use most places, and ensure there is always a timezone attached when necessary. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-23Change Django urls.py importDan McGee
Until Django 1.3, the functions include(), patterns() and url() plus handler404, handler500 were located in a django.conf.urls.defaults module. In Django 1.4, they live in django.conf.urls. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-23reporead: use Django 1.4 bulk_create() for package filesDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-23reporead: use Django 1.4 select_for_update()Dan McGee
As per TODO comments in the existing code. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-23reporead: blow up when package found with wrong architectureDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-21Ensure clocks Expires header works in 59th minute of hourDan McGee
This was a rather careless oversight on my part. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-21Add number of keys signed to master keys pageDan McGee
This really just makes me look bad, but an interesting fact none the less for people to look at. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-16reporead: rename Pkg to RepoPackageDan McGee
The bytes saved on the shorter name aren't worth it. Also ensure 'desc' is always initialized to None in case packages do not provide one. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-07Exclude all compressed manpagesDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-02-22import_signatures: code cleanup and ensure we update validity statusDan McGee
This adds a namedtuple so we aren't using magic numbers when processing our 'edges' (signatures). We also ensure we update any existing signature objects with their validity if they were later revoked. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-02-12Reduce false positives returned by uncompressed manpages reportDan McGee
Explicitly exclude '.html' files from the report. This fixes the issue noted in FS#28372. We also add `is_directory=False` to both the manpage and info page queries; this could help the database come up with a better plan. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-02-11reporead: only reset flag date if upstream version changesDan McGee
This preserves the flag date if only a simple pkgrel bump occurred, which makes sense more often than not for rebuilds. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-02-07Make rematch command do a bit moreDan McGee
Now that we have a few objects that can potentially link back to developers, allow flag requests to also be rematched. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-01-29Remove ambiguity in PGPSignature verbose field namesDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-01-19reporead: simplify and fix transaction management in update_common()Dan McGee
We can use the easier transaction.commit_on_success() decorator if we be sure to explicitly mark the transaction dirty. This fixes the issue where a raised exception in this code called neither commit nor rollback. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-01-05Adjust page and content caching lengths and decoratorsDan McGee
Remove never_cache from many places now that we don't actually need it since we aren't caching by default. Adjust our cache_function decorator times be shorter values, and also randomize them a bit to make cache invalidations not all line up. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-22Fix mismatched URL keyword argDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-12reporead: more efficient deletion of filesDan McGee
Rather than delegating to Django and batch deletion by ID, force issuing of a single delete query to clear out all existing file objects when necessary. This should speed up the deletion and update of packages with a lot of files by a non-trivial amount. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-11PyLint suggested cleanupsDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-09Prefilter packages when looking up uncompressed man/info pagesDan McGee
This vastly speeds up the reports if you just want to look at your own packages and not the complete list, especially if the list of packages you maintain is relatively short. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-07Convert master key datetimes to datesDan McGee
The extra precision is not necessary and just makes entry of the data harder anyway.
2011-12-07Enhance devel model adminDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-05Add a stub admin command to help fix permissions/content typesDan McGee
I needed this today to get the application working from scratch on another host. Probably not all there yet, but we'll see how far it gets us. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-05Add import signatures management commandDan McGee
This allow importing signatures from a provided gpg keyring, such as that produced by the generate_keyring management command that already exists. These will eventually be used for producing stats involving developer signing keys and their certification by master keys. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-05Add new PGPSignature modelDan McGee
This will be used to track cross-developer and master key signatures to build a visualization in the web interface of key signatures, as well as be able to provide info on who is verified, who is not, etc. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-12-03reporead: don't update timestamp on --forceDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
generated by cgit v1.2.3-54-g00ecf (git 2.45.2) at 2024-11-02 07:26:13 +0000