summaryrefslogtreecommitdiff
path: root/devel
AgeCommit message (Collapse)Author
2011-02-27reporead: small cleanupsDan McGee
2011-02-23reporead performance improvementsDan McGee
When importing over a million files, it makes sense to take the slightly faster route and call the PackageFile() constructor directly rather than going through the related manager's create method. We can also get huge performance improvements, especially with files databases, by using the 'io' rather than 'codecs' module. The former is now implemented in C in 2.7 and results in a no-work import (so measuring only the DB read speed) of extra.files.tar.gz from ~30 seconds to ~5 seconds. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-22Use new split package file fields everywhereDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-16Read in optional deps and show in web interfaceDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-15Move license to a related modelDan McGee
This allows us to store multiple licenses per package in a more elegant fashion, and will later allow us to search and filter on this information. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-15Clean up Package related objects codeDan McGee
Main change is just to move groups from the default packagegroup_set location to a related_name of groups. Also refer to the Package class directly rather than by text string if we have it available. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-31Defer format string substitution to loggerDan McGee
Don't use 'fmtstr % (arg1, arg2)' type format; logger can be passed a format string and the arguments to populate it. Saves a bit of work for strings that never end up getting displayed anyway. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-08Add developer world clocks pageDan McGee
This should make it easier for everyone to figure out whether someone is awake, sleeping, and all that fun stuff. It does require everyone to update their profile and fill in the field, but that shouldn't be too hard of a task. Don't suggest jokes to me Saturday morning on IRC unless you really want to see them implemented. Thanks, Pierre! Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-29Commit during reporead on a per-arch basisDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-13Move more URLs out of root urlconfDan McGee
Things are a bit cleaner now. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-11-27Remove print statementDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-11-04reporead: fix small issues when loading files databasesrelease_2010-11-04Dan McGee
We didn't verify that the version in the files database was the same as in the SQL side of things, so we could load old files for a new package and lose track of this fact. When loading files, ensure the database version matches the version in the package before continuing with the file load operation. There are also a few other small updates in here, like skipping the sanity check for filesonly as we never delete packages, and removing some unnecessary string concatenation operations. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-11-04reporead: Use transparent compression on openDan McGee
And make filename check more lenient. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-11-04reporead: orphaning code cleanup and debug removalDan McGee
Cleanup to some of the orphan code cleanup, especially so we are never lying in the percentage we print, and remove a bunch of debug prints that aren't all that useful. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-19Unify hyphenation of out-of-date textDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-06Make it actually possible to upload a new dev imageDan McGee
We need to both submit the form with the correct encoding and then bind the form itself to request.FILES and not just request.POST. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-05Adding changing of user profile detailsAngel Velasquez
The idea of this patch is allow to the developers who have an account, to change their data without asking some admin to do it for them. Dan: put private email address field back as it is used for a different purpose; add some help text and field names as appropriate. Signed-off-by: Angel Velasquez <angvp@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-04Make user profile a OneToOneFieldrelease_2010-10-05Dan McGee
We had this set up as a unique ForeignKey before, which adds some indirection due to the RelatedManager object being there. By making it a OneToOneField, we can get the profile object directly, enforce uniqueness, and also use it in select_related() calls to make our profiles page a bit more efficient. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-04reporead: ignore nicknames in name matching codeDan McGee
2010-09-24Make general mirror list view publicDan McGee
Hide some columns when not logged in because they aren't relevant for the general public, but this will work nicely as a base page for all of our known mirrors. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-12reporead: revamp database parsing codeDan McGee
This needed a little sprucing up as it has grown quite organically over the life of this script. Make things a bit more pythonic through the use of iterators rather than collection indexing, and try to generalize the special cases of things a bit. Also catch encoding problems early and fail gracefully rather than blow up the entire package parser. A failed decode of a file should cause us to just skip it rather than stop the entire parser. Worst case, this leaves that package out of the web interface. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-12Check for correct permission when creating usersrelease_2010-09-12Dan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-10Restore flagged package count by maintainer in dashboardDan McGee
We need to do a little dropping into SQL to accomplish this, but it isn't all that bad to actually do and we can do the whole thing in one query. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-10reporead: allow traceback on USR1 signal as wellDan McGee
When I have caught reporead behaving badly on the production box, I haven't been able to successfully get a traceback without killing the process. Hopefully using a different signal will allow me to actually capture some data. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-10Remove SomethingFishyExceptionDan McGee
Just use a plain Exception instead since we don't get any added value by subclassing. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-06Move mirror models out of main appDan McGee
South actually makes this relatively painless if you get everything right, so might as well start getting these out of the legacy main application to eventually eliminate models being separate from their views. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-03Use direct_to_template in all remaining possible placesDan McGee
Rather than the need to include RequestContext() calls directly, we can just use direct_to_template to do all the work for us. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-28PyLint suggested cleanupsDan McGee
We had a bunch of extra imports, non-conventional variable names, spacing issues, etc. that were relatively low-hanging fruit to clean up. Fix them and make the code a bit cleaner in the process. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-27Add table sorting to a handful of pagesDan McGee
Anything that it makes sense to add it to gets the treatment here. Anything with pagination can wait as that will be tougher. We also need to deal with odd/even formatting. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-04reporead: Add ability to generate traceback via signalrelease_2010-08-08Dan McGee
Every once in a while we see this command hanging on the main server but it isn't making any system calls, so it is hard to tell where it is getting stuck. Add a signal handler on SIGQUIT that will listen and print a traceback when signaled. This is the easiest thing to implement; future additions may need to be able to hook up to a remote debugger (e.g. pdb) if this doesn't work. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-01Fix stupid mistake in user parsingDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-28Clean up find_user() code a bitrelease_2010-07-28Dan McGee
With suggestions from Jason Chu, make the code a bit less repetitive with regards to exception handling and fallthrough to the next method of finding the user. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-28Add missing group delete in reporeadDan McGee
Otherwise we get duplicate groups each time we update the package, and any group removals would never actually happen. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-28Add packager support to reporeadDan McGee
This is a bit more work than just a simple field addition. We attempt to map packager specs (e.g. "A. U. Thor <author@example.com>") to actual Django users in a relatively robust way- first try matching on User.email, then fall back to UserProfile.public_email, then finally try a name-based match. For those packages we can't generate a mapping, the raw string is still stored so it can be displayed. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-25Add package groups model and display to packagesDan McGee
They show up but aren't hotlinked to anything...just yet. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-06Update code to use new flag_date columnDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-05Show incomplete todo list packages in dev dashboardDan McGee
Implements FS#20081. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-06-22Add 'never_cache' decorator in a bunch of placesDan McGee
Now that we cache everything, we need to ensure anyone doing edits and such gets the live data and not some cached version that was already updated and is now stale. Add the never_cache decorator to any of the CUD screens as well as a few others that might benefit from always being regenerated. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-06-08Use Sites framework instead of hardcoded domain nameDan McGee
Instead of putting 'www.archlinux.org' all over the place, use the Django sites framework to pull the site name out of the database. Now these amazing things will work if you are running locally and decide to change the site! Signed-off-by: Dan McGee <dan@archlinux.org>
2010-05-26Have reporead populate filename columnDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-05-24reporead: use the DB package we already haverelease_2010-05-24Dan McGee
Rather than go to the database for every single package on something like a files update, use the one we already have. Duh. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-05-24Fix null field issues exposed by Django 1.1.2Dan McGee
Apparently Django 1.1.1 let null fields pass right through but this now causes reporead to blow up in 1.1.2. Fix the issue and get things working again by allowing nulls where it probably makes sense and including a migration to fix the issue, which for the real database will be a no-op. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-05-24Change link to new dev site locationDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-05-04Ensure changing profile email doesn't reset passwordDan McGee
We weren't checking to see if the password form fields were empty before setting the user password, causing it to get reset if anything was filled out and submitted on this page. FS#19345. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-04-16reporead: allow removal of last package in an archDan McGee
We had a situation where the last 'any' architecture package was present in the [testing] repo and never got removed because we never did the db_update() call on that architecture. Instead of looping all possible architectures and only calling if len() > 0, always call db_update() for both the primary architecture and the 'any' architecture. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-27Simplify code now that pkgbase is always definedrelease_2010-03-28Dan McGee
This allows some of our queries to get a lot simpler as well as removing the pkgbase_safe property added earlier. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-27Make reporead always populate pkgbaseDan McGee
And also add a data migration to add the value retroactively for anything already in our database. We simply fall back to pkgname if pkgbase isn't available. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-27Move package maintainer off of package modelDan McGee
This is an attempt to fix our long-standing problems dealing with maintainer information. Move the actual maintainer information off of the package model into a PackageRelation object, which has some flexibility to later represent more than just maintainership. This solves multiple problems: * If a package gets accidentally deleted, so did the maintainer info * Testing packages have always shown up as orphans * With split packages, it was easy to miss some of the sub-packages This commit does not include the deletion of the original maintainer column; that will come at a later time when I feel more confident that the data was migrated correctly. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-10Fix one missed testing repo checkDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-01Merge branch 'package-files'Dan McGee