summaryrefslogtreecommitdiff
path: root/feeds.py
AgeCommit message (Collapse)Author
2015-04-15Merge tag 'release_2014-10-28' into archweb-genericLuke Shumaker
Fix django_countries issues
2015-04-13Merge tag 'release_2014-10-20' into archweb-genericLuke Shumaker
A few Jinja2 fixes and some perf changes
2014-10-26Add last modified date to releasesDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2014-10-26Double batch size in BatchWritesWrapperDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2014-10-26Kill now unneeded workaround for Django issue #9800Dan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2014-10-26Remove Etag header from feedsDan McGee
We have Last-Modified here, and from what I can tell with some more reading and playing with caching, it isn't necessarily wise to set both of them in the same response. Set the one that we actually trust. Signed-off-by: Dan McGee <dan@archlinux.org>
2014-10-19Remove usage of templates for RSS feedsDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2014-10-19Simplify last modified and etags processing for feedsDan McGee
We had this elaborate system set up with caching and invalidation, which is overkill since we cache the result of the view anyway. Just hit the database when needed to find the last change to the respective model class and be done with it. Signed-off-by: Dan McGee <dan@archlinux.org>
2014-05-25Merge tag 'release_2014-03-22' into archweb-genericLuke Shumaker
Fix long filenames; local storage for filter prefs Conflicts: templates/public/download.html
2014-01-11Speed up feeds generation by batching writesDan McGee
The XML generation underlying our package feeds was doing 1600+ calls to the write() method on the outfile. For some reason, the Python standard library insists on calling flush() after every write, which really makes performance take a nosedive. Wrap the write calls and do them in batches to remove some of the overhead and make feed generation a bit snappier. Signed-off-by: Dan McGee <dan@archlinux.org>
2014-01-09Remove remaining references to release file_size fieldDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-07-14Merge branch 'archweb' into archweb-generic2Luke Shumaker
2013-07-13Move simple feeds templates directly into feed classesDan McGee
No need to call out to the template engine to format... nothing at all. Just fetch the attribute directly and save the render step. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-21catch a few more instances of archlinuxLuke Shumaker
2013-04-21Merge branch 'archweb' into archweb-generic2Luke Shumaker
Conflicts: devel/views.py feeds.py public/views.py settings.py sitestatic/archweb.js templates/base.html templates/devel/profile.html templates/mirrors/status.html templates/news/view.html templates/packages/flaghelp.html templates/packages/opensearch.xml templates/public/download.html templates/public/feeds.html templates/public/index.html templates/registration/login.html templates/releng/results.html templates/todolists/public_list.html
2013-03-27Pull references to Arch Linux in code into BRANDING_ variables in settingsLuke Shumaker
This does not include templates
2013-01-19Add basic release list and details viewsDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-01-19Add more metadata to releng Release modelDan McGee
Add a file_size field which we will use in the RSS feed, and also add a field for future storage of the torrent data itself. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-01-19Initial cut of a Releases RSS feedDan McGee
This is our first use of enclosures on the site as well. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-01-13Add __name__ attribute to class-based Feeds viewsrelease_2013-01-13Dan McGee
This should hopefully allow these views to not be labeled as '_wrapped_view' in performance monitoring output. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-11-06Remove explicit timezone handling from feedsDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-11-06Make guid feeds helper a @staticfunctionDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-07-31feeds: add all arches for repo feedDan McGee
If you wanted to see all updates regardless of architecture for [testing] before, there wasn't really a way to do so. Add one. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-07-31Update several bits and pieces for staging packagesDan McGee
This will prevent [staging] packages from cluttering normal user's view on the website, but allow us to still import everything from this repository for developer use. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-05-20Ensure we use last_modified date from News in headersDan McGee
We were actually using the postdate attribute rather than last_modified, which means any News objects that get edited would not trigger an update of the feed. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-23Use python hashlib directlyDan McGee
Django hashcompat is now deprecated. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-15Refactor common select_related into manager methodDan McGee
For a Package object query, we almost always did .select_related('arch', 'repo). Refactor this into the manager as a 'normal()' method so we can avoid sprinkling the same logic everywhere. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-07Consolidate caching black magicDan McGee
Get the stuff used to retrieve and refresh the latest date values all in the same place, and make it a bit more beautiful by refactoring it all into a common set of methods. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-07Use UTC datetime objects everywhereDan McGee
Rather than the twisted mix of local times and UTC times we currently have. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-07Add a horrible hack to allow feed guid value to not be a permalinkDan McGee
Django, you make the simplest things so damn hard sometimes. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-07Ensure feed GUIDs are unchanging and uniqueDan McGee
Implement 'tag:' style URIs for the GUID field on our RSS feeds. This ensures new package updates show up as new, and we aren't jumping back and forth between generated GUIDs having 'http://' and 'https://' prefixes. Much of the work here is to attempt to keep old news GUIDs constant so we don't once again make everything show up as new in newsreaders. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-15Use date from model in news feedDan McGee
Now that this a datetime and not just a date, we can use it directly. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-11-27Fix feeds for case-sensitive databasesDan McGee
This worked in MySQL because of it's case-insensitive matching, but does not work in other databases unless we coerce the value. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-13Connect post_save signals where they will always be triggeredDan McGee
We need to do this in the models.py files, otherwise the post_save signal might not be connected right away on launch of the application. Move them over there, add a dispatch_uid so it only gets hooked up once, and do some other function moving around so we don't have circular imports. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-08Fix potential race conditions in cachingDan McGee
Use a 'set to None' sentinel to indicate data updates are in progress and we need to hold off a bit on caching a new value. This logic is gleamed from the "Scaling Django" slides presented by Mike Malone and available freely on SlideShare. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-07Ensure last modified times use UTCDan McGee
We were cheating before and using non-UTC times; adjust the values we get back from the database as appropriate so our times are not bogus. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-07Store package feed last modified date in memcachedDan McGee
Just like what we did for news items, we can do the same for packages. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-07Store latest news date in memcachedDan McGee
This saves two database queries each request, meaning no database hits at all if we are just going to return a 304 response. It also requires adding a post_save signal to ensure our cache is updated with the correct latest news date upon saving a news item. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-10-07Factor out common last modified code for news feedDan McGee
This will set up retrieving this value from memcached as well as some other changes to come. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-14Improve request handling for feeds that haven't changedDan McGee
By using the condition decorator (in a slightly odd way because these are class-based views), we can cut down a lot on the response time for returning 304 status code for feeds that haven't changed. The decorator means we no longer have to completely render the view to see if we can return a 304 status code. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-08Use arch.agnostic flag everywhereDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-08Move news model to an appropriate placeDan McGee
Never would have guessed it should actually be in news/models.py. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-28PyLint suggested cleanupsDan McGee
We had a bunch of extra imports, non-conventional variable names, spacing issues, etc. that were relatively low-hanging fruit to clean up. Fix them and make the code a bit cleaner in the process. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-27Remove an unnecessary iexactDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-06-08Update feeds to new 1.2 frameworkDan McGee
Feeds are now views-based and don't need the dictionary anymore. get_object() now takes named arguments as well making it a bit more understandable when reading the code. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-05-04feeds: move link from method to attributeDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-02-10Remove archweb prefix from all importsDan McGee
Unnecessary, and lets us standardize on not using it everywhere. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-02-04feeds: add per arch, per repo feed abilityDan McGee
Make the feed framework a lot more flexible and give the possibility to have a feed for each architecture. You can drill down even more than also get a feed for a particular repo; some might find this helpful for something like tracking [testing]. Implements FS#12939. I also bumped up the number of items available in each of these feeds; since it is full of a bunch of small items it might be more helpful to have more available and it should also prevent fewer ones from being missed. The UI isn't exactly spectacular, but I figured some sort of page is better than none listing all the various feeds you can pull from. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-01-31Make the feed titles more descriptiveDan McGee
FS#16752. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-01-31Use select_related() to make a few more places more performantDan McGee
Especially when looking at packages, we always want the arch and repo. Another big hunk of changes deals with the very inefficient signoffs code. Signed-off-by: Dan McGee <dan@archlinux.org>