summaryrefslogtreecommitdiff
path: root/sitemaps.py
AgeCommit message (Collapse)Author
2015-06-05Revert "Force minimum last modified date for package files pages in sitemap"Dan McGee
This reverts commit 790def2f4db2afec150bab8dc842f6068fb751db. This has been out there long enough that the fallout should be fixed. Go back to just using the default date. Signed-off-by: Dan McGee <dan@archlinux.org>
2015-04-12Force minimum last modified date for package files pages in sitemapDan McGee
We fixed the embarrassing '/usr/bin/None' bug here; force Google and others to reindex these pages if possible by marking them as modified. Signed-off-by: Dan McGee <dan@archlinux.org>
2014-11-08Add a todolists sitemapDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2014-11-08Add a news-specific sitemapDan McGee
This follows the spec at https://support.google.com/news/publisher/answer/74288?hl=en#tagdefinitions Signed-off-by: Dan McGee <dan@archlinux.org>
2014-11-02Add new StaffGroup objectDan McGee
This will allow us to be a bit more dynamic in showing the people listings on the website. We'll be adding a Support Staff category to recognize those that do things around here but aren't technically developers. Signed-off-by: Dan McGee <dan@archlinux.org>
2014-09-01De-emphasize package files pagesDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-06-20Add link to release listing page from public indexDan McGee
We didn't link this page from anywhere so it was hard to find. Also add it to the sitemap so it gets indexed. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-20Package sitemap adjustmentsDan McGee
Adjust changefreq and priority based on the repository the package is in. Testing and staging have quick turnover so set the changefreq to 'daily'. Additionally, give staging packages a super low priority of 0.1 and testing a slightly lowered priority of 0.4. We didn't include staging packages before, but since search engines are finding them anyway, we might as well. This gives us a chance to share how important the page is as well, which they could only guess before. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-08Add a releases model sitemapDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-01-18Include 'files_last_update' in package sitemap queryDan McGee
We need this for the package files sitemap; else we are doing one query per package to retrieve this field. Whoops. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-01-12Add mirror status page to base sitemapDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-09-30Optimize packages sitemap generationDan McGee
* Select arch and repo related models as we need the name off each * Only select the fields we actually need from the database * Exclude [staging] packages Signed-off-by: Dan McGee <dan@archlinux.org>
2012-09-20Defer content and other text items when building news sitemapDan McGee
We can get away with not retrieving these larger pieces of data when building the news sitemap. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-08-04Slight adjustments to sitemaps priority valuesDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2012-04-02Skip default ordering in sitemaps outputDan McGee
The output is not required to be ordered by the specification, so save some effort by skipping any sorting. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-23Make all datetime objects fully timezone awareDan McGee
This is most of the transition to Django 1.4 `USE_TZ = True`. We need to ensure we don't mix aware and non-aware datetime objects when dealing with datetimes in the code. Add a utc_now() helper method that we can use most places, and ensure there is always a timezone attached when necessary. Signed-off-by: Dan McGee <dan@archlinux.org>
2012-03-16Use varying changefreq in news sitemapDan McGee
For those items less than one day old, mark change frequency as 'daily'; for those less than one week old, mark as 'weekly'. Finally, bump everything else up from 'never' to 'yearly' to ensure it gets crawled at least once in a while. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-30Integrate master key into rest of siteDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-11Add a split packages sitemaprelease_2011-11-11Dan McGee
With very low priority, but this should at least give a few more cross-linking pages to any crawlers using sitemaps. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-15Add a base pages sitemapDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-15Use correct date in news sitemapDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-15Refactor common select_related into manager methodDan McGee
For a Package object query, we almost always did .select_related('arch', 'repo). Refactor this into the manager as a 'normal()' method so we can avoid sprinkling the same logic everywhere. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-09-08Move news model to an appropriate placeDan McGee
Never would have guessed it should actually be in news/models.py. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-28Move package utility functionsDan McGee
We now have a few of them, so move them to their own file like we do in the other applications. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-27Add a package groups sitemapDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-10Sitemaps: split and index, add package filesDan McGee
Split it up as it is growing rather large, and add all the package files views to the mix so Google or anyone can pick those up. They get a slightly lower priority (if that even matters). Signed-off-by: Dan McGee <dan@archlinux.org>
2010-02-10Remove archweb prefix from all importsDan McGee
Unnecessary, and lets us standardize on not using it everywhere. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-02-02Make sitemap generation suck a lot lessDan McGee
Add a select_related() clause that was really needed. Signed-off-by: Dan McGee <dan@archlinux.org>
2009-11-09added sitemaps from archweb_pubIsmael Carnales