summaryrefslogtreecommitdiff
path: root/git-interface/git-update.py
AgeCommit message (Collapse)Author
2016-10-08Reorganize Git interface scriptsLukas Fleischer
Move the Git interface scripts from git-interface/ to aurweb/git/. Use setuptools to automatically create wrappers which can be installed using `python3 setup.py install`. Update the configuration files, the test suite as well as the INSTALL and README files to reflect these changes. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-09-29Use setuptools to install Python modulesLukas Fleischer
Instead of using relative imports, add support for installing the config and db Python modules to a proper location using setuptools. Change all git-interface scripts to access those modules from the search path. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-09-29git-update: Move entry point to a main() methodLukas Fleischer
Move the main program logic of git-update to a main() method such that it can be used as a module and easily be invoked by setuptools wrapper scripts. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-22git-update: Do not overwrite the repo variableLukas Fleischer
The repo variable is already used to store the pygit2.Repository. Fixes a regression introduced in d273ee5 (Use the official provider list to detect duplicates, 2016-05-17). Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-22git-update: Close cursor before closing databaseLukas Fleischer
When using SQLite as backend, we need to close the cursor before closing the database to avoid the following error: sqlite3.OperationalError: unable to close due to unfinalized statements or unfinished backups Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-10Lazy-add new package basesLukas Fleischer
Create new package bases just before saving package metadata. This protects from stray package bases left behind when new packages are rejected, e.g. when the user tries to push a package that is available from the official repositories already. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05git-update: Remove package details before updatingLukas Fleischer
Explicitly remove all package sources, dependencies, relations, licenses and groups before inserting new ones. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05git-interface: Do not use UNIX_TIMESTAMPLukas Fleischer
Avoid using UNIX_TIMESTAMP which is not part of the SQL standard. Retrieve the current UNIX time in Python and substitute it into the SQL queries instead. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05git-interface: Do not use rowcountLukas Fleischer
Avoid using Cursor.rowcount to obtain the number of rows returned by a SELECT statement as this is not guaranteed to be supported by every database engine. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05git-interface: Factor out configuration file parsingLukas Fleischer
Add a new module that automatically locates the configuration file and provides methods to obtain the values of configuration options. Use the new module instead of ConfigParser everywhere. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05git-interface: Add database abstraction layerLukas Fleischer
Add a new class that connects to the database specified in the configuration file and provides an interface to execute SQL queries. Prepared statements with qmark ("?") placeholders are supported. Replace all direct database accesses with calls to the new abstraction layer. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05git-update: Make maximum blob size configurableLukas Fleischer
Support setting the maximum blob size in the configuration file. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05git-update: Use AUR_PRIVILEGED for forced pushesLukas Fleischer
Instead of looking up the account type of the current user again, use the AUR_PRIVILEGED environment variable to check whether the user is allowed to perform non-fast-forward ref updates. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05git-update: Fix some issues reported by pyflakesLukas Fleischer
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-04git-update: Notify privileged users of forced uploadsLukas Fleischer
Show a warning when a Trusted User or a developer creates a package that is blacklisted or already provided by an official package. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-05-18Use the official provider list to detect duplicatesLukas Fleischer
Instead of automatically adding packages from the official binary repositories to the package blacklist, use the official provider list to prevent users from uploading duplicates. This does not only result in reduced disk usage but also has a nice visible side effect. The error messages printed by the update hook now look like error: package already provided by [community]: powerline-fonts instead of error: package is blacklisted: powerline-fonts which was confusing to most end users. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-07Add support for package update notificationsLukas Fleischer
Introduce a new notification option to receive notifications when a new commit is pushed to a package repository. Implements FS#30109. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-07Rename the CommentNotify table to PackageNotificationsLukas Fleischer
As a preparatory step to adding support for package notifications on events other than comments, rename the database table accordingly. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-07git-update: Replace aurinfo.py with python-srcinfoJohannes Löthberg
python-srcinfo is a more transparent and simpler library for parsing SRCINFO files. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-06Fix issues reported by pyflakesLukas Fleischer
Fix several style issues and remove unneeded imports/assignments. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-10-03Do not show ".SRCINFO unchanged" warning on restoreLukas Fleischer
Since 612300b (Show a warning if .SRCINFO is unchanged, 2015-09-29), the git-update script displays a warning when a ref update does not affect the content of the package base meta data. We also invoke git-update to rebuild the package base details in the aurweb database when a package base is restored via the SSH interface. In that case, fake information is passed to the update hook: Both the old and the new object IDs refer to the current HEAD. Check for such "Everything up-to-date" updates and not display the ".SRCINFO unchanged" in these cases. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-09-29Show a warning if .SRCINFO is unchangedLukas Fleischer
Warn users when a remote ref update does not change the content of .SRCINFO such that users are reminded of updating package meta data. Implements FS#46130. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-09-16git-update: Use proper stop value for sliceLukas Fleischer
Fixes a regression introduced in 4112e57 (Add a restore command to the SSH interface, 2015-08-14). Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-16Add a restore command to the SSH interfaceLukas Fleischer
Implement a new command that can be used to restore deleted package bases without having to push a new commit. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-14git-update: Add commentsLukas Fleischer
Add some comments to explain the major steps performed in the update hook. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-14git-update: Remove superfluous assignmentLukas Fleischer
The pkgbase variable already contains the package base name at this point, no need to reassign it. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-14git-update: Move blacklist reading further downLukas Fleischer
Since c4870a9 (git-update: Only check HEAD for blacklisted packages, 2015-06-04), only the HEAD commit package name is looked up in the blacklist. This means that we no longer need to read the blacklist before running the commit walker. Moving the blacklist reading code further down makes the code easier to read. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-07-16git-update: Error out if PKGBUILD is missingLukas Fleischer
Fixes FS#45646. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-07-04git-update: Fix error when printing SRCINFO errorsJohannes Löthberg
This commit fixes a bug introduced by ae2907a (git: Use .format everywhere instead of %, 2015-06-27) where passing the error tuple to .format wasn't prefixed with an asterisk. Fixes FS#45545. Reported-by: Marty <vadmium+al@gmail.com> Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-27git-update: Improve error message on invalid pkgbaseLukas Fleischer
When the package base does not match the repository name, we currently show an error message that only mentions the package base name: error: invalid pkgbase: yate-qt4 To the end user this sounds like the package base name is malformed itself (and we show exactly the same error message when it contains invalid characters, indeed). Clarify that there is a name mismatch by also printing the expected value: error: invalid pkgbase: yate-qt4, expected: yate4-qt Inspired by FS#45483. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-27git-update: Allow privileged users to bypass blacklistJohannes Löthberg
Fixes FS#45410. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-27git: Use .format everywhere instead of %Johannes Löthberg
% formatting is deprecated, and .format should be used instead. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-20Make url and pkgdesc fields optionalJohannes Löthberg
The url and pkgdesc PKGBUILD variables are optional, so they should be in the AUR as well. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-20git-update: Remove regex matching of URLJohannes Löthberg
The regex check that a URL is either http or ftp doesn't really have any use, and it forces you to leave proper URLs for eg projects hosted on gopher empty. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-14git-update: Fix parsing of versioned optdependsLukas Fleischer
Fixes FS#43649. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-11git-update: Allow non-fast-forwards for TUsLukas Fleischer
Although we theoretically never want to lose history, there may be rare occasions when a forced push is required (e.g. if illegal data is pushed). Allow Trusted Users and Developers to perform non-fast-forward pushes. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-11git-update: Deny non-fast-forwardsLukas Fleischer
To make sure we never lose any history, non-fast-forwards are forbidden. Instead of relying on receive.denyNonFastForwards, add a simple check to the update hook. This has the added benefit of more flexibility. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-11Do not use symrefs for manually created Git refsLukas Fleischer
When creating the refs for the package base branch and the HEAD ref workaround, directly use commit IDs instead of symrefs. It looks like Git cannot pack symrefs, so this is expected to improve space efficiency. Also, since the refs are now only created (or updated) when new objects are actually received, this fixes a problem with dangling refs that were created when a user initialized a repository but did not push anything. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-08git-update: Avoid duplicate notificationsLukas Fleischer
Fixes FS#45254. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-08git-update: Drop writing git description fileJohannes Löthberg
All PKGBUILDs are stored in a single repository now, so there is no use for writing out a description file for the repo anymore. This fixes a bug where pkgdescs containing UTF-8 characters are rejected due to it trying to write the file out using ASCII. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-08git-update: Improve error message for non-blob objectsLukas Fleischer
When a repository contains a directory, a user is currently faced with the following error message: remote: Traceback (most recent call last): remote: File "hooks/update", line 194, in <module> remote: if repo[treeobj.id].size > 250000: remote: AttributeError: '_pygit2.Tree' object has no attribute 'size' remote: error: hook declined to update refs/heads/master Explicitly check for directories and other non-blob objects and display a more intuitive message. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-08Use extract_arch_fields when checking for source filesJohannes Löthberg
This fixes a bug where pushes are rejected in split PKGBUILDs if there are no arch-independent source array. Fixes FS#45253 Reported-by: Marti <marti@juffo.org> Reported-by: Maxime Gauduin <alucryd@archlinux.org> Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-08git-update: Increase maximum blob size to 250kBLukas Fleischer
Large blobs are needed in some packages, such as customized Linux kernels including kernel configurations. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-04git-update: Prevent from overwriting packagesLukas Fleischer
Make sure we do not overwrite a package belonging to another package base. We forgot to add this check to git-update when porting the package submission script to Python in commit 74edb6f (Use Git repositories to store packages, 2014-06-06). Reported-by: Johannes Löthberg <johannes@kyriasis.com> Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-04git-update: Only check HEAD for blacklisted packagesLukas Fleischer
When a part of a package base is moved to the official repositories, it needs to be dropped from the AUR package. However, we want to allow that the moved packages still appear in the history. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-04git-update: Only check HEAD for matching pkgbaseLukas Fleischer
When renaming a package base, we want to keep commits of the former package. Move the check that compares the package base name with the repository name such that only HEAD is validated. Fixes FS#45065. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-02Restructure scriptsLukas Fleischer
* Move scripts/git-integration/ to git-interface/. * Move scripts/aurblup/aurblup.py to scripts/aurblup.py. Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>