librefetch(8) -- downloads or creates a liberated source tarball ================================================================ ## SYNOPSIS `librefetch` [] []
`librefetch` `-`[`g`|`S`|`M`|`h`] ## DESCRIPTION `librefetch` is a program to streamline creation of custom source tarballs for `PKGBUILD(5)` files. If a URL mentioned in the array in a `PKGBUILD` is in a location that Parabola uploads "custom" source tarballs to (or configured locations), and no file is at that URL, librefetch will automatically create it for you. This works because a post-install script for the package configures `librefetch` as the download agent for `https://` URLs in `makepkg.conf`; allowing it to jump in and create a file if need be. Because of this, it is almost never necessary to call `librefetch` manually. The post-install script also configures `librefetch` as the download agent for `libre://` URLs, for compatibility with `PKGBUILD` files that used a previous version of librefetch. There are 5 modes: * `download`: Download the tarball from the configured mirror. * `create`: Create the tarball from a `PKGBUILD`/`SRCBUILD`. * `checksums`: Generate integrity checks for source files. * `srcbuild`: Print the effective build script. * `makepkg`: Generate and print the location of the effective makepkg script. * `help`: Print `librefetch` usage information. The normal mode of operation is `download` mode. If `download` mode fails, it may choose to try `create` mode. ## OPTIONS * `-C`: Force `create` mode (don't download) * `-D`: Force `download` mode (don't create) * `-p` : Use an alternate build script for `create` mode (instead of `PKGBUILD`). If an `SRCBUILD` file exists in the same directory, it is used instead. * `-g` | `--geninteg`: Use `checksums` mode: Generate integrity checks for source files. * `-S` | `--srcbuild`: Use `srcbuild` mode: print the effective build script. * `-M` | `--makepkg`: Use `makepkg` mode: generate and print the location of the effective makepkg script. * `-h` | `--help`: Use `help` mode: Show usage information. Other options, if they are documented in `makepkg -h`, are passed to the modified copy of makepkg created during `create` mode. ## DOWNLOAD MODE If begins with the string `libre://`, it is replaced with the first value in , as configured in `librefetch.conf(5)`; this is for compatibility with `PKGBUILD` files that used a previous version of librefetch. It uses , as configured in `librefetch.conf` to attempt to download the source tarball from that URL. If that fails, and following conditions are met, it proceeds to `create` mode: * The `-D` flag has not been specified to inhibit `create` mode. * The begins with one of the values in . The latter requirement allows librefetch to be used as a generic HTTP(S) download agent, that can automatically create files from whitelisted locations. ## CREATE MODE The principle of `create` mode is that a special `PKGBUILD(5)` (called `SRCBUILD(5)`) installs source files to <$pkgdir>, and the resulting "package" is then used as a source tarball. The `SRCBUILD` exists in the same directory as the `PKGBUILD`. It can be created manually, or generated on-the-fly from the `PKGBUILD`. Extra steps are taken to ensure that as long as the same directory contents go in, an identical tarball will come out--the checksum of the file should not change based on when it is built or who builds it. The `SRCBUILD` is either created, or sanitized if it already exists. If the output filename does not end with `.sig` or `.sig.part`, then the `SRCBUILD` is fed to a modified version of `makepkg(8)`. If the output filename does end with `.sig` or `.sig.part`, then it uses GPG to create a signature. If the file it is trying to sign does not exist yet, librefetch recurses on itself to create it. The reason `makepkg` must be modified is that we need the resulting tarball to be deterministic (as well as not containing package metadata). When this documentation speaks of a file being modified, it is a temporary copy of the file that is modified, your original file will remain intact. ## SRCBUILD GENERATION As explained in the `CREATE MODE` section, in `create` mode, this program generates an `SRCBUILD` file. For debugging purposes, this file can be printed instead of executed with `srcbuild` mode. ### PRE-EXISTING SRCBUILD The use of `SRCBUILD` files pre-dates `librefetch`. By convention, they set and in `package()` in order to modify the behavior of `makepkg`. Because a modified version of `makepkg` is used, this interferes with the correct behavior. To compensate for this, lines containing "`PKGDEST=`" or "`PKGEXT=`" are deleted from the `SRCBUILD`. The general idea is that `build()` makes any modifications to the source, then `package()` copies it from <$srcdir> to <$pkgdir>. ### SRCBUILD FROM PKGBUILD Possibly more elegant than having a separate `SRCBUILD` file is having an `mksource()` function in the main `PKGBUILD`. This results in less boilerplate and fewer files to edit. Note that this only happens if a file named `SRCBUILD` doesn't already exist; when migrating a package from a manually created `SRCBUILD` to this method, the `SRCBUILD` must be deleted (or renamed) for this to work. The dynamically created `SRCBUILD` is created by copying `PKGBUILD` to a temporary file, then re-setting variables and re-defining functions. Following is a table of the translations. Variables source = mksource noextract = mknoextract *sums = mk*sums (md5, sha1, sha224, sha256, sha384, sha512) depends = checkdepends = makedepends = mkdepends *_$CARCH = Functions prepare() { :; } build() { mksource; } check() { :; } package() { cp -a "$srcdir"/*/ "$pkgdir/"; } The `mksource()` function does not need to be defined. If it isn't defined, then no transformations will be made to the source between it being extracted to <$srcdir> and copied to <$pkgdir>. In summary: * Set and to act as and , respectively. * Declare a `mksource()` function to make modifications to the source, if necessary. Other changes: * is set to , or the first element of the array (the effect is that split packaging is turned off). * is set have `makepkg` avoid making changes to <$pkgdir>. The exact change is: options=(!strip docs libtool staticlibs emptydirs !zipman purge !upx) * has VCS directories added to it: PURGE_TARGETS=(.bzr/ .cvs/ .git/ .hg/ .svn/ .makepkg/) ### MAKEPKG MODIFICATIONS The following modifications are made to makepkg: * Allow us to manipulate the output file (<$pkg_file>) * Do not include metadata in the output file (<${comp_files[@]}>) * Force 'ustar' tar format, don't allow it to upgrade to 'pax' to store extended file attributes. * Don't symlink the resulting file into the current directory. * interprets an item as a directory if it ends with a slash ("/"). * Timestamps in <$pkgdir> are reset to the date specified in [1], or "1990-01-01 0:0:0 +0" if it's not set, so that the resulting tarball will be the same, regardless of when it was created. * Sort the files included in the tarball; normally the order of files in a tarball is essentially random (even if it tends to be the same when re-created on the same machine). * append `-libre` to <$srcdir> * append `-libre` to <$pkgbasedir> (which becomes <$pkgdir>) * Don't check if the package has already been built. For debugging purposes, this modified makepkg can be printed instead of executed with `makepkg` mode. Before it is run in create mode, `PKGEXT`, `PKGDEST`, and `pkg_file` are set as environment variables. ## CONFIGURATION See `librefetch.conf(5)` for details on configuring librefetch using the `librefetch.conf` file. ## SEE ALSO librefetch.conf(5), makepkg(8), PKGBUILD(5), SRCBUILD(5) ## NOTES 1. specification for build systems https://reproducible-builds.org/specs/source-date-epoch/