summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
Diffstat (limited to 'doc')
-rw-r--r--doc/Makefile2
-rw-r--r--doc/createworkdir62
-rw-r--r--doc/fullpkg7
-rw-r--r--doc/treepkg.markdown130
-rw-r--r--doc/workflows.markdown60
5 files changed, 261 insertions, 0 deletions
diff --git a/doc/Makefile b/doc/Makefile
new file mode 100644
index 0000000..7af3750
--- /dev/null
+++ b/doc/Makefile
@@ -0,0 +1,2 @@
+libre_datadir=$(docdir)/libretools
+include ../common.mk
diff --git a/doc/createworkdir b/doc/createworkdir
new file mode 100644
index 0000000..d74198a
--- /dev/null
+++ b/doc/createworkdir
@@ -0,0 +1,62 @@
+== CreateWorkDir
+
+This script recreates a proper dir tree for packaging. It's aim is to help
+you be organized with the work you do as a packager, and stablish a
+certain standard for packages' ubication, so you don't have to loose much
+time with them. Just package and upload!
+
+It will create a dir tree like this:
+workdir/
+ abslibre/
+ .git/
+ libre/<PKGBUILDS>
+ libre-testing/<PKGBUILDS>
+ staging/
+ libre/
+ libre-testing/
+ repos/
+ libre/
+ i686/
+ x86_64/
+ libre-testing/
+ i686/
+ x86_64/
+
+*Related Variables*
+ - WORKDIR
+
+
+=== staging/
+
+This directory contains one dir for each repository, where the resulting
+packages are in moved for syncing against the main repository on
+Parabola's server. This dir is architecture independent.
+
+*Related Variables*
+ - REPOS
+
+
+=== abslibre/
+
+This is the git repo for Parabola's PKGBUILDs. Here you can find the ABS
+tree for our packages, and also where you'll have to put new ones for
+commit.
+
+(You'll need push access to Parabola's main server, but pulling is
+public.)
+
+*Related Variables*
+ - ABSLIBREGIT
+
+
+=== repos/
+
+Contains the repo dir structure. Probably not useful for uploading
+packages, but it will be for creating new repos for public access.
+
+It contains a dir for every repository, and inside them, a dir for every
+supported architecture.
+
+*Related Variables*
+ - REPOS
+ - ARCHES
diff --git a/doc/fullpkg b/doc/fullpkg
new file mode 100644
index 0000000..5645fae
--- /dev/null
+++ b/doc/fullpkg
@@ -0,0 +1,7 @@
+# FullPKG
+FullPKG is a tool to build a package from ABS, find all needed dependencies
+along the way and build them too if necessary.
+
+## Before running fullpkg
+Update PKGBUILD path cache by running `toru -u <repo>` on your ABS root. Once
+for each repo you'll be using.
diff --git a/doc/treepkg.markdown b/doc/treepkg.markdown
new file mode 100644
index 0000000..7f7ece1
--- /dev/null
+++ b/doc/treepkg.markdown
@@ -0,0 +1,130 @@
+# TreePKG
+
+`treepkg` is a tool for recursively building packages from an ABS tree. It has
+been tested while building packages for the mips64el port and it has proven to
+be useful.
+
+It was written having in mind the experience of `fullpkg`, which converted to
+`fullpkg-ng` and then splitted into `fullpkg-build` and `fullpkg-find`. It's
+aim is to simplify algorithms implemented on the fullpkg family, while solving
+some design issues that made fullpkg miss some packages sometimes.
+
+
+## Requirements
+
+`treepkg` needs the help of `toru-path` for "indexing" an ABS tree. `toru-path`
+stores a tokyocabinet database of "pkgname" => "path" pairs, where pkgname is
+replaced by the "pkgbase", "pkgname", and "provides" fields of a PKGBUILD,
+followed by the path of the current PKGBUILD.
+
+This information is then used by `treepkg` to know where to find the PKGBUILD
+of a package. The fullpkg family needed to guess this by traversing the full
+ABS tree, and therefore it was unable to find pkgnames that differ from
+pkgbase.
+
+So, to use `treepkg` you need to run `toru-path` after the ABS tree update.
+
+> Split PKGBUILDs make it difficult to extract metadata if it's stored inside
+> package() functions. This will happen with the provides field and `treepkg`
+> won't find that linux-libre-headers provides linux-headers, for instance.
+
+## How does it work
+
+`treepkg` must be run from the path of the PKGBUILD you're going to build (this
+may change over time). This will be DEPTH=0 and it will create a temporary
+directory to work on in the format /tmp/pkgbase-treepkg-random-string. Inside
+this directory, it will copy the needed PKGBUILDs prefixed with their depth
+number. The first package will always be copied to 000\_pkgbase.
+
+From then on, treepkg sources the PKGBUILD and runs itself over all pkgnames on
+the depends and makedepends array, only if it detects their versions aren't
+already built or deprecated by newer ones, using the `is_built` utility.
+
+While processing this info, it will increase the depth of the packages. It'll
+also write a CSV file with the knowledge it acquires from the ABS tree (useful
+for debugging). This file is called BUILDORDER.
+
+When this process ends (no more needed dependencies are found), the temporary
+work dir is traversed in inverse order (from higher depths to 0) running the
+FULLBUILDCMD from libretools.conf and then the HOOKLOCALRELEASE variable, which
+*must* provide a way to `repo-add` the packages to an available repository, so
+the next build will find and install the newer version using pacman.
+
+For instance, having this as the first pacman repository (on /etc/pacman.conf):
+
+ [stage3]
+ Server = /var/cache/pacman/pkg
+
+And this on /etc/makepkg.conf:
+
+ PKGDEST=/var/cache/pacman/pkg
+
+Your HOOKLOCALRELEASE script should look like this:
+
+ # Get needed vars
+ source /etc/makepkg.conf
+ source /etc/libretools.conf
+ source PKGBUILD
+
+ unset build package check
+
+ fullver=$(full_version ${epoch:-0} ${pkgver} ${pkgrel})
+ pkgs=()
+
+ # Generate canonical package paths
+ msg "Adding packages to [stage3]..."
+ for name in ${pkgname[@]}; do
+ msg2 "${name} ${fullver}"
+ pkgs+=("${PKGDEST}/${name}-${fullver}-*.pkg.tar.*")
+ done
+
+ # Add the packages to a local
+ repo-add ${PKGDEST}/stage3.db.tar.gz ${pkgs[@]}
+
+ # Stage the packages for later releasing
+ librestage $1
+
+> Note the first HOOKLOCALRELEASE argument is the remote repository name (core,
+> extra, etc.)
+
+There's a special case when a dependency depends on another that was put on
+a depth level lower than itself. In this case the build order will be wrongly
+assumed and you may end up with broken packages.
+
+To explain it with an example:
+
+ ghostscript (0) - fontconfig (1)
+ \ cups (1) - fontconfig (ignored)
+
+The second time fontconfig appears, it will be ignored. In this case cups will
+build against an fontconfig version that will be outdated by the fontconfig
+version at depth 1. In this cases, `treepkg` will detect it cached the
+dependency on a lower depth, and will "bury" it to a depth higher than the
+current one. Thus this will become the build path:
+
+ ghostscript (0) - fontconfig (buried)
+ \ cups (1) - fontconfig (2)
+
+> Note: currently, `treepkg` doesn't perform recursive burying, so if you hit
+> a really long build tree with some circular dependencies you may find
+> packages buried several times and queued to build before their actuals deps.
+
+## Tips
+
+`treepkg` accepts two arguments: 1) the temporary work dir and 2) the next
+depth level it should use (if current equals 0, it'll pass 1). You don't need
+to pass this arguments when running it manually, they're used internally to
+automatically construct the build path.
+
+But if a build failed, `treepkg` will cancel itself immediately informing you
+where the leftovers files were left. If you pass this path to `treepkg` as the
+first argument, it will resume the build, skipping to the last package being
+packaged.
+
+You can take the opportunity given by this to modify the build path or the
+PKGBUILDs, without having to run `treepkg` twice. For instance you can remove
+a package from the build order, or move it manually, or update the PKGBUILD
+that made `treepkg` fail in the first place. You can force a skipped package
+(after building it manually) by using `touch built_ok` on the PKGBUILD dir.
+
+You don't probably want to mess with the second argument though.
diff --git a/doc/workflows.markdown b/doc/workflows.markdown
new file mode 100644
index 0000000..f55ae7e
--- /dev/null
+++ b/doc/workflows.markdown
@@ -0,0 +1,60 @@
+# Workflows
+
+Describe your packaging workflow here!
+
+
+## fauno's way
+
+During packaging, I don't usually restart a build from scratch if I have to
+make changes to the PKGBUILD. I use a lot of commenting out commands already
+ran, `makepkg -R`, etc. When I used `libremakepkg` I ended up using a lot more
+`librechroot` and working from inside the unconfigured chroot, because
+`makechrootpkg` (the underlying technology for `libremakepkg`) tries to be too
+smart.
+
+When I started writing `treepkg` I found that mounting what I need directly on
+the chroot and working from inside it was much more comfortable and simple than
+having a makepkg wrapper doing funny stuff (for instance, mangling makepkg.conf
+and breaking everything.)
+
+This is how the chroot is configured:
+
+* Create the same user (with same uid) on the chroot that the one I use regularly.
+
+* Give it password-less sudo on the chroot.
+
+* Bind mount /home to /chroot/home, where I have the abslibre-mips64el clone.
+
+* Bind mount /var/cache/pacman/pkg to /chroot/var/cache/pacman/pkg
+
+* Put these on system's fstab so I don't have to do it everytime
+
+* Configure makepkg.conf to PKGDEST=CacheDir and SRCDEST to something on my home.
+
+Workflow:
+
+* Enter the chroot with `systemd-nspawn -D/chroot` and `su - fauno`.
+
+* From another shell (I use tmux) edit the abslibre or search for updates with
+ `git log --no-merges --numstat`.
+
+* Pick a package and run `treepkg` from its dir on the chroot, or retake
+ a build with `treepkg /tmp/package-treepkg-xxxx`. (Refer to doc/treepkg
+ here).
+
+What this allows:
+
+* Not having to worry about the state of the chroot. `chcleanup` removes and
+ adds packages in a smart way so shared dependencies stay and others move
+ along (think of installing and removing qt for a complete kde rebuild).
+
+* Building many packages in a row without recreating a chroot for every one of
+ them.
+
+* Knowing that any change you made to the chroot stays as you want (no one
+ touches your makepkg.conf)
+
+* Hability to run regular commands, not through a chroot wrapper. I can `cd` to
+ a dir and use `makepkg -whatever` on it and nothing breaks.
+
+* No extra code spent on wrappers.