[archiso] Use dm-snapshot instead of aufs2 (A.K.A. "The Big Commit")

* Use device mapper + snapshot module, instead union layer filesystem.
  * A block-level approach vs vfs-level.
  * No more unofficial (Linux) things.
  * More memory is needed.
* Refactor mkarchiso.
* Refactor hooks/archiso.
* Fix install/archiso_pxe_nbd
  (due recent change in mkinitcpio-0.6.15 on checked_modules()/all_modules())
  [Thanks Dave for the improved workaround]
* New configs/releng to build official images.
  * Works with a Bash script instead of Makefile.
    (better control and easy to maintain)
* Remove configs/syslinux-iso.
* Remove archiso2dual script. Integrate functionality in configs/releng.
* New configs/baseline to build the most basic live medium or use as template.
* New README (draft). [Thanks Dieter for fixing english grammar]

Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar>

1 files changed, 156 insertions, 117 deletions

diff --git a/README b/README
index e3001be..dee3a50 100644
--- a/README
+++ b/README
@@ -1,117 +1,156 @@
-Archiso For Dummies Like Me and You
--------------------------------------
-
-
-- What the hell is Archiso?
-
-Archiso is a small set of bash scripts that is capable of building fully
-functional Arch Linux based live CDs. It is a very generic tool, so it
-could potentially be used to generate anything from rescue systems,
-to install disks, to special interest live CD systems, and who knows what
-else. Simply put, if it involves Arch on a shiny coaster, it can do it.
-
-
-- Alright, so how does one install it?
-
-First off, Archiso has some dependencies:
-    - mkinitcpio
-    - cdrkit
-    - squashfs-tools
-    - aufs2 (only needed in target media)
-    - aufs2-util (only needed in target media)
-    - devtools for mkarchroot
-    - syslinux
-    - nbd
-    - mkinitcpio-nfs-utils
-
-Archiso itself can be installed with the handy dandy included Makefile,
-and the incantation 'make install'.
-
-
-- Great, so how do you use this thing?
-
-The heart and soul of Archiso is mkarchiso. All of its options are
-documented in its usage output, so we won't go into detail here.
-Instead, let's go over the general process.
-
-The first thing you should probably do is create a directory to work
-in, and cd to it. This'll help keep things organized. Next, you'll want
-to create a mkinitcpio config file that suits your needs. Typically this
-means modifying whatever hooks you want. A typical set of hooks for
-archiso looks something like this:
-
-HOOKS="base udev archiso archiso_pxe_nbd pata scsi sata usb fw pcmcia filesystems usbinput"
-
-It's probably worth mentioning that hardware autodetection and things
-of that nature do not belong here. Only what's necessary to get the system
-on its feet, and out of the initcpio really belong here, fancier stuff
-can be done on the booted system anyway.
-
-You'll also want to create a list of packages you want installed on your
-live CD system. A file full of package names, one-per-line, is the format
-for this. Typically you'll want BASE and a kernel as a bare minimum, but
-you're free to install whatever else you want. This is *great* for
-special interest live CDs, just specify packages you want and gogogo.
-
-The last item of importance is what are called addons. Basically this
-means any other crap you might want to include on your live CD, including
-binary package repos, special configurations, random files, we don't
-know, be creative. mkarchiso expects them all to be put in a single
-directory, with an fstab-like config file. Currently two types of addons
-are supported, squashfs images that get layered onto the root union, and
-plain directories which can be bind mounted anywhere under the root.
-
-If you want to add a squashfs union layer:
-- Set up whatever you want to include in a separate directory someplace,
-  as if that directory was / . Then run mksquahfs on it, and copy the
-  created image to your addons directory.
-- Add an entry to your addons config file (which must be named 'config',
-  by the way). Typical squashfs entries look like this:
-  live_overlay.sqfs       /                   squashfs
-  Where the first component is the path to the image relative to your
-  addons directory, the second is the mountpoint (irrelevant for
-  squashfs, they will all get layered at /) and of course the third
-  component is the type.
-- Be aware that the order of entries on the config matters! Entries will
-  be layered on top of one another, later entries are mounted _UNDER_
-  earlier entries (an unfortunate counterintuitive result of the way we
-  have to mount the unions).
-
-If you want to add plain directories to bind mount:
-- Set up your directory somewhere, and copy it to your addon directory.
-- Add an entry to your addons config file, example entry:
-  core                    /packages           bind
-  where the first component is the path to the directory relative to
-  your addons directory, the second component is where you'd like it
-  bind-mounted relative to the live CD's root filesystem, and the last
-  component is the type.
-
-
-- How can build installation mediums like provided by Arch Linux?
-
-- Just follow these next steps as root.
-- Note that mkarchroot is optional, but with it, will ensure to have
-  a clean enviroment for building isos.
-- This clean chroot, will take about 400MB (+130MB with all needed tools).
-- After make, max space usage is about 2GB.
-- In last step instead of just execute make, can be more selective:
-  Execute "make net-iso" or make "core-iso".
-  Do not execute make net-iso after make core-iso, otherwise net-iso
-  will be really a core-iso.
-
-
-pacman -S devtools --needed
-mkarchroot /tmp/somedir base
-mkarchroot -r bash /tmp/somedir
-# vi/nano /etc/pacman.d/mirrorlist and uncomment your prefered mirror.
-pacman -S git squashfs-tools syslinux devtools cdrkit make nbd mkinitcpio-nfs-utils
-cd /tmp
-git clone git://projects.archlinux.org/archiso.git
-cd archiso/archiso
-make install
-cd ../configs/syslinux-iso/
-make
-
-Done!
-
-vim: textwidth=72
+INDEX
+-----
+
+* Image types generated by mkarchiso.
+* File format for aitab.
+* Why the /isolinux and /arch/boot/syslinux directories?
+* Building the most basic Arch Linux live media. (configs/baseline)
+* Building official Arch Linux live media. (configs/releng)
+
+
+
+*** Image types generated by mkarchiso.
+
+* image-name.sfs     SquashFS image with all files directly on it.
+* image-name.fs.sfs  SquashFS with only one file inside (image-name.fs),
+                     which is an image of some type of filesystem
+                     (ext4, ext3, ext2, xfs), all files reside on it.
+* image-name.fs      Like image-name.fs.sfs but without SquashFS.
+                     (For testing purposes only. The option copytoram
+                     for archiso hook does not have any effect on these images)
+
+
+
+*** File format for aitab.
+
+The aitab file holds information about the filesystems images that must be
+created by mkarchiso and mounted at initramfs stage from the archiso hook.
+It consists of some fields which define the behaviour of images.
+
+# <img>         <mnt>                 <arch>   <sfs_comp>  <fs_type>  <fs_size>
+
+<img>      Image name without extension (.fs .fs.sfs .sfs).
+<mnt>      Mount point.
+<arch>     Architecture { i686 | x86_64 | any }.
+<sfs_comp> SquashFS compression type { gzip | lzo | xz }.
+           A special value of "none" denotes no usage of SquashFS.
+<fs_type>  Set the filesystem type of the image { ext4 | ext3 | ext2 | xfs }.
+           A special value of "none" denotes no usage of a filesystem.
+           In that case all files are pushed directly to SquashFS filesystem.
+<fs_size>  An absolute value of file system image size in MiB.
+           (example: 100, 1000, 4096, etc)
+           A relative value of file system free space [in percent].
+           {1%..99%} (example 50%, 10%, 7%).
+           This is an estimation, and calculated in a simple way.
+           Space used + 10% (estimated for metadata overhead) + desired %
+
+Note: Some combinations are invalid, example: sfs_comp=none and fs_type=none
+
+
+
+*** Why the /isolinux and /arch/boot/syslinux directories?
+
+The /isolinux directory holds files needed for the ISOLINUX boot loader
+module of SYSLINUX. ISOLINUX can not find config files on
+/arch/boot/syslinux, like other boot loaders modules (EXTLINUX, SYSLINUX, etc).
+When make your custom boot-pendrive, you need to copy /arch directory to it.
+/isolinux/isolinux.cfg just holds /arch/boot/syslinux/syslinux.cfg
+
+
+
+*** Building the most basic Arch Linux live media. (configs/baseline)
+
+* First install devtools if needed, mkarchroot needs it.
+  [host] # pacman -S devtools
+
+* Create a chroot to work on it.
+  (prefix with linux32 if you want to build a 32 bits enviroment under 64 bits)
+  [host] # mkarchroot /tmp/chroot base
+
+* Enter it. (prefix with linux32 if needed).
+  [host] # mkarchroot -r bash /tmp/chroot
+
+* Create a loopback device.
+  (mkarchiso does not create it, use other number if not available)
+  [chroot] # mknod /dev/loop0 b 7 0
+
+* Setup a mirror.
+  [chroot] # echo 'Server = MIRROR/archlinux/$repo/os/$arch' >> /etc/pacman.d/mirrorlist
+
+* Install aditional packages needed for mkarchiso.
+  (git is only needed to get a copy of archiso.git)
+  [chroot] # pacman -S git squashfs-tools syslinux devtools cdrkit make
+
+* Install archiso.
+  [chroot] # cd /tmp
+  [chroot] # git clone git://projects.archlinux.org/archiso.git
+  [chroot] # cd archiso/archiso
+  [chroot] # make install-program
+
+* Build a basic iso.
+  [chroot] # cd /tmp/archiso/configs/baseline
+  [chroot] # ./build.sh
+
+* Exit from chroot.
+  [chroot] # exit
+
+Note: If you want to customize, just see the configs/releng directory which is
+used to build official images with much more things.
+
+
+
+*** Building official Arch Linux live media. (configs/releng)
+
+Note: These steps should be done with 64 bits support.
+
+* Prepare a 32 bit chroot enviroment.
+
+linux32 mkarchroot /tmp/chroot32 base
+linux32 mkarchroot -r bash /tmp/chroot32
+echo 'Server = MIRROR/archlinux/$repo/os/$arch' >> /etc/pacman.d/mirrorlist
+pacman -S squashfs-tools syslinux devtools cdrkit make mkinitcpio-nfs-utils nbd --noconfirm --needed
+exit
+
+* Prepare a 64 bits chroot enviroment.
+
+mkarchroot /tmp/chroot64 base
+mkarchroot -r bash /tmp/chroot64
+echo 'Server = MIRROR/archlinux/$repo/os/$arch' >> /etc/pacman.d/mirrorlist
+pacman -S squashfs-tools syslinux devtools cdrkit make mkinitcpio-nfs-utils nbd --noconfirm --needed
+exit
+
+* Create a shared directory which archiso can access from both chroot enviroments.
+
+mkdir /tmp/shared
+cd /tmp/shared
+git clone git://github.com/djgera/archiso.git -b dm-snapshot
+cd
+mount --bind /tmp/shared /tmp/chroot32/tmp
+mount --bind /tmp/shared /tmp/chroot64/tmp
+
+* Enter 32 bits chroot enviroment, install mkarchiso,
+  then build core and netinstall single images.
+
+linux32 mkarchroot -r bash /tmp/chroot32
+mknod /dev/loop1032 b 7 1032
+cd /tmp/archiso/archiso
+make install-program
+cd ../configs/releng/
+./build.sh all_iso_single
+
+* Enter 64 bits chroot enviroment, install mkarchiso,
+  then build core and netinstall single images.
+
+mkarchroot -r bash /tmp/chroot64
+mknod /dev/loop1064 b 7 1064
+cd /tmp/archiso/archiso
+make install-program
+cd ../configs/releng/
+./build.sh all_iso_single
+
+* Finally build core and netinstall dual images
+  from any of the chroot enviroments. (The final result is the same).
+
+mkarchroot -r bash /tmp/chroot64
+cd /tmp/archiso/configs/releng/
+./build.sh all_iso_dual