ltsBlog

Introduction

This is the first installment in a series of articles introducing one to programming and hacking in general.

One pitfall that comes with teaching users unfamiliar with a shell to program is that they often struggle with concepts such as arguments and Standard In/Out. So, at Nathan’s suggestion have decided to begin with shell scripting.

The “shell” is the program that you use to interact with the operating system and launch programs. Many users are familiar with graphical shells, such as explorer.exe in MS Windows NT, X in *NIX, or Quartz on Mac OS X. However, “shell” is usually used to mean a command-line shell. I will try to teach about shells in the Bourne family, but will focus on GNU BASH. Bourne shells are nice in the fact that they are both a well-designed language, and a complete user interface.

Acquiring BASH

Almost any modern UNIX-like system will include BASH. If you already have a Mac OS X, BSD, or GNU/Linux machine then you probably already have BASH. If you use MS Windows, I would recommend erasing it and installing a Free UNIX-like system. However, since this may not be an option, you can install a program called Cygwin that makes Windows like a *NIX system.

If you use Cygwin, be sure to have it install BASH, and `bc’, `sed’, and `grep’, and nano (at least) (I don’t think it installs bc or nano by default).

• To access BASH on Mac OS X, launch the “Terminal” app.
• To access BASH on GNU/Linux, launch `xterm’ or another terminal emulator; or hit <ctrl>+<alt>+F1 for a full-screen terminal.
• To access BASH on Cygwin there should be a program in the “All Programs” menu from the Start menu.

Your first Shell script

Open a BASH session. You should see a “$ ” prompt, possibly with other text such as username and the current directory (folder). Because of this when you see a command like “$ cp foo foo.bak”, the “$ ” only indicates that it is a shell command; you don’t actually type the dollar sign.

Use the $ nano script1.sh command to create and edit the file “script1.sh” with the nano text editor.

In my examples I use nano, but feel free to use another text editor. I chose GNU nano because it is new-user friendly, comes pre-installed on Mac OS X, most modern *NIXen, and is easily installed with Cygwin. While there are many better new-user friendly text editors, such as Notepad++ and gedit, they are not the same across all systems. Additionally, there are many more sophisticated editors such as Emacs or VI that are awesome, but take some time to learn. I personally use GNU Emacs.

For the first line of the file, type:

#!/bin/bash

This tells the operating system how to run text files as programs. Most programs are binary files that the computer can run directly, and humans can’t really read. However, text scripts that reverse this; we can read them, but the computer needs help. In this case, it checks to see if the file begins with a shebang (hash[#]-bang[!]), and if it does, uses the binary program file on the line after it to read the program. Since we are writing a BASH script, we list “/bin/bash”, which is where BASH is usually installed.

Now, if you are even slightly familiar with the command line, you know that a command-line shell primarily takes a simple list of commands, in order. Writing a shell script like this is exactly like typing it directly at the command-line. Well, actually, there are 2 differences:

1. The shell won’t be interactive, it won’t print the prompt before each command.
2. The shell will exit when it’s done, instead of waiting for another command.

Now, the first program anyone ever learns to write is “hello world”, so:

#!/bin/bash
echo 'Hello, world!'

Once you have entered this into your editor, save and return to your interactive shell. In nano, do this by entering <ctrl>+o to save, and <ctrl>+x to exit back to the shell. Now, in order to run the script as a program, we must let the computer know that it is a program, tell the computer that it is “executable”. Do this by running the command “$ chmod +x script1.sh” (change mode +executable on the file “script1.sh”).

Now, actually running your program. I could simply tell you run “$ ./script1.sh”, but I’ll instead take the time to explain why this is how you run the program, and why you need “./” for it, and you don’t for most programs. There are two ways to tell BASH what program to run:

1. Give just the command/file name, and check the PATH and shell built-ins for the command.
2. Give the path to the specific file.

The first way is by far the most common, let me explain how it works. Take for example the “echo” command we used. Since it does not contain a “/” it must be a filename, rather than a path. BASH can be first checks it’s list of built-in commands, and uses built-in echo. However, you can configure BASH to disable most built-in commands, in which case it would not find the built-in echo, and begin to look in the PATH. PATH is a special environmental variable that tells shells where to look for programs. To see what your PATH is, run “$ echo $PATH”. For example, my current PATH is:

/home/luke/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Now, path is a list of directory/folder locations, separated by colons. The shell will look in the first folder for a file with the name of the command. So, on my system, BASH will look for the file “/home/luke/bin/echo”, which doesn’t exist, so it will go on and look in the next folder, until it eventually finds “/bin/echo”, when it will run that file. If the shell can’t find a command with that name, it will display an error message. The second way, we give the path directly to the command we want to run, the folder it’s in, and the file in it. So, if I didn’t want to re-configure BASH, but wanted to run the non-built-in version of echo, I could type “$ /bin/echo 'Hello, world!'”. BASH knows that this is a complete path because it does contain a “/”. Now, you can either give the an absolute path, or a relative path. An absolute path gives is like saying “at this address”, where a relative path is like saying “1 mile east from here”. Now, on a UNIX-like operating system, the file-system begins with “/” as the root folder, all other folders go inside of it, so if the path begins with a “/”, then it is an absolute path. If the path does not begin with a slash, then the first item listed is directly inside of the current directory. So, since our program is directly in the same directory as we are, so it would just be “script1.sh”, right? Well, then bash doesn’t know that it’s a path, rather than a command, so how do we add a slash, but not have to list exactly where we currently are? The simple answer is that on UNIX-like systems all directories always have at least 2 sub-directories, “.” and “..”, both with special meanings. “.” is the same folder that it’s in, so “/home/luke/././././” is the same as just “/home/luke/”. “..” is the parent directory, the directory that this one is inside of; so “/home/luke/../” is the same as “/home/”. So, when we want to say that something is directly in the current directory, and we need a slash, we can use “./script1.sh”, which is the same location as “script1.sh”, but BASH knows that it’s a path.

Variables

TODO

Standard Out

TODO

Standard In

TODO

Flow control

TODO

Afterward/Other resources

As will be norm, if you wish to become proficient in a language, you must do more than just read my Lessons, you must find other resources, but more importantly, do something, find hands-on experience. Just this weekend, I wrote a system monitor (a program to display things like processor use, memory consumption, battery level, etc.) in BASH, and even though I already considered myself proficient with both my operating system, and BASH, in doing so learned several things about both (namely, BASH traps).

Anyway, about other resources: I often have trouble recommending resources to people, since I feel that there are so few good resources available, and that 90% of the resources available are crap. However, I feel confident recommending the book Advanced Bash-Scripting Guide, available online at The LDP in many formats (to view in your web browser, go for the HTML version). One page that is especially useful, even to experienced coders is its Reference Cards page. While the book is a great resource, there are several minor issues with it that I need to address before I feel comfortable recommending it:

• It frequently says “Linux” when it means “GNU/Linux”, which is especially unfortunate because there are times when it really does mean just “Linux”.
• It frequently uses the word “hack” when it means “crack”.
• Don’t believe it 100% when it says something about defaults, or that some operating system does something. For example, it claims several times that all GNU/Linux (well, it says just “Linux”) systems that BASH is the default shell. This is generally true, many do, but several do not, namely Ubuntu, for which the default is DASH.

Less specifically, O’Reilly Media is well-known among hackers for providing quality resources, often written by hackers themselves. Especially the books with an animal on the front, those are good stuff.