Unix & Bash Tutorial

In this tutorial you will be introduced to the Unix filesystem and bash. We will cover navigating the file system, viewing and editing files, combining bash commands, searching for files, and file permissions. Finally, you will write your first bash script!

1 - Introduction to the Unix File System

1.1 - Navigating the File System

1.1.1 - Windows

On Windows, you’re used to opening the File Explorer and clicking on folders until you find the file you’re looking for. For example, let’s take a look at an example Documents folder:

We see a folder (also called a directory) inside of Documents: C:\User\bsmits\Documents\example_folder\.

Inside Documents/example_folder/ we see a file named example_file.txt and another nested directory example_nested_folder\.

If we go one level deeper, we see example_file2.txt.

We can represent this directory structure visually like this:

C:\
├── Users\
├──── bsmits\
├────── Documents\
├──────── example_folder\
├────────── example_nested_folder\
├──────────── example_file2.txt
├────────── example_file.txt

Every file and directory has a path that tells you exactly where that file/directory is located. Paths can be relative or absolute, which we’ll talk about soon.

1.1.2 - Linux

In Unix (or Linux), files, directories, and applications work the same way on the command line as they do on a Windows/Mac computer, but instead of using the mouse to click, find, and open them, you type commands into the terminal that tell the computer what to do.

A terminal gives you access to the command line. If you open a terminal, it will look similar to this:

What you see in your terminal is the command prompt which tells you some useful information. The characters before the @ symbol are your username. In this example, my username is bsmits. The characters after the @ symbol are your hostname, the name of your computer. In this example, my hostname is sporcbuild.

After the hostname, you will see your current working directory, the directory you are executing commands inside of. The tilda (~) is a special notation for your home directory. In the Windows example above, you see a Users/bsmits/ directory – that’s my home directory. Every user on a computer has their own home directory (also called a ‘homedir’) where they can store their files. On Linux, my homedir is /home/bsmits/. Finally, the $ symbol means you’re in the command line and can start typing.

By default, your command prompt is for the bash shell. A shell is just a program that provides a text-only user interface for a Unix operating system. There are other kinds of shells for programs other than bash, but we only care about bash right now.

You can see your current working directory by running the command pwd:

To list the files and directories inside of your working directory, you can use the ls command:

In the example above, you’ll see that regular files are displayed in white, directories in blue, and executable files in green. We’ll talk about executable files later.

If you want to change your directory, you can run the command cd <directory_name>:

To move your current working directory one level up, you can use cd ..:

The .. is special notation for one directory level up. If you want to return to your homedir from any directory, you can cd ~:

1.2 - Relative vs Absolute Paths

You may have noticed that pwd didn’t just tell us we were in the bsmits directory, it told us we were in /home/bsmits/. That’s because there are two ways to specify a file/directory path: relative paths and absolute paths.

Your file system has many directories, but at the top, there is the root directory (/) that contains all files and directories on your computer. In Windows, this is the C-drive: C:\.

When we use an absolute path, we are telling bash (or another program) the full path from the root directory. So, in reality, our bsmits directory is inside the /home directory, which is inside the root directory. So the absolute path is /home/bsmits.

However, with cd, we were using relative paths. The relative path to our bsmits directory depends on what our current working directory is. If our current directory is /home, then the relative path to bsmits is just bsmits. If our current directory is bsmits/test, then the relative path to bsmits is .. (one directory up).

Relative paths can be tricky (and can even lead to security vulnerabilities). If you’re just manually navigating the file system, relative paths are fine. But if you’re using paths inside of a script/program, you should try to use absolute paths whenever possible.

1.3 - Standard Output, Standard Error, and Standard Input

On Unix systems, there are three special files: standard input, standard output, and standard error.

When you open a terminal, all three of these special files also get opened. When you ran cd ~ above, the output of the command was written to standard output (often called stdout), and your terminal displayed those new lines written to stdout to you.

When you ran cd ~ above and hit the ‘Enter’ key, the command that you typed was written to standard input (often called stdin).

If you try to cd to a directory that doesn’t exist, then you would get an error message like this:

$ cd /this/directory/doesnt/exist/
bash: cd: /this/directory/doesnt/exist: No such file or directory

The output that you see was written to standard error (often called stderr). Your terminal is hiding stdin, stdout, and stderr from you, but knowing these terms will be useful when we move on to scripting later.

1.4 - Viewing and Editing Files/Directories

1.4.1 - Viewing Files

You can view files on the command line a few different ways. Let’s imagine we have a file called dataset1_results.csv. You can see the full contents of this file using the command cat <file>:

$ cat datasets1_results.csv
UNIQUE_ID,DATE,VALUE
0,2022-08-14,0.0
1,2022-08-15,0.5    
2,2022-08-16,1.0    
3,2022-08-17,1.5    
4,2022-08-18,2.0    
5,2022-08-19,2.5    
6,2022-08-20,3.0
7,2022-08-21,3.5
8,2022-08-22,4.0
9,2022-08-23,4.5
10,2022-08-24,5.0

If we only want to see the first few lines of the file, we can use the command head -n <number_of_lines> <file>:

$ head -n 3 datasets1_results.csv
UNIQUE_ID,DATE,VALUE
0,2022-08-14,0.0
1,2022-08-15,0.5

Aside: That -n is a command flag, a way to send special options or arguments to a command. If you want to see the arguments available to a command, most commands have the --help or -h flag which will print out available argument flags. You can also view the manual page for a command by running man <command>.

By default, head shows the first 10 lines of a file if you don’t specify -n.

The tail command does the opposite of head, showing you the last lines of a file:

$ tail -n 3 datasets1_results.csv
8,2022-08-22,4.0
9,2022-08-23,4.5
10,2022-08-24,5.0

If a file is still being written to, you can follow the last lines of the file using tail -f <filename>.

1.4.2 - Editing Files

You can create, modify, rename, move, and delete files (and directories) from the command line as well. To create a new empty file, you can use the touch command:

$ touch newfileName.txt

To delete a file, you can use the rm command:

$ rm newFileName.txt

To move a file, you can use the mv <source> <destination> command:

$ mv newFileName.txt ../

The above moves newFileName.txt up one directory.

We can also use mv to rename files:

$ mv newFileName.txt moreDescriptiveFileName.txt

We can also make copies of files using the cp command. cp follows the same source and destination syntax as mv:

$ cp dataset1_results.csv dataset1_results.csv.backup

We’ll talk about editing the contents of files later.

1.4.3 - Editing Directories

We can create new directories using the mkdir command:

$ mkdir newDirectory

We can also remove directories using rmdir:

$ rmdir newDirectory

If the directory we want to remove is not empty, you will get the following output:

$ rmdir newDirectory
rmdir: failed to remove 'newDirectory/': Directory not empty

If it’s not empty, you can manually remove all of the files or directories inside it, and then run rmdir newDirectory. A shortcut for this is rm -rf newDirectory, but you should only use that if you are 110% positive that you don’t need anything inside newDirectory.

The r flags means recursive – it will remove everything inside of newDirectory before removing newDirectory. The -f flag means force removal.

You can move directories using the same mv command:

$ mv newDirectory ../

You can also rename directories using the mv command:

$ mv newDirectory moreDescriptiveDirectoryName

You can use the cp command to copy an empty directory. If you want to copy a directory and everything inside of it, you need to use the recursive flag (-r):

$ cp -r results results_backup

1.5 - Special Bash Syntax

1.5.1 - Pipes

You may be asking yourself: How do I take the output of one command and hand it to another command?

You can do this using the pipe (|) character. In the following example, the output of sort my-data.csv gets piped (i.e. fed) as the input to the uniq command:

$ sort my-data.csv | uniq

The sort command will sort all of the lines in a file. The uniq command filters adjacent matching lines.

1.5.2 - Redirects

What if you want to save the output of a command to a file? That’s what redirects are for. Redirects redirect the output of a command to a file. In the following example, the output of sort my-data-group-1.csv will be appended to (i.e. placed at the end of) the file all-data.csv:

$ sort my-data-group-1.csv >> all_data.csv

If you want to overwrite a file with the output of a command, you could do that using a single redirect:

$ sort my-data-new.csv > my-data-old.csv

You can also feed the contents of a file to a command using a reverse redirect. In the following example, each line in program-args.txt is fed one at a time to my-program.sh:

$ ./my-program.sh < program-args.txt

In the above example, ./ is special notation meaning execute a file called my-program.sh in my current working directory. We’ll talk about execution later.

What if you want to see the output of a file in stdout and write the output to a file? For that you can use the tee command with a pipe:

$ <command_that_writes_to_stdout> | tee <output_file>

1.5.3 - Combining Pipes & Redirects

You can also combine pipes are redirects. In the following example, the output of my-script.py is fed to sort, and then the output of sort is appended to the file sorted-output.txt:

$ ./my-script.py | sort >> sorted-output.txt

1.5.4 - Chaining Bash Commands

What if you want two run two commands, but only hit ‘Enter’ once? There are a few ways to do this, each with slightly different behavior.

A semicolon (;) used between two commands results in the right command running after the left command finishes (either succeeding or failing). For example:

$ ./my-script.sh; ./my-second-script.sh

If you only want the right command to run if the left command succeeds, you would use a double ampersand (&&):

$ ./my-script.sh && ./my-second-script.sh

If you want to run two commands at the same time, you can use a single ampersand (&):

$ ./my-script.sh & ./my-other-script.sh

In the example above, the right command will run in the background, meaning you won’t see any of its output.

Using a single ampersand (&) without a right command results in the left command running in the background:

$ ./my-script.sh &

1.6 - Searching

The find and grep commands are used for searching in different contexts.

1.6.1 - Searching for Files & Directories

To search for files and directories based on their names, you would use the find command:

$ find <path> <options>

To find all files starting from the directory you’re in, you would use:

$ find . -type f

The -type f flags means find only files. You can use the -type d flag to specify only directories:

$ find . -type d

If your current working directory is not the one you want to search, you can specify a different path:

$ find ../data/ -type f

If you want to narrow down your search results to files with “results” in their name, we can use the -name flag:

$ find ../data/ -type f -name "*results*"
../data/dataset1_results.csv

Note that we have to use asterisks (*) to specify that we don’t care about the name of the file that comes before “results” (“dataset1_”) or after “results” (“.csv”). If we didn’t have asterisks, we would not see any output from the previous command unless there was a file named exactly results.

You can also use the -name flag when searching for directories:

$ find . -type d -name "*data*"
./data/

If you want to find all files with the extension .csv, you can do that like this:

$ find . -type f -name "*.csv"

1.6.2 - Searching Within Files

To search for a string of characters within a file (or multiple files), you can use GREP (Global Regular Expression Print). The basic syntax for grep is:

$ grep <options> <pattern> <file>

So, if we want to search for the date “2022-08-19” in dataset1_results.csv, we would run:

$ grep "*2022-08-19*" dataset1_results.csv
7:5,2022-08-19,2.5

Your <pattern> for grep can be a simple string or a regular expression. For strings, spaces matter. Consider:

$ grep "and" theNextGreatNovel.txt

In the above example, any instance of “and” inside theNextGreatNovel.txt would match, i.e., “and”, “command”, “hand”, “land”, etc.

If we want to search for only the word “and”, we can use spaces:

$ grep " and " theNextGreatNovel.txt

If the case matters for a search, you would use the -i flag. For example:

$ grep "dr" employee_information.txt

The example above would match “DR”, “dr”, “Dr”, and “dR”. To only match “DR”:

$ grep -i "DR" employee_information.txt

If you want to search within a file for lines that do not match a pattern, you can invert the search using the -v flag:

$ grep -v "2022-08-19" dataset1_results.csv
UNIQUE_ID,DATE,VALUE
0,2022-08-14,0.0
1,2022-08-15,0.5    
2,2022-08-16,1.0    
3,2022-08-17,1.5    
4,2022-08-18,2.0    
6,2022-08-20,3.0
7,2022-08-21,3.5
8,2022-08-22,4.0
9,2022-08-23,4.5
10,2022-08-24,5.0

1.7 - Miscellaneous Bash Commands

1.7.1 - man

If you don’t know what a command does, most commands support the --help or -h flag which will print helpful information about a command and its arguments/options.

You can also use the man command to show the reference manual for a command:

$ man <command>
$ man ls

1.7.2 - history

If you forgot a command that you ran previously, you can pull up your bash history using the history command:

$ history
 2963  find results/ -type f -name "*results*"
 2964  find results/ -type f -name "*.csv"
 2965  find . -type f -name "*.csv"
 2966  grep "*2022-08-19*" results/dataset1_results.csv 
 2967  grep "2022-08-19" results/dataset1_results.csv 
 2968  grep -n "2022-08-19" results/dataset1_results.csv 
 2969  grep -v "2022-08-19" results/dataset1_results.csv 
 2970  history

1.7.3 - clear

If your terminal screen is cluttered and you want to empty it, you would simply use the clear command. You can still see previous output by scrolling up.

1.7.4 - htop

htop allows you to monitor your computational resources, such as what processes are running and how much memory and processing power they are using. On it’s own, htop will show all processes running. To see just processes that you’re running, you can use the -u flag with your username:

$ htop -u <username>

2 - Text Editors

There are generally three different command line text editors that people use. The one you use is a matter of preference, but there are differences between them.

2.1 - nano

nano is the simplest of the three editors and doesn’t take long to learn. nano is best used for basic tasks such as quickly editing small portions of a file. Commands for nano are displayed at the bottom of the editing screen so you will not have to remember them. To use nano, simply type nano <filename>.

To make edits, you just type like you normally would in a Microsoft Word document. When you’re done making changes, you press CTRL+X and nano will prompt you to save or not. Press y, then his Enter to save, n to delete your changes, or CTRL+C to exit the prompt and go back to editing.

To save without exiting nano, you can press CTRL+O and then hit Enter.

2.2 - vim

vim is more difficult to learn than nano, but it is more powerful and includes syntax highlighting, which is very useful when writing code. vim will require some effort to become proficient in, but it’s worth it for it’s efficiency. A full tutorial on vim can be found here. To use vim, type vim <filename>.

You’ll notice that when you open vim, you can’t type and make changes. Before you can do that, you have to press i. Once you do that, you will notice -- INSERT -- appears at the bottom left of the screen to tell you that you’re in insert mode.

Now you can make your changes. You can undo changes by exiting insert mode (press ESC) and then pressing the u key.

Once you’re done editing, press (ESC) to exit insert mode. To save, press CTRL+B and then type :w and hit Enter.

To save and exit, press CTRL+B, then type :wq and hit Enter.

To exit without saving, press CTRL+B, then type :q! and hit Enter.

There are many, many more things you can do in vim, so make sure you check out the tutorial linked above.

2.3 - emacs

In a basic sense, emacs is similar to vim; it’s more advanced than nano and has syntax highlighting. Reasons to choose vim over emacs (or vice versa) typically depend on your needs. Most of us here at Research Computing use vim. Here is a tutorial and a quick reference for emacs.

2.4 - Text Editor Comparison

3 - File System Permissions

3.1 - Basic Permissions

Every file and directory has separate permissions for read (r), write (w), and execute (x).

For files, these permission bits are straightforward. If you have read permissions, you can read the file (e.g. cat, head, tail); if you have write permissions, you can edit the file (e.g. vim); and if you have execute permissions, you can execute the file.

For directories, these bits mean slightly different things. If you have read permissions, you can list (ls) the contents of a directory; if you have write permissions, you can create new files/directories, modify them, and delete them; if you have execute permissions, you can change directory (cd) into that directory.

Note: It’s not intuitive, but you can read a file if (and only if) you have read permissions for the file and execute permissions to that file’s parent directory.

3.1.1 - Permission Levels

There are three levels of permissions for every file and directory: owner (u), group (g), and other (o). Each permission level has its own read, write, and execute bits.

Let’s take a look at the output of ls -l:

$ ls -l
permissions        user  group    file/dir  name
drwxr-x---  [...]  ben   faculty  [...]     .
drwx------  [...]  ben   faculty  [...]     ..
-rwxrwxrwx  [...]  ben   faculty  [...]     allopen.sh
drwx------  [...]  ben   faculty  [...]     mydir
-rw-------  [...]  ben   faculty  [...]     myfile.txt
-rwx------  [...]  ben   faculty  [...]     myprog.py
drwxrwx---  [...]  ben   faculty  [...]     ourdir
-rwxrwx---  [...]  andy  faculty  [...]     ourprog.rb

If you look at the “permissions” column, you can see 10 characters:

• The 1st character indicates the type of file you’re looking at
  • -: file
  • d: directory
  • l: symbolic link
• The 2nd-4th characters are the read, write, and execute bits for owner permissions
• The 5th-7th characters are the read, write, and execute bits for group permissions
• The 8th-10th characters are the read, write, and execute bits for other permissions

3.1.2 - Group Membership

If you run the groups command, you can see what groups your user account belongs to:

$ groups
staff rc-users rc-admins its_staff users ...

You can also run group <user_name> to see the groups that a different user belongs to. Groups can make permissions management easier, as you will see in the following examples.

Groups can also be part of other groups.

If you are the owner of a file/directory, then you have the read, write, and execute permissions for the owner.

If you’re not the owner, but are a member of the group that a file/directory belongs to, then you share the read, write, and execute permissions for that group.

Finally, if you’re not the owner or a member of the group a file/directory belongs to, then you share the other read, write, and execute permissions.

3.1.3 - Permissions Quiz

Try to answer these questions on your own. Answers are included at the end of the tutorial.

1. Assuming andy is a member of the faculty group, can andy execute myprog.py?

2. Assuming ben and andy are both members of the faculty group, can they both execute ourprog.rb?

3. Can andy read ourprog.rb?

4. Assuming andy is a member of the faculty group, can andy change directory (cd) into mydir/?

5. Can the user jane (a member of the student group) read ourprog.rb?

3.2 - Changing Permissions

The easiest way to change permissions on a file or directory is to use the chmod command:

$ chmod <options> <file/directory name>

3.2.1 - Using Symbolic Notation

You can use the r (read), w (write), and x (execute) characters with the u (owner), g (group), and o (other) characters to modify permissions:

• Setting permissions with =:
  • chmod u=rw . sets the read and write permissions for the owner (i.e. user) on the current directory.
  • chmod ug=w script.sh sets the write permissions for the owner and group for script.sh.
• Adding permissions with +:
  • chmod g+rw . adds read and write permissions for the current directory’s group. If the group already had read and/or write permissions, then nothing changes.
  • chmod a+r . adds read permission to the owner, group, and other. The a character here means user, group, and other.
• Removing permissions with -:
  • chmod g-w . removes write permissions for the current directory’s group. If that group did not have write permissions, nothing changes.
• You can change permissions recursively using the -R flag
  • Note: Be careful making recursive permission changes. Some programs (e.g. ssh) won’t function properly if the permissions set on the files/directories used by the program (e.g. ~/.ssh/) aren’t correct.
  • There are also security concerns if certain system files/directories don’t have the correct permissions.

You will probably only care about permissions if you’re trying to execute a file. If you create a new file, you will typically have to chmod u+x newFile.sh before you can execute it.

3.2.2 - Using Octal Notation

Remember those r, w, and x bits? Well, those bits have values associated with them:

Octal notation can be used to set permissions, but it’s difficult to use for adding or removing permissions:

$ chmod ### file.txt

So, let’s say we want these permissions for file.txt:

• Owner: rwx
• Group: r-x
• Other: r-x

We can look at the top row of the table above and see that the decimal value for rwx is 7, and the decimal value for r-x is 5. When using octal notation, you give chmod three digits. The first digit represents owner permissions, the second digit represents group permissions, and the third digit represents other permissions.

To set these permissions for file.txt, we would run:

$ chmod 755 file.txt

3.2.3 - Default Permissions

When a file or directory is created, the operating system consults the umask (User Mask) to determine default permissions. Umasks are set by the systems administrator, so they will vary between computers. Some common defaults include:

• rw------- (077)
• rw-r--r-- (022)
• rw-rw---- (007)

Default permissions get set when a file or directory is created by subtracting octally (digit by digit):

• For files, subtract octally 666
• For directories, subtract octally 777

If we have a umask 022 and we create a new file: 666 - 022 = 644.

• 6 for owner permissions means rw-
• 4 for group and other permissions means r--
• Our new file has the permissions: rw-r--r--

You most likely will not have to worry about umasks. If you’re having permissions problems, we can help you sort them out.

3.2.4 - Changing Owner and Group

You can change the owner or group of a file/directory (if you have permission to do so) using the chown command:

$ chown <options> <owner_name>:<group_name> <file/directory>

To change the owner of some-script.sh to ben, you would run: chown ben some-script.sh. To change the owner (to ben) and group (to faculty), you would run: chown ben:faculty some-script.sh.

You can also use the recursive flag (-R) with chown, but this can have unintended side effects, so we don’t recommend it. Example:

$ chown -R <owner_name>:<group_name> <directory>

3.3 - Executing Files

On Windows and Mac, the file extension (e.g. “.doc”, “.txt”, “.py”) tells the operating system what program to open a file with. On Unix, you need to be explicit.

So, to run a python file from the terminal: python3 somePythonFile.py

To run a bash file: bash someBashFile.sh

A quick Google search for other programming languages will tell you how to run them from the command line.

But there’s another way to run files on the command line. At the top of your file, you can include a shebang (#!) statement, which tells bash what program to execute a file with.

So, for a python file, if you make the first line in the file #!/usr/bin/env python, then you can execute the file simply by typing $ ./somePythonFile.py. Bash will look at the shebang statement, then look in /usr/bin/env/ for a program called python. If bash finds it, it will do $ python somePythonFile.py for you in the background.

For bash files, you would do #!/bin/bash. Different programming languages get installed in different locations on the file system, so if you’re not sure where your program is installed, you can run which <program_name> to find out:

$ which ruby
/usr/bin/ruby

3.3.1 - Executing as Other Users

Normally, when you execute a program, the operating system ignores the owner permissions and tries to run the program with your user account. So, if a program is owned by andy, but you have permission to execute it, it will execute that program as you.

This allows the root user (which has rwx permissions for everything) to own programs that cannot be modified, but can be executed. For example, the passwd command changes your password. You want to be able to run passwd as yourself, but you also want to make sure the passwd program cannot be altered.

NOTE: Do not run passwd on the cluster. The cluster uses your RIT username and password. If you need to change your password, then visit start.rit.edu.

The command su allows you change the current user to a different user. You should not need to do this on the cluster.

The command sudo allows you to run commands/programs as root. You do not have permission to use sudo by default. If you think you need sudo privileges, then please talk to us.

4 - Writing a Bash Script

Alright, you’ve got a solid handle on the file system and some useful bash commands. Now it’s time to learn the basics of bash so you can write your first bash script!

4.1 - Bash Variables & Comments

A variable is just a special name that holds a value. You can create a variable and assign it a value like this:

STR="Hello World!"

In the above, the value “Hello World!” gets assigned to the variable STR. Note that there are no spaces around the equals (=) sign.

The above STR variable is an example of a string, a type of variable that is a sequence of characters. Variables can also be numerical:

NUM=1234

You can also assign the output of a command to a variable. Let’s say we want to get today’s date and save that to a variable. We can use the date command to get today’s date:

$ date
Tue 13 Sep 2022 08:25:05 AM EDT

Now, inside a bash script, we can save the output of date to a variable:

TODAY=$(date)

In the syntax above, you are telling bash to execute whatever is inside of $(...), in this case date.

So let’s write a simple bash script so we can test all of this out. Create a file called hello_world.sh and put the following inside of it:

#!/bin/bash

# Get today's date
TODAY=$(date)
STR="Hello World!"

# Print out today's date
echo $STR
echo "It's ${TODAY}"

Save the file.

Note the syntax above. To access the value of a variable in bash, you have to reference it using the dollar sign ($). If it’s inside of quotes, you need to do ${<variable_name>}. You will also notice the comments above, denoted by the pound (#) sign. In bash, anything on a line following a # will be a comment and will not be executed. Comments are useful for explaining complicated blocks of code.

Change the file’s permissions so you can execute it: chmod u+x hello_world.sh. Now execute the file and you should see output similar to this:

$ ./hello_world.sh
Hello World!
It's Tue 13 Sep 2022 08:34:42 AM EDT

If you run the file again, you should see that the date has changed because the date command got run again:

$ ./hello_world.sh
Hello World!
It's Tue 13 Sep 2022 08:34:45 AM EDT

You can also create arrays, lists of values. Let’s say we want a list of programming languages:

# Obviously not a complete list
LANGS=('Bash' 'Python' 'Ruby' 'C')

Now we can access those values using their index. Each element in an array has an index starting from 0. Add the following to your script:

# Access each element in an array
LANGS=('Bash' 'Python' 'Ruby' 'C')
echo ${LANGS[0]}
echo ${LANGS[1]}
echo ${LANGS[2]}
echo ${LANGS[3]}
# Access the entire array
echo ${LANGS[@]}

If you run the script now, you will see output similar to:

$ ./hello_world.sh
Hello World!
It's Tue 13 Sep 2022 09:35:33 AM EDT
Bash
Python
Ruby
C
Bash Python Ruby C

That’s it! That’s your first bash script!

4.2 - Bash Loops & Conditionals

Let’s say you have a list of names, and you want to count how many times a specific name appears in the list:

NAMES=('Ben' 'Paul' 'Ben' 'Emilio' 'Jen' 'Andy' 'Ben' 'Kirk' 'Doug')

To do this, we need to use loops and conditionals.

To loop through NAMES, the syntax looks like this:

for i in "${NAMES[@]}"; do
  echo $i
done

The code above will take each element in the array NAMES and assign it to the variables i so you can repeat a certain function (in this case echo) for each element. So the output of the above would be:

Ben
Paul
Ben
Emilio
Jen
Andy
Ben
Kirk
Doug

Okay, so now we want to count how many times the name “Ben” appears in NAMES. To make comparisons, you use if-statements:

if [[ <some_condition> ]]; then
  # do something
elif [[ <some_other_condition> ]]; then
  # do something else
else
  # do a different thing
fi

So, applying this to our task:

NAMES=('Ben' 'Paul' 'Ben' 'Emilio' 'Jen' 'Andy' 'Ben' 'Kirk' 'Doug')
CNT=0
for i in "${NAMES[@]}"; do
  if [[ $i == "Ben" ]]; then
    CNT=$((CNT+1))
  fi
done

echo $CNT

The output of the above code should be: 3.

You can also loop over the files in a directory:

for i in ~/books/*; do
  echo $i
done

If you wanted to loop through the values 1-5:

for i in {1..5}; do
  echo $i
done

4.3 - Bash Operators

Bash, like any programming language, has operators which allow variables to be set, modified, and compared. You already saw one operator, the = sign, which let you set the value of a variable.

With bash, you can do basic arithmetic like this:

NUM1=100
NUM2=2

echo $(($NUM1 + $NUM2)) # Addition
echo $(($NUM1 - $NUM2)) # Subtraction
echo $(($NUM1 * $NUM2)) # Multiplication
echo $(($NUM1 / $NUM2)) # Division

If you execute that, you will see the following, as expected:

102
98
200
50

You can also compare variables, which we already saw with strings. Below is a list of the most common conditional operators:

4.4 - Bash Command Line Arguments

Suppose you want to write a script that uses a different value every time you execute it. You can do this with command line arguments. For example:

$ ./coolBashScript.sh <some_argument>

And then inside of coolBashScript.sh, you can access <some_argument> using $1. If you have multiple arguments, the are numbered sequentially, e.g. $1, $2, $3, etc.

Here’s an example script that takes in two arguments are multiplies them together:

$ cat bash_mult.sh
#!/bin/bash

echo $(($1 * $2))

4.5 - Bash Challenge Script

There’s much more to bash, but that should be enough to get your started. So, here’s your challenge:

Write a bash script that takes in a number and a string as command line arguments. If the number is odd, print out the string five times with a different multiple of the number appended each time. If the number is even, print out the string five times with a random number (between 0 and 99) appended each time.

You will have to look up how to do random numbers in bash.

Use the following example output as your guide:

• Number: 3, String: Ben, Output:

  • Ben3
    Ben6
    Ben9
    Ben12
    Ben15
    

• Number: 10, String: Paul, Output:

  • Paul17
    Paul42
    Paul3
    Paul86
    Paul74
    

Please try to do this on your own. One possible answer is provided in Section 6.2

5 - Conclusions

Congratualations on completing this tutorial! Now you have a solid grasp on the Unix filesystem, including how to navigate files/directories, how to create and modify files/directories, and how filesystem permissions work. You also learned about text editors. Finally, you learned the basic of bash and wrote your first bash script!

Now you’re ready to move on and learn how to create and submit jobs on the cluster in our Slurm Quick Start Tutorial.

5.1 - Permissions Quiz Answers

1. Assuming andy is a member of the faculty group, can andy execute myprog.py?
   • No, because andy is not the owner of myprog.sh (ben is), and the faculty group does not have execute permissions for myprog.sh.
2. Assuming ben and andy are both members of the faculty group, can they both execute ourprog.rb?
   • Yes, because the faculty group has execute permissions for ourprog.sh.
3. Can andy read ourprog.rb?
   • Yes, because andy has read permissions for ourprog.rb and execute permissions for the parent directory of ourprog.rb.
4. Assuming andy is a member of the faculty group, can andy change directory (cd) into mydir/?
   • No, because andy is not the owner of mydir/ and the faculty group does not have execute permissions for mydir/.
5. Can the user jane (a member of the student group) read ourprog.rb?
   • Only if the student group is a member of the faculty group (because the faculty group has read permissions for ourprog.rb).

5.2 - Challenge Script Answer

$ cat challenge.sh
#!/bin/bash

NUM=$1
STR=$2

if [[ $((NUM%2)) -eq 0 ]]; then
  for i in {1..5}; do
    echo $STR$(($RANDOM%100))
  done
else
  for i in {1..5}; do
    echo $STR$(($NUM * $i))
  done
fi

5.3 - Unix Command Command Reference

5.4 - Other Resources

• Unix Command Cheat Sheet
• HPC Carpentry has an in depth tutorial on that covers much of the same material on this page, but in more depth.