Carpentries Shell Lesson

Leighton Pritchard

University of Strathclyde

2025-11-04

1. Summary and Setup

Software and files

  • University PC: use Git Bash
  • Windows laptop: use Git Bash or WSL
  • Mac laptop: use Terminal (or equivalent)
  • Linux laptop: use Bash shell

Important

Figure 1: lesson data download

https://carpentries.github.io/workshop-template/install_instructions/

2. Nelle’s pipeline

Nelle’s project

Nelle Nemo, a marine biologist, has just returned from a six-month survey of the North Pacific Gyre, where she has been sampling gelatinous marine life in the Great Pacific Garbage Patch.

She has 1520 samples that she’s run through an assay machine to measure the relative abundance of 300 proteins.

She needs to run these 1520 files through an imaginary program called goostats.sh. In addition to this huge task, she has to write up results by the end of the month, so her paper can appear in a special issue of Aquatic Goo Letters.

Nelle’s pipeline

Nelle will need to know how to:

  • navigate to a file/directory
  • create a file/directory
  • check the length of a file
  • chain commands together
  • retrieve a set of files
  • iterate over files
  • run a shell script containing her pipeline

Filesystem structure

Figure 2: Nelle’s filesystem

Home directories

Figure 3: Nelle’s filesystem home directories

Challenge (1min)

ls -rt displays which file last?

ls -t lists items by time of last change, ls -r reverses the list order

  • filename that starts earliest in alphabet
  • most recently modified file
  • earliest modified file
  • filename that starts latest in alphabet

Challenge (1min)

Starting from /Users/nelle/data how can you navigate to Nelle’s home directory?

  • cd .
  • cd /
  • cd ~
  • cd home
  • cd
  • cd ..

Challenge (1min)

If pwd returns /Users/backup what command gives the output below?

pnas_sub pnas_final/ original

Figure 4: Filesystem structure
  • ls pwd
  • ls -r -F
  • ls -r -F /Users/backup

4. Shell command syntax

Shell command components

Figure 5: Shell command syntax

5. Working with files and directories

Good names for files and directories

Use descriptive filenames

2025-10-11_lcms_clean.csv rather than some_data.csv

Use a limited set of characters

Stick to lower-case and a restricted set of special characters to avoid clashes with special instructions

[a-z][0-9].-_

Don’t use spaces

  • % ls my-folder will show the contents of my-folder
  • $ ls my folder will attempt to show the contents of my and folder

Don’t start a name with a dash

Commands treat these as options

File extensions

  • Most filenames come in two parts, like draft.txt
    • draft is the filestem
    • .txt is the extension

File extensions indicate content type

  • .txt: plain text
  • .pdf: PDF document
  • .cfg: configuration file

Extension matching content is a convention only

Changing a file extension does not change the file content

Challenge (1min)

You accidentally called your file statstics.txt

How can you rename it to statistics.txt?

  • cp statstics.txt statistics.txt
  • mv statstics.txt statistics.txt
  • cp statstics.txt .
  • mv statstics.txt .

Challenge (1min)

Which ls command produces the output below?

alkanes/ethane.pdb alkanes/methane.pdb

  • ls alkanes/*t*ane.pdb
  • ls alkanes/*t?ne.*
  • ls alkanes/*t??ne.pdb
  • ls alkanes/ethane.*

5. Pipes and Filters

Combining multiple commands

Figure 6: Redirection and pipes

Challenge (1min)

Which command shows the three files with fewest lines?

  • wc -l * > sort -n > head -n 3
  • wc -l * | sort -n | head -n 1-3
  • wc -l * | head -n 3 | sort -n
  • wc -l * | sort -n | head -n 3

6. Loops

The structure of a loop

# The word "for" indicates the start of a "For-loop" command
for thing in list_of_things 
#The word "do" indicates the start of job execution list
do 
    # Indentation within the loop is not required, but aids legibility
    operation_using/command $thing 
# The word "done" indicates the end of a loop
done

Challenge (3min)

Write your own loop

Write a loop that echoes all ten numbers from 0 to 9 on screen

Write your own loop

Write a loop that echoes all ten numbers from 0 to 9 on screen

Solution

$ for loop_variable in 0 1 2 3 4 5 6 7 8 9
> do
>     echo $loop_variable
> done

7. Shell Scripts