This article is the first lesson for learning Shell, mainly covering the definition and manipulation of Shell variables:

  • Variable names and variable types
  • The echo command and escape characters
  • How strings behave differently in single quotes versus double quotes
  • Using {} to delimit the scope of a string
  • The unset and readonly keywords

1. Variable Definition

A variable is defined in Shell as follows:

var_name=variable

Here, var_name is the variable name and variable is the variable’s value. There must be no spaces on either side of the = between the variable name and value!

1.1 Variable Names

Shell variable names must satisfy the following rules:

  • Can only contain letters, digits, and underscores, and can’t start with a digit
  • Case-sensitive
  • Must not be a Shell keyword (you can get the list with the help command)

Based on these rules, valid variable names include:

hello
hello000
hello_world_00
_konichiwa
nihao_

Invalid variable names:

help # a keyword
ni hao # contains a space
9am # starts with a digit

1.2 Variable Values

Variable values in Shell come in two kinds: strings and arrays.

A variable’s value can come from either a direct assignment or the output of a command.

1.2.1 Strings

Strings are the common data type in Shell. Shell has no int, bool, or other such types — everything except arrays (covered below) is a string.

All of the following variables are strings:

a="xy"
b=1
c="1"
d="hello world"
e="x\n"
f='x'
g='x\n'

When defining a string’s value, you can add quotes to explicitly mark its boundaries, and the actual result won’t include those quotes. Note the difference between single and double quotes (covered below).

a="xy"
a='xy'
a=xy

In all three definitions above, a’s value is xy (without any quotes).

Of course, you can also use the output of another command as a variable’s value, using either `cmd` (backticks, the key just below esc) or $(cmd):

x=`pwd`
x=$(pwd)

1.2.2 Arrays

Defined with parentheses (), with elements separated by spaces:

var_list=(1 2 3)

An array in Shell is similar to a Python dict, having both a key and a value — it just so happens that the key is an integer. So you can also define an array like this:

var_list=(0=[hhh] 1=[2])

An array’s elements are strings, as introduced above. Shell only supports one-dimensional arrays — it does not support higher-dimensional arrays.

Of course, an array in Shell can also come from a command’s output, using either `cmd` (backticks, the key just below esc) or $(cmd):

a=`ls -a`
b=$(ls -a)
echo 'files are `ls -a`'
echo "files are `ls -a`"
echo "files are $(ls -a)"

1.3 Variable Reference

After defining a variable, you’ll want to use its value somewhere else.

1.3.1 The echo Command

Before covering variable reference in Shell, let’s first cover Shell’s echo command, whose job is to print out a value (similar to print in Python or printf in C++).

echo "haha" # haha

The echo command also takes flags: adding -e enables escape sequences, while -E disables them (no escaping is the default). The e stands for “escape.” Shell’s escape sequences are as follows:

Escape sequenceMeaning
\\Backslash
\aAlert, bell
\bBackspace (deletes the preceding character)
\fForm feed (FF), moves the current position to the start of the next page
\nNewline
\rCarriage return
\tHorizontal tab
\vVertical tab

For the difference between \n and \r, see another article on this site.

echo -e "x\n"
# x
#

echo -E "\n"
# \n

1.3.2 Variable Reference

  • To reference an already-defined variable, add $ before the variable name (don’t add $ when defining a variable or reassigning it)
  • When getting a variable’s value with $, you can add {} to explicitly delimit the variable’s scope

To illustrate this, let’s first cover string concatenation in Shell — writing two strings next to each other.

a=5
b=6
echo a # output a
echo $a$b # output 56
echo $a # output 5
echo ${a}x # output 5x
echo $ax # output None (no output)

Curly braces {} are used to delimit scope — ${a}x makes it clear the variable is a, not ax.

A Shell variable can be wrapped in quotes, and the actual content won’t include those quotes. Single and double quotes aren’t interchangeable, though:

  • Content inside single quotes is never evaluated as a variable — it’s output exactly as written
  • Content inside double quotes has variables evaluated first, and the result is then output
a=5
echo 'a'  # output a
echo "a" # output a
echo '$a' # output $a
echo "$a" # output 5
echo $a # output 5

1.3.3 Variable Substitution

In Shell, forcibly echoing an undefined variable outputs nothing at all, and doesn’t raise an error either. To handle the issue caused by an undefined variable, Shell provides variable substitution syntax for handling the case where a variable hasn’t been assigned.

FormDescription
${var}The value of var
${var:-word}If var is defined, outputs var’s value; if var is empty, undefined, or has been deleted (unset), returns word instead, without changing var’s value.
${var:=word}If var is defined, outputs var’s value; if var is empty or has been deleted (unset), returns word, and also sets var’s value to word.
${var:?message}If var is defined, outputs var’s value; if var is empty or has been deleted (unset), sends message to standard error. This can be used to check whether var was assigned properly. If this substitution appears inside a shell script, the script will stop executing.
${var:+word}If var is defined, returns word instead, without changing var’s value. If var isn’t defined, returns empty.

1.4 unset and readonly

After defining a variable in Shell, you can delete it using unset.

a=4
echo $a # output 4
unset a
echo $a # output None (no output)

In Shell, once a variable has been deleted, referencing it again returns None. This differs from most programming languages, which typically raise an error when handling an undefined variable. (This behavior in Shell also makes debugging shell scripts harder.)

Add the readonly keyword when defining a variable:

readonly x=5
x=6 # readonly variable

Once readonly has been added in front of a variable, it can no longer be deleted.

2. String Operations

Every Shell variable is either an array or a string. Beyond just retrieving a variable’s value, you can also perform more operations. Common operations include:

  • Getting a string’s length
  • String concatenation
  • String extraction (substring)
  • String comparison
  • Operations on numeric strings

2.1 Getting a String’s Length

x="hello world"
echo ${#x} # 11

# is the symbol used for comments in shell — but placed in front of a variable, surrounded by {}, it instead means “get the length.”

2.2 String Concatenation

This just means combining multiple strings into one — simply place them next to each other.

x=5
y='u'
z=$x$y
echo $z # 5u
z=${x}${y}
echo $z # 5u

2.3 String Extraction

Extracting a portion of the original string according to certain rules.

2.3.1 Extracting by Length

To extract from the variable x:

${x:start:length}
  • Extracts from left to right
  • start is the starting index
    • The index of the first element of an array in Shell is 0
    • start can be a plain number, indicating the starting index
    • start can also be of the form 0-n, meaning it counts the starting character from the end
  • If length extends past the end of the string, everything from start to the end is extracted
x="hello world"
y=${x:1:2} # 1 is the start index left to right, 2 is the length
echo $y # el
y=${x:0-5:2} # right to left
echo $y # wo

2.3.2 Extracting by a Specific Character

  • # extracts the characters to the right, % extracts the characters to the left
  • One # (or %) means the first match from left to right; two means the last match from left to right
  • * represents any character
# use # to extract the characters to the right, from left to right
url="http://aoi.ai/index.html"
echo ${url#*/}    # /aoi.ai/index.html, the first match's right-hand characters, from left to right
echo ${url##*/}   # index.html, the last match's right-hand characters, from left to right

str="---aa+++aa@@@"
echo ${str#*aa}   # +++aa@@@
echo ${str##*aa}  # @@@

# use % to extract the characters to the left, from left to right
url="https://aoi.ai/index.html"
echo ${url%/*}  # https://aoi.ai
echo ${url%%/*}  # http:

str="---aa+++aa@@@"
echo ${str%aa*}  # ---aa+++
echo ${str%%aa*}  # ---

2.4 String Comparison

2.4.1 The test and if Functions

You can compare strings in Shell. The result of the comparison can be passed to if and similar constructs. For example:

# use test function
if test "$a" = "$b";then
 command_1
 else
 command_2
fi

# use [ ]
if [ "$a" = "$b" ]; then
command_1
else
command_2
fi

The semicolon ; isn’t strictly necessary. test "$a" = "$b" performs the comparison, and the result is passed to if — if the result of test is true, command_1 runs, otherwise command_2 runs. In practice, test can be written using square brackets [].

2.4.2 String Equality and Inequality

Two strings are equal if they’re exactly identical, and unequal otherwise. Use = (with spaces on both sides) to test equality, and != (with spaces on both sides) to test inequality.

a=5
b=5
if [ $a = $b ]; then
echo "equal"
fi

In practice, you’ll also come across a double equals sign (==):

  • In bash, a single equals sign and a double equals sign are equivalent
  • In other shells, only a single equals sign means equality

So the recommended style is to use a single equals sign.

2.4.3 Numeric String Operations

Although Shell doesn’t have a dedicated numeric type, if a string is in integer form, it can be treated as a number: besides testing for equality, you can also compare magnitude:

MeaningOperator
equal-eq
not equal-ne
greater than-gt
less than-lt
greater or equal-ge
less or equal-le

Note that both sides of these operators must be integers, or an error will occur.

a=3
b=2
if [ $a -eq $b ]; then
echo "$a equals $b"
else
echo "$a does not eqaul $b"
fi

2.4.4 Basic Arithmetic on Numbers

Shell has no dedicated numeric type — numbers are treated as strings too. If you actually need to perform basic arithmetic, you need to reach for an additional tool.

Integer Arithmetic

Use the expr command, following the pattern expr var_a operator var_b. Common integer operators:

OperatorSymbol
Integer addition+
Integer subtraction-
Integer multiplication\*
Integer division/
Integer modulo (remainder)%
Left parenthesis\(
Right parenthesis\)
    1. Division here is integer division, so the result is an integer, with any fractional part simply dropped
    expr -5 / -3 # 1
expr -5 / -3 # -1
expr 5 / -3 # -1
expr 5 / 3 # 1
    1. The sign of the remainder follows the sign of the first operand
    expr 5 % 3 # 2
expr 5 % -3 # 2
expr -5 % -3 # -2
expr -5 % 3 # -2
    1. Parentheses change operator precedence, and spaces are required between operands
    expr \( 5 + 3 \) \* 2
    1. expr calls can be nested for chained computation
    expr $(expr 4 + 5) \* 8 # 72
expr `expr 4 + 5` \* 8 # 72

Floating-Point Arithmetic

Floating-point arithmetic requires bc:

echo 5.5 + 5.5 + 5.5 | bc # 16.5
echo 5.5 \* \( 5.5 + 5.5 \) | bc # 60.5

Here | is the pipe operator, feeding the previous command’s output as the next command’s input. Adding scale lets you set the number of decimal places kept.

echo 5.5 / 10 | bc # 0
echo "scale=4; 5.5 / 10" | bc # 0.5500

3. Array Operations

3.1 Accessing an Array

3.1.1 Accessing a Single Element

Use ${a[index]}; array indices in Shell start from 0.

a=(1 2 3)
echo "${a[2]}" # 3

index can be a positive integer, indicating the position counted from the left. If the given index doesn’t exist, an empty result is returned.

a=(1 2 3)
echo "${a[4]}" # outputs a blank line, meaning nothing was found

You can also use the 0-n form, just as with string extraction, to index from right to left — counting starts at 1 in this case:

a=(1 2 3)
echo "${a[0-1]}" # 3

3.1.2 Iterating Over an Array

To output every element of an array at once, use @ or *:

a=(1 3 5)
echo "${a[@]}" # 1 3 5
echo "${a[*]}" # 1 3 5

Shell also has a for construct for iterating:

for i in ${a[@]};
do
command_1
done

Here, a is an array. The object after in shouldn’t be the array itself — it has to be written as ${a[@]} or ${a[*]}:

a=(1 2 3)

for i in ${a[@]};
do
echo $i
done
# 1
# 2
# 3

for i in ${a[*]};
do
echo $i
done
# 1
# 2
# 3

for i in $a;
do
echo $i
done
# 1

Both $a[*] and $a[@] represent all the elements of the array — so what’s the difference between them?

  • When not enclosed in double quotes (” ”), both output all arguments in the form "$1" "$2" … "$n".
  • When enclosed in double quotes (” ”):
    • "$a[*]" treats all the arguments as a single whole, outputting them in the form "$1 $2 … $n"
    • "$a[@]" keeps each argument separate, outputting them in the form "$1" "$2" … "$n"
a=(1 2 3)

for i in "${a[@]}";
do
echo $i
done
# 1
# 2
# 3

for i in "${a[*]}";
do
echo $i
done
# 1 2 3

Using the * and @ symbols, you can get every element of an array. Similar to getting a string’s length, you can also get the number of elements in an entire array:

a=(1 2 3)
echo "${#a[@]}"

3.1.3 Array Slicing

This means extracting part of an array, similar to string access:

a=(1 2 3)
echo "${a[@]:1:2}" # 2 3
echo "${a[@]:0-1:2}" # 3

3.2 Modifying an Array

To modify an element of an array, use plain assignment (don’t add a dollar sign).

a=(1 2 3)
echo "${a[1]}" # 2
a[1]=4
echo "${a[1]}" # 4

To append an element to the end of an array, just set index to one past the array’s current length.

a=(1 2 3)
echo "${#a[@]}"  # 3
a[3]=4
echo "${#a[@]}"  # 4
a[8]=6
echo "${#a[@]}"  # 5

This index can be any positive number. The array a itself has a length of 3; you can set the value of a[3], or you can also set the value of a[5]. At this point the array starts to resemble a Python dictionary a bit — the key is the index, and it returns empty if the index doesn’t exist. Even though a[8] was forcibly set, the overall array’s length remains 5.

3.3 Deleting from an Array

Use unset:

a=(1 2 3)
unset a[0]
echo "${a[0]}" # blank

3.4 Replacing an Array Element

You can also replace an element in an array — this replaces the value, not the key, and it doesn’t modify the original array.

a=(1 2 3)
echo "${a[@]/3/777}" # 1 2 777
echo "${a[@]}"  # 1 2 3

Summary

This article introduced variable definition and operations in Shell: covering both string operations and array operations. String operations included testing equality, integer arithmetic, and floating-point arithmetic; array operations covered adding, deleting, looking up, and modifying array elements.