Oluchi John

Posted on Apr 26 • Edited on Aug 12

Shell Programming

#shell #linux #bash #gitbash

In today's rapidly evolving tech landscape, mastering the command line streamlines the development process for developers. The command line can be accessed through various shell environments like Bash and PowerShell. This article explores the shell environment, showing its role in executing complex operations, navigating files, processing data, and text manipulation.

Definition of shell

A shell is a user interface that allows users to interact with a computer's operating system using text-based commands. Think of it as a translator between the user and the operating system.
The shell takes the human-readable commands entered by the user and translates them into instructions that the operating system understands and executes, it also displays the results or output of these commands back to the user. This type of interaction, called a command-line interface (CLI), gives you more direct control over the system compared to graphical user interfaces (GUIs).
With Command Line Interface, you can run programs, manage files, and perform various tasks.

Importance of Shell

Shell allows users to interact with the operating system. This makes it easier to execute complex tasks by typing specific commands.
It allows users to automate repetitive tasks using written shell scripts.
It helps administrators and users access remote systems through protocols like SSH
It can be used to access and manage files and directories.
It diagnoses issues and checks system status.

Shell Environments

There are several shells available, each with its own unique features and syntax. Some of the most widely used shells include:

Bourne-Again Shell(BASH)
PowerShell
Z Shell (ZSH)

Bourne-Again Shell(BASH): This is one of the shells on Linux distributions and macOS. Its syntax is compatible with the original Unix shell, making it a popular choice.

PowerShell: PowerShell is a shell environment pre-installed on Windows systems by default and it is object-oriented.

Z Shell(ZSH): Zsh is known for its extensive customization options and advanced features. It offers improved tab completion, history management, and plugin support.

This article uses the Bash shell for code examples and commands. If you're unfamiliar with Bash, consider using Git Bash for Windows or the built-in Terminal app on macOS. Both offer a Bash-like experience that you can follow along with in this article.

Shell Navigation and operations

Navigating the file system through the command line is one of the foundational skills in mastering the shell environment. The ability to change directories, manage files, and understand pathnames is essential for effective command-line operations. A directory is known as a folder in the Windows system.
The various commands for navigating the file system are :

Commands	Function
cd(Change Directory)	This command is used to change your working directory. You can change to a different directory by typing cd followed by the directory name or path. For example: `cd newfile.`
pwd(Print Working Directory)	This command shows the path of the current directory you're working with.
ls	This command is used to list the contents of a directory.

Command syntax

Some commands have components that modify their function. These components are called options and arguments.
Options are used to adjust the operation of the command while arguments are what the command operates on. For example:
ls -l myfile
-l is the option while myfile is the argument.

Here is a table that contains command options and their function:

Commands	Function
ls -a	List all file contents including hidden ones
ls -t	List all file contents by the time they were last modified
cd -	This will take you to the previous directory
cd ..	This will take you to the parent directory
cd ~username	This will take you to the directory of the specified username
rm -r	This will delete directories and their contents.
ls -l	list all file contents in a long format
ls -al	list all file contents, including hidden ones in a long format.

File and directory manipulation

mkdir: This command is used to create directories, for example: mkdir newDirectory
touch: This command creates an empty file, for example: touch Mynewfile creates a file named "Mynewfile"
mv: This command can both move files/directories and rename them, for example: mv oldfile newfile, this moves or renames oldfile to newfile.
cp: This command is used to copy files and directories, for example: cp Myfile backup copies "Myfile" to a "backup" directory.
rm: This command deletes files, for example:rm unwanted.txt
rmdir: This command deletes empty directories, for example:`rmdir emptydir

File Handling and Text Processing

Working with Files and Directories:

There are text editors that allow you to create, view, and modify text files directly from the command line. Some of these popular text editors include:

Viewing File Content

cat: The cat command displays the entire contents of a file in the terminal, for example: cat filename
less: The less command allows you to view the contents of a file one screen at a time, enabling navigation and searching.
head and tail: These commands display the first or last few lines of a file, respectively.

Text Processing with grep, sed, and awk

grep: The grep command is a powerful tool for searching for specific patterns or text within files. It can be combined with other commands for more advanced operations.

Example: grep "keyword" myFile

sed: The sed command stands for stream editor, it performs text transformations on an input stream. It's particularly useful for finding and replacing text within files.

Example: sed 's/old/new/' myfile, this will replace "old" with "new" in a file called "myfile"

awk: The awk command processes and analyzes text line by line. It's often used for data extraction and reporting.

Syntax: awk 'pattern { action }' file
pattern specifies the condition for executing the action.
action is the block of code to be executed if the pattern matches.
file is the input file to be processed. If not specified, awk reads from standard input.

Example:awk '{sum += $1} END {print sum}' myfile, this will calculate the sum of numbers in a column and print their sum.

Remote System Administration with Shell

Shell access to remote systems is achieved through protocols like Secure Shell (SSH). SSH is a private key that enables you to access a remote server; it ensures a secure connection by encrypting the communication between the user's machine and the remote server.

Brief Overview Of How It Works

To establish an SSH connection, administrators provide the remote server's address and authentication credentials such as a username and password or a key pair

ssh your_username@remote-machine-address

Replace "your_username" with your actual username, and "remote-machine-address" with the provided address.

If you're using key-based authentication, it could look like this:

ssh -i /path/to/your/private/key your_username@remote-machine-address

Don't worry if you don't see characters appearing as you type; it's a security feature.
Once it is done, this allows administrators to execute commands, manage files, and perform various administrative tasks on the remote machine.

File Permissions and Ownership

Operating systems offer access rights to files and directories in the shell environment. This determines what actions can be taken on the files and directories. This allows remote workers to access files from a different system. You can use ls -l command to view the permission settings of a file.
There are three basic permissions types :

Read(r): This allows a user to view a directory or file content.
Write(w): This allows a user to modify or delete a file, as well as create or remove files within a directory.
Execute(x): This grants permission for file execution.

File permissions are set for three categories of users:

Owner: The user who created the file or directory.
Group: A group of users.
Others: Anyone who does not fall into the owner or group categories.

  - rwx   rwx     rwx
    |      |       |
   owner   group   users

This means the owner, group, and others can read, write, and execute the file. The - indicates it is a file, d indicates a directory.
rw-rw-rw- means the owner, group, and others can only read and write the file.

File permissions can also be represented
numerically:
Numeric Representation is a three-digit number where each digit corresponds to a permission type and user category.

 -rwx      rwx    r--
    |      |       |
   111     111     100

111 in binary = 7, 100 in binary = 4
Therefore -rwx rwx r-- is represented numerically as 774.

Changing Permissions

The chmod command is used to modify file permissions. You can change permissions either numerically or symbolically.
Numerical example: chmod 774 myfile
Symbolic example: chmod u+rwx myfile

Changing Ownership (chown)

The chown command is used to change the owner and group of a file or directory.

chown username filename
This assigns ownership of filename to the user named username

chown username:groupname filename
This assigns ownership of filename to user named username and sets the group ownership to groupname

chown -R username:groupname directory
This recursively changes the ownership of all files and subdirectories within directory to username and groupname.

Input and Output Redirection

There are three standard streams that are used for input and output in the Linux operating system.

Standard Input(stdin)
Standard Output(stdout)
Standard Error(stderr)

Standard Input (stdin): This is the default stream where a program reads its input. By default, stdin is the keyboard.
Standard Output (stdout): This is the default stream where a program writes its output. By default, stdout is the terminal.
Standard Error (stderr): This is the default stream where a program writes its error messages. By default, stderr is also the terminal, separate from stdout to allow error messages to be redirected separately.

Redirection is an essential technique for manipulating the input and output of a command.
Redirection allows you to redirect where the output of a command goes and where a command gets its input.

Output redirection: The > symbol is used to redirect the output of a command to a file. If the file doesn't exist, it's created; if it exists, it's overwritten. Example: cat myfile > newfile will redirect the contents of myfile to newfile.
Append redirection: The >> symbol is similar to the output redirection but instead of overwriting the file, it adds the output to the end of an existing file. Example: cat myfile >> newfile will add the contents of myfile to the end of the newfile contents.
Input redirection: < redirects the input from a file.

Example: sort < unsorted_list.txt takes input from unsorted_list.txt and sorts it.
Error Redirection (2> and 2>>):

2> redirects stderr to a file, overwriting the file if it exists.
2>> appends stderr to a file if it exists, or creates a new file if it doesn't.
Example: command 2> error_log.txt redirects the error messages of command to error_log.txt.

Pipes (|)

A pipe | is used to send the output of one command as the input to another. This is used to connect commands.
ls -l | grep "file"
In the above example, the ls -l command list files in detailed format, and the output is then filtered by grep to display only lines containing the word "file."