Logging Your Steps

Welcome back gentle reader. My name is Silent-Mobius, AKA Alex M. Schapelle, log utility to your syslog engine, and I wish to introduce you to topics of Open-Source, Linux/UNIX, DevOps and others.

In this data log, as you may guess, we’ll dwell on logs, and mostly go through on how to use those log techniques in Bash shell programming. Just like in my previous article, I'll dive into basics, as well as provide my personal suggestions on how to write, save, rotate and manage logs, in hopes that you'll find it useful. Thus, let us begin:

What is Logging ?

In computing, logging is the act of keeping a log of events that occur in a computer system, such as problems, errors or just information on current operations. These events may occur in the operating system or in other software. A message or log entry is recorded for each such event. These log messages can then be used to monitor and understand the operation of the system, to debug problems, or during an audit. Logging is particularly important in multi-user software, to have a central overview of the operation of the system.

Yet what are reason for using logs?

In short, yet precise answer, provided to me by a simultaneous running process called Y-Go-lan:

• Debugging: Logs help in identifying issues by providing insights into the script’s execution flow and errors.
• Auditing: For system administrators and developers, logs can track script execution for auditing purposes.
• Monitoring: Logs help to monitor the health and performance of long-running scripts.
• Documentation: They provide a history of script activities, especially useful when troubleshooting or verifying script actions.

What does bash script has to do with it ?

When it comes to script languages, Python and Ruby in particular, The logging mechanisms, are built-in in terms of external library, that enable all the reasons mentioned above. Yet it does not include bash, mostly because, developers and scripting language developer do not consider bash, as scripting language, or as a zombie process once told me: "Bash is just a glue", and no reboot can help that zombie process.
Another example of shell/bash significance would be scripting language native libraries: they rely on bash/shell when it comes to working with OS. For single threaded ones: native scripting language libraries run shell commands as part of working with OS.

Let's try to present the solution on the articles headline.

What is left behind ?

When we write shell program, we usually try to create some sort of automatic printing and saving method involving I/O redirections, OS api or utilities provided by OS to document the tasks executed during our program.
Logging is an essential part of shell scripting, especially when it comes to debugging, auditing, and tracking system activities. In shell scripts, logging allows you to capture relevant information, such as script execution details, error messages, or system status, into log files. This article will explore various techniques and built-in tools available for logging in shell scripting, particularly in Bash.

Common Logging Techniques in Shell Scripts

• Basic Logging to a File

The simplest way to log information in a shell script is by redirecting the output to a log file using redirection operators. You can log both standard output (stdout) and standard error (stderr) to a file.

#!/bin/bash

# Log file location
LOG_FILE="/var/log/myscript.log"

# Log function
function log_message() {
  echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}

# Example usage
log_message "Script started."

# Simulate a task
if [[ -d "/some/directory" ]]; then
  log_message "Directory exists."
else
  log_message "Directory does not exist."
fi

log_message "Script ended."

Explanation:

• The log_message function appends messages with a timestamp to the log file.
• This method logs the output by appending (>>) the log message to the file.

Logging with Timestamps

Adding timestamps helps in tracking when an event occurred. The date command is used to append the current date and time to log entries.

#!/bin/bash

LOG_FILE="/var/log/myscript.log"

function log_message() {
  echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}

log_message "Script started."
log_message "Performing some tasks..."

This ensures each log entry is properly timestamped.

Logging Standard Output and Error Separately

To separate standard output (stdout) and standard error (stderr), you can use different redirection operators for each.

#!/bin/bash

LOG_FILE="/var/log/myscript.log"
ERROR_LOG_FILE="/var/log/myscript_error.log"

# Log standard output
echo "This is a normal message" >> "$LOG_FILE"

# Log errors
echo "This is an error message" >&2 >> "$ERROR_LOG_FILE"

Alternatively, redirect both stdout and stderr to the same log file:

#!/bin/bash

LOG_FILE="/var/log/myscript.log"

# Redirect both stdout and stderr
exec >> "$LOG_FILE" 2>&1

echo "This is a normal message"
echo "This is an error message" >&2

Using tee for Real-Time Logging

The tee command allows you to log output to a file while simultaneously displaying it on the terminal. This is useful if you want to monitor the script’s progress in real-time while keeping a log.

#!/bin/bash

LOG_FILE="/var/log/myscript.log"

# Use tee to log output and display it on the terminal
echo "Starting script..." | tee -a "$LOG_FILE"

In this case, the -a flag appends the output to the log file.

Logging Error Messages Using trap

The trap command allows you to catch signals and errors in your script and handle them. You can use trap to log when an error occurs or when the script exits.

#!/bin/bash

LOG_FILE="/var/log/myscript.log"

# Function to log errors
function log_error() {
  echo "$(date '+%Y-%m-%d %H:%M:%S') - Error occurred. Exiting." >> "$LOG_FILE"
}

# Trap errors and exit signals
trap log_error ERR

# Simulate an error
echo "This will cause an error"
non_existent_command

In this case, if any command exits with a non-zero status, the log_error function will be called.

This sends the log message to the system’s syslog service, where it can be filtered and stored based on the system's logging configuration.

Built-in Logging Tools in Bash

Bash offers several tools and commands for enhanced logging:

• logger Command

The logger command allows you to log messages to the system’s syslog service.

logger "This is a log message to syslog."

This is a useful tool for system administrators who want to log messages that are automatically handled by the system’s logging framework.

Log Rotation for logger (Using logrotate)

For long-running scripts that produce a lot of logs, log rotation is essential to prevent log files from growing indefinitely. You can use the logrotate tool to manage log files.

Install logrotate if not already installed:

sudo apt-get install logrotate

Create a logrotate configuration file:

Example configuration for /etc/logrotate.d/myscript:

    /var/log/myscript.log {
        weekly
        rotate 4
        compress
        delaycompress
        notifempty
        create 0644 root root
    }

Note: This configuration rotates the log file weekly, keeps 4 archives, and compresses old logs.

• exec for Redirecting Output

The exec command is a built-in Bash command that can redirect output for the entire script. For example, redirecting both stdout and stderr:

#!/bin/bash

LOG_FILE="/var/log/myscript.log"
exec >> "$LOG_FILE" 2>&1

echo "This is a normal message"
echo "This is an error message" >&2

This is useful when you want to log everything from the script without needing to manually redirect output for each command.

Best Practices for Logging

• Use Log Levels: Implement different log levels such as INFO, WARNING, ERROR, etc., to categorize log messages and make logs more readable.
• Use Unique Log Files: Use different log files for different types of logs (e.g., info.log, error.log, debug.log).
• Rotate Logs Regularly: Use logrotate or a similar mechanism to manage log file sizes and prevent disk space issues.
• Keep Log Files Secure: Make sure log files are secured with appropriate permissions to prevent unauthorized access.

As A Conclusion

Logging in shell scripting is crucial for tracking script execution, debugging issues, and maintaining logs for future reference. There are multiple techniques available, from basic file redirection to advanced log management tools like logger, trap, and logrotate. Using the right approach to logging ensures that your scripts are easier to maintain, troubleshoot, and audit.
Hope that this article was full of useful information, and you have upgraded your in-memory database. Like, comment and share, and also : Do Try To Have Fun.

Thank you