Master File Transfers and Backups with Rsync and SCP: A Beginner’s Guide

Introduction

Managing file transfer and backups are en essential skill everyone should must learn. Two most popular tools for this purpose are scp and rsync. While scp is simple to securely send or receive file to and from remote server to local server, rsync is popular for data backup, data synchronization and server mirroring. This blog will help you understand how to get started with both the tools and choose which one to use wisely.

1. rsync:
rsync is a fast and flexible tool to copy and synchronize files and directories, either locally or between systems. It is commonly used for backups, mirroring, and transferring data efficiently.

2. SCP (Secure Copy Protocol):
A simple tool for securely copying files between systems over SSH. It encrypts data during transit, ensuring security. Best for quick, straightforward file transfers.

When to Use

Rsync:
For syncing large datasets efficiently.
When saving bandwidth is important.
To mirror directories with file attributes preserved.

SCP:
For secure, quick file transfers.
When simplicity is more important than advanced features.
For smaller or occasional transfers.

Installing rsync

rsync is by default installed on Linux system, in case rsync is not available try installing using:

sudo apt install rsync

Most useful rsync option:

• -a : Archive Mode for recursive copying, preserving permissions,timestamps, and other attributes
• -v : Verbose Mode, shows details of what happening
• -z : Compress Mode, allows to compress the file during transfer to save the bandwidth.
• -h : Makes numbers easier to read (e.g., file sizes in KB, MB)
• -P : Displays the progress of the file transfer.
• -r : Recursively copies files and directories, but doesn’t preserve timestamps or permissions.
• --dry-run : Allow user to simulate the rsync transfer without copying or syncing the files.

Note: You will need to have the username ,password and IP address of the remove server where you want to sync,copy or mirror your files using rsync and scp.

Rsync for file Transfer

1. Local Transfer

1. Remote Transfer

1. Remote Synchronization

Using --backup-dir for enhanced backup system:

This option will allow the user to back up the file and also keep the record of previous version of the file for effective backup system.

On the remote backup-server it can be seen that there are two file recorded, on /Archive folder a back-file.txt is create which hold the previous version of the backup-file. On the other hand, there is also a recent backupfile updated with recent changes.

Converting the command into simple backup script.

This is the simple script to backup file with rsync.

#!/bin/bash
#@Author: Rupesh Sharma

SOURCE_FILE="/home/vagrant/backup/backup-file.txt"
BACKUP_SERVER="vagrant@192.168.56.20:/home/vagrant/rsync"
ARCHIVE_DIR="/home/vagrant/rsync/Archive"
DATE_TIME=$(date +%Y-%m-%d-%H:%M:%S)
rsync -vazh --backup-dir="$ARCHIVE_DIR/$DATE_TIME" "$SOURCE_FILE" "$BACKUP_SERVER"

echo "Backup Completed for : $DATE_TIME

"

SCP(Secure Copy Protocol)

Similar to rsync SCP is by default installed on Linux system, in case SCP is not available try installing using:

sudo apt install openssh-client

1. Transferring file to Remote server

Transferring file using scp for local system to remote system.
verifying the file transferred from local to remote system.

1. Pulling file from Remote server to local server using local server. Pulling file from Remote server to local server using local server

Conclusion

To conclude with, both rsync and scp are indispensable tools for file transfer and synchronization. While scp is ideal for simple, secure transfers, rsync excels in efficiency, making it perfect for backups and large-scale synchronizations.