Welcome to Butterfly Backup’s documentation!
Indices and tables
Requirements
Butterfly Backup is a simple wrapper of rsync written in python; this means the first requirements is python3.8 or higher. The other requirement is a openssh and rsync (version 2.5 or higher). Ok, let’s go!
Test
If you want to try or test Butterfly Backup before installing it, run the test:
arthur@heartofgold$ git clone https://github.com/MatteoGuadrini/Butterfly-Backup.git
arthur@heartofgold$ cd Butterfly-Backup
arthur@heartofgold$ bash test_bb.py
Installation
Install Butterfly Backup is very simple; run this:
arthur@heartofgold$ git clone https://github.com/MatteoGuadrini/Butterfly-Backup.git
arthur@heartofgold$ cd Butterfly-Backup
arthur@heartofgold$ sudo python3 setup.py install -f # -f is for upgrade
arthur@heartofgold$ bb --help
arthur@heartofgold$ man bb
The upgrade is also simple; type the same commands.
Core concept
Butterfly Backup it is a server to client tool. It must be installed on a server (for example a workstation PC), from where it will contact the clients on which it will perform the backup. Also for the restore the concept is the same; the server pushes files into the client.
Backups are organized according to precise cataloguing; this is an example:
arthur@heartofgold$ tree destination/of/backup
.
├── destination
│ ├── hostname or ip of the PC under backup
│ │ ├── timestamp folder
│ │ │ ├── backup folders
│ │ │ ├── backup.log
│ │ │ └── restore.log
│ │ ├─── general.log
│ │ └─── symlink of last backup
│
├── export.log
├── backup.list
└── .catalog.cfg
This is how the operating system sees the backups on the file system:
destination: it is the destination of all backup of all machine under Butterfly Backup system; this is root.
hostname or ip of the PC under backup: it is the root of every machine under backup; contains all the backups.
timestamp folder: this is a single backup folder; contains all the data that have been backed up.
backup.log: if enabled, it is the log generated by rsync for backup operation.
restore.log: if enabled, it is the log generated by rsync for restore operation.
general.log: if enabled, it is the general log of the machine; contains all the info, warnings and errors generated during the backup.
symlink of last backup: this is a symbolic link of the last backup made. Isn’t supported if you use an MS-DOS file system.
backup.list: if enabled, it is the log generated by list operation.
export.log: if enabled, it is the log generated by export operation.
.catalog.cfg: is the log that contains the catalog of all backups. The file is the reference of Butterfly Backup to perform future backups.
Important
Be careful to not delete .catalog.cfg file. If you want to initialize the backup catalog, just run this command: bb config --init /catalog/path
.
Butterfly Backup has, in its core, six main operations:
arthur@heartofgold$ bb --help
usage: bb [-h] [--verbose] [--log] [--dry-run] [--force] [--no-color] [--explain-error] [--version] {config,backup,restore,archive,list,export} ...
Butterfly Backup
options:
-h, --help show this help message and exit
--verbose, -v Enable verbosity
--log, -l Create logs
--dry-run, -N Dry run mode
--force, -O Force an action without prompt
--no-color, -w Remove color into terminal
--explain-error, -x Print python traceback
--version, -V Print version
action:
Valid action
{config,backup,restore,archive,list,export}
Available actions
config Configuration options
backup Backup options
restore Restore options
archive Archive options
list List options
export Export options
Valid action
config: operations involving OpenSSH and its configurations (not mandatory).
backup: transactions that call rsync, in a single process or across multiple processes in case of a list.
restore: transactions that call rsync for pushing files to a client.
archive: transaction that zip old backups and move them to a new destination.
list: query the catalog.
export: export a single backup to other path.
It also has three flags that can be very useful, especially in case of error.
- -h, --help
show help message and exit
- --verbose, -v
Enable verbosity
- --log, -l
Create a log
- --dry-run, -N
Dry run mode, test your command
- --force, -O
Force an action without prompt
- --no-color, -w
Remove color into terminal
- --explain-error, -x
Print python traceback
- --version, -V
Print version
Config
Configuration mode is very simple; If you’re already familiar with the exchange keys and OpenSSH, you probably won’t need it. If you don’t want to discuss the merits of exchange keys and start, then go ahead. First, you must create a configuration (rsa keys). Let’s see how to go about looking at the help:
arthur@heartofgold$ bb config --help
usage: bb config [-h] [--verbose] [--log] [--dry-run] [--force] [--no-color] [--explain-error] [--new | --remove | --init CATALOG | --delete-host CATALOG HOST | --clean CATALOG | --delete-backup CATALOG ID]
[--deploy DEPLOY_HOST] [--user DEPLOY_USER]
options:
-h, --help show this help message and exit
--verbose, -v Enable verbosity
--log, -l Create logs
--dry-run, -N Dry run mode
--force, -O Force an action without prompt
--no-color, -w Remove color into terminal
--explain-error, -x Print python traceback
Init configuration:
--new, -n Generate new configuration
--remove, -r Remove exist configuration
--init, -i CATALOG Reset CATALOG file. Specify path of backup folder.
--delete-host, -D CATALOG HOST
Delete all entry for a single HOST in CATALOG.
--clean, -c CATALOG Cleans the CATALOG if it is corrupt, setting default values.
--delete-backup, -b CATALOG ID
Delete specific backup ID from CATALOG
Deploy configuration:
--deploy, -d DEPLOY_HOST
Deploy configuration to client: hostname or ip address
--user, -u DEPLOY_USER
User of the remote machine
Two macro-options are available:
- Init configuration: Generate new or remove configuration.
- --new, -n
Generate new configuration.
- --remove, -r
Remove exist configuration.
- --init, -i
Reset catalog file. Specify path of backup folder.
- --delete-host, -D
Delete all entry for a single host in catalog.
- --clean, -c
Cleans the catalog if it is corrupt, setting default values.
- --delete-backup, -b
Delete specific backup id from catalog.
- Deploy configuration: Deploy configuration to client: hostname or ip address.
- --deploy, -d
Deploy configuration to client: hostname or ip address.
- --user, -u
User of the remote machine.
At this point, we create the configuration:
arthur@heartofgold$ bb config --new
warning: Private key ~/.ssh/id_rsa exists
If you want to use the existing key, run "bb config --deploy name_of_machine", otherwise to remove it, run "bb config --remove"
In this case, the RSA key already exists. Now try delete and create a new keys:
arthur@heartofgold$ bb config --remove
Are you sure to remove existing rsa keys? To continue [Y/N]? y
success: Removed configuration successfully!
arthur@heartofgold$ bb config --new
success: New configuration successfully created!
Once you have created the configuration, keys should be installed (copied) on the hosts you want to backup.
arthur@heartofgold$ bb config --deploy host1
Copying configuration to host1; write the password:
/usr/bin/ssh-copy-id: debug: Source of key(s) to be installed: "/home/arthur/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: debug: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: debug: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
arthur@host1's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'arthur@host1'"
and check to make sure that only the key(s) you wanted were added.
success: Configuration copied successfully on host1!
This command will try to copy the configuration with the current user. If you want to use a different user (e.g.: root), run this:
arthur@heartofgold$ bb config --deploy host1 --user root
Copying configuration to host1; write the password:
/usr/bin/ssh-copy-id: debug: Source of key(s) to be installed: "/home/arthur/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: debug: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: debug: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@host1's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@host1'"
and check to make sure that only the key(s) you wanted were added.
success: Configuration copied successfully on host1!
Cygwin on Windows
In order to run Butterfly Backup on Windows hosts, you need to install cygwin. Download https://cygwin.com/ cygwin and follow this instructions to install the necessary packages.
:: INSTALL SOFTWARE
:: Automates cygwin installation
SETLOCAL
:: Change to the directory of the executing batch file
CD %~dp0
:: Configure our paths
SET SITE=http://mirrors.kernel.org/sourceware/cygwin/
SET LOCALDIR=%LOCALAPPDATA%/cygwin
SET ROOTDIR=C:/cygwin
:: These are the packages we will install (in addition to the default packages)
SET PACKAGES=mintty,wget,ctags,diffutils,openssh,rsync,cygrunsrv,nano
SET PACKAGES=%PACKAGES%,gcc-core,make,automake,autoconf,readline,libncursesw-devel,libiconv,zlib-devel,gettext
:: Do it!
setup.exe -q -D -L -d -g -o -s %SITE% -l "%LOCALDIR%" -R "%ROOTDIR%" -C Base -P %PACKAGES% >nul
setx PATH "%PATH%;/cygdrive/c/cygwin/bin;C:\cygwin\bin" /m
ENDLOCAL
C:
chdir C:\cygwin\bin
mkpasswd -cl > C:\cygwin\etc\passwd
bash ssh-host-config -y
cygrunsrv -S sshd
Important
Verify if port 22 is in LISTEN: run netstat -aon | findstr :22
If you want to initialize or reset the catalog file, run this:
arthur@heartofgold$ bb config --init /mnt/backup -v
Important
This command preserves existing backups on the file system. It will eliminate only those archived or deleted.
Backup
There are two backup modes: single and bulk. Let’s see how to go about looking at the help:
arthur@heartofgold$ bb backup --help
usage: bb backup [-h] [--verbose] [--log] [--dry-run] [--force] [--no-color] [--explain-error] (--computer HOSTNAME | --list LIST) --destination DESTINATION [--mode {full,incremental,differential,mirror}]
(--data {user,config,application,system,log} [{user,config,application,system,log} ...] | --custom-data CUSTOMDATA [CUSTOMDATA ...] | --file-data FILEDATA) [--user USER] --type {unix,windows,macos}
[--compress] [--retention [DAYS [NUMBER ...]]] [--parallel PARALLEL] [--timeout TIMEOUT] [--skip-error] [--rsync-path RSYNC] [--bwlimit BWLIMIT] [--ssh-port PORT] [--exclude EXCLUDE [EXCLUDE ...]]
[--start-from ID]
options:
-h, --help show this help message and exit
--verbose, -v Enable verbosity
--log, -l Create logs
--dry-run, -N Dry run mode
--force, -O Force an action without prompt
--no-color, -w Remove color into terminal
--explain-error, -x Print python traceback
Backup options:
--computer, -c HOSTNAME
Hostname or ip address to backup
--list, -L LIST File list of computers or ip addresses to backup
--destination, -d DESTINATION
Destination path
--mode, -m {full,incremental,differential,mirror}
Backup mode
--data, -D {user,config,application,system,log} [{user,config,application,system,log} ...]
Data of which you want to backup
--custom-data, -C CUSTOMDATA [CUSTOMDATA ...]
Custom path of which you want to backup
--file-data, -F FILEDATA
File with custom path of which you want to backup
--user, -u USER Login name used to log into the remote host (being backed up)
--type, -t {unix,windows,macos}
Type of operating system to backup
--compress, -z Compress data
--retention, -r [DAYS [NUMBER ...]]
First argument are days of backup retention. Second argument is minimum number of backup retention
--parallel, -p PARALLEL
Number of parallel jobs
--timeout, -T TIMEOUT
I/O timeout in seconds
--skip-error, -e Skip error
--rsync-path, -R RSYNC
Custom rsync path
--bwlimit, -b BWLIMIT
Bandwidth limit in KBPS.
--ssh-port, -P PORT Custom ssh port.
--exclude, -E EXCLUDE [EXCLUDE ...]
Exclude pattern
--start-from, -s ID Backup id where start a new backup
- Backup options
- --computer, -c
Select the ip or hostname where to perform backup.
- --list, -l
Select the file list of the ip or hostnames, where to perform backups.
[File_list.txt]
host1
192.168.0.1
host2
…
- --destination, -d
Select the destination folder (root).
- --mode, -m
Select how rsync perform backup:
- full:
Complete (full) backup.
- incremental:
Incremental backup is based on the latest backup (the same files are linked with hard link). A Full backup is executed if this mode fails to find one.
- differential:
Incremental backup is based on the latest Full backup (the same files are linked with hard link). A Full backup is executed if this mode fails to find one.
- mirror:
Complete mirror backup. If a file in the source no longer exists, BB deletes it from the destination.
- --data, -D
Select the type of data to put under backup: The values change depending on the type of operating system:
user -> folder containing the home.
config -> folder containing the configurations of the machine.
log -> folder containing the log.
application -> folder containing applications.
system -> the entire system starting from the root.
- --custom-data
Select the absolute paths to put under backup.
- --file-data
Select a file with the absolute paths to put under backup.
[File_list.txt]
/path1
/path2/with spaces/
“/path3/with quotes”
…
- --user, -u
Login name used to log into the remote host (being backed up)
- --type, -t
Type of operating system to put under backup:
unix -> All UNIX os (Linux, BSD, Solaris).
windows -> Windows Vista or higher with cygwin installed.
macos -> MacOSX 10.8 or higher.
- --compress, -z
Compresses the data transmitted.
- --retention, -r
Number of days for which you want to keep your backups and minimum number of backup retention (optional).
The second number is a minimum number of backup which you want keep.
- --parallel, -p
Maximum number of concurrent rsync processes. By default is 5 jobs.
- --timeout, -T
Specify number of seconds of I/O timeout.
- --skip-error, -e
Skip error. Quiet mode.
- --rsync-path, -R
Select a custom rsync path.
- --bwlimit, -b
Bandwidth limit in KBPS.
- --ssh-port, -P
Custom ssh port.
- --exclude, -E
Exclude pattern. Follow rsync “Exclude Pattern Rules”
- --start-from, -s
The new backup is based on another backup, specified by its ID.
Flowchart of the differences between Differential and Incremental backup:
+----------+ +----------+ +----------+
| | | | | |
| Full | <-+ Inc. | <-+ Inc. |
| | | | | |
+----------+ +----------+ +----------+
+----------+ +----------+ +----------+
| | | | | |
| Full | <-+ Dif. | | Dif. |
| | | | | |
+----+-----+ +----------+ +----+-----+
^ |
| |
+-----------------------------+
Single backup
The backup of a single machine consists in taking the files and folders indicated in the command line, and put them in the cataloging structure indicated above.
This is a few examples:
arthur@heartofgold$ bb backup --computer host1 --destination /mnt/backup --data User Config --type MacOS
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
info: Start backup on host1
success: Command rsync -ah --no-links arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_28
Important
Without specifying the “mode” flag, Butterfly Backup by default selects Incremental mode
arthur@heartofgold$ bb backup --computer host1 --destination /mnt/backup --mode Full --data User Config --type MacOS --user root
root@host1's password:
info: Start backup on host1
success: Command rsync -ah --no-links root@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_30
arthur@heartofgold$ bb backup --computer host1 --destination /mnt/backup --data User Config --type MacOS --verbose --log
debug: Build a rsync command
debug: Last full is 2018-08-08 10:30:32
debug: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_28 -vP
debug: Create a folder structure for MacOS os
debug: Include this criteria: :/Users :/private/etc
debug: Destination is /mnt/backup/host1/2018_08_08__10_42
info: Start backup on host1
debug: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_42
receiving file list ...
39323 files to consider
Users/
...
...
sent 18.91K bytes received 1.59M bytes 25.74K bytes/sec
total size is 6.67G speedup is 4143.59
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_42
Note
If the backup destination is an MS-DOS fs, it will not support symlink creation. The message that will be returned is as follows:
warning: MS-DOS file system doesn't support symlink file.
Bulk backup
Bulk backup follows the same logic and the same options as a single backup, but accepts a file containing the names or ips of the machine to be backed up.
This is a few examples:
arthur@heartofgold$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
error: SSH connection failed on host2:22
error: SSH connection failed on host3:22
info: Start backup on host1
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_50
Important
Port 22 (OpenSSH standard) is tested in order to verify the reachability of the machine. If the machine is not reachable, an error is generated:
error: SSH connection failed on host2:22
This example, is the same as the previous one, with the only difference being that the parallel flag is specified at 2. This means that maximum two backup jobs will run at the same time. When a first process ends, another one is started.
arthur@heartofgold$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --log
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
error: SSH connection failed on host2:22
error: SSH connection failed on host3:22
info: Start backup on host1
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
This is the same example but specifying a retention at 3 (days). This means that in doing so we want to keep only 3 days backups.
arthur@heartofgold$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --retention 3 --log --verbose
debug: Build a rsync command
debug: Last full is 2018-08-08 10:30:32
debug: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 -vP
debug: Create a folder structure for MacOS os
debug: Include this criteria: :/Users :/private/etc
debug: Destination is /mnt/backup/host1/2018_08_08__10_30
info: Start backup on host1
error: SSH connection failed on host2:22
error: SSH connection failed on host3:22
debug: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
receiving file list ...
39323 files to consider
Users/
...
...
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
debug: Check cleanup this backup aba860b0-9944-11e8-a93f-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_28
success: Cleanup /mnt/backup/host1/2018_08_08__10_28 successfully.
debug: Check cleanup this backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30
debug: No cleanup backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30
Here we find the same example above but specifying a retention at 2 (days) and 5 backup copies. This means that at least 5 backup copies will be kept, even if they are older than two days.
arthur@heartofgold$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --retention 3 5 --log --verbose
debug: Build a rsync command
debug: Last full is 2018-08-08 10:30:32
debug: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 -vP
debug: Create a folder structure for MacOS os
debug: Include this criteria: :/Users :/private/etc
debug: Destination is /mnt/backup/host1/2018_08_08__10_30
info: Start backup on host1
error: SSH connection failed on host2:22
error: SSH connection failed on host3:22
debug: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
receiving file list ...
39323 files to consider
Users/
...
...
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
debug: Check cleanup this backup aba860b0-9944-11e8-a93f-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_28
success: Cleanup /mnt/backup/host1/2018_08_08__10_28 successfully.
debug: Check cleanup this backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30
debug: No cleanup backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30
Important
If there is only one Full in our catalog, the retention policy will never delete that Full backup, but only Increments. We recommend doing more Full for better retention management.
List
When we run backup commands, a catalog is created. This serves both for future backups and all the restores that are made through Butterfly Backup. To query this catalog, the list command exists.
arthur@heartofgold$ bb list --help
usage: bb list [-h] [--verbose] [--log] [--dry-run] [--force] [--no-color] [--explain-error] --catalog CATALOG [--backup-id ID | --archived | --cleaned | --last | --detail ID] [--oneline] [--computer HOSTNAME]
[--only-id]
options:
-h, --help show this help message and exit
--verbose, -v Enable verbosity
--log, -l Create logs
--dry-run, -N Dry run mode
--force, -O Force an action without prompt
--no-color, -w Remove color into terminal
--explain-error, -x Print python traceback
List options:
--catalog, -C CATALOG
Folder where is catalog file
--backup-id, -i ID Backup-id of backup
--archived, -a List only archived backup
--cleaned, -c List only cleaned backup
--last, -L Only last backup
--detail, -d ID List detail of file and folder of specific backup-id
--oneline, -o One line output
--computer, -H HOSTNAME
List only match hostname or ip
--only-id, -y List only id
- List options
- --catalog, -C
Select the backups folder (root).
- --backup-id, -i
Select backup id in the catalog.
- --archived, -a
List only archived backups.
- --cleaned, -c
List only cleaned backups.
- --last, -L
Only last backup is selected with details.
- --computer, -H
List only match hostname or ip.
- --detail, -d
List detail of file and folder of specific backup-id.
- --oneline, -o
One line and concise output.
- --only-id, -y
List only id
First, let’s query the catalog:
arthur@heartofgold$ bb list --catalog /mnt/backup
BUTTERFLY BACKUP CATALOG
Backup id: aba860b0-9944-11e8-a93f-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:28:12
Backup id: cc6e2744-9944-11e8-b82a-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:30:59
Backup id: dd6de2f2-9a1e-11e8-82b0-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:58:59
Press q
for exit. Then we select a backup-id:
arthur@heartofgold$ bb list --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0
Backup id: dd6de2f2-9a1e-11e8-82b0-005056a664e0
Hostname or ip: host1
Type: Incremental
Timestamp: 2018-08-08 10:58:59
Start: 2018-08-08 10:58:59
Finish: 2018-08-08 11:02:49
OS: MacOS
ExitCode: 0
Path: /mnt/backup/host1/2018_08_08__10_58
List: backup.log
etc
Users
If you want to export the catalog list instead, include the log flag:
arthur@heartofgold$ bb list --catalog /mnt/backup --log
arthur@heartofgold$ cat /mnt/backup/backup.list
Instead if you want to see the details of your backup, add details flag:
arthur@heartofgold$ bb list --catalog /mnt/backup --detail dd6de2f2-9a1e-11e8-82b0-005056a664e0
Now that we have identified a backup, let’s proceed with the restore
Restore
The restore process is the exact opposite of the backup process. It takes the files from a specific backup and push it to the destination computer.
arthur@heartofgold$ bb restore --help
usage: bb restore [-h] [--verbose] [--log] [--dry-run] [--force] [--no-color] [--explain-error] --catalog CATALOG (--backup-id ID | --last) [--user USER] --computer HOSTNAME [--root-dir ROOT_DIR]
[--type {unix,windows,macos}] [--timeout TIMEOUT] [--mirror] [--acl] [--skip-error] [--rsync-path RSYNC] [--bwlimit BWLIMIT] [--ssh-port PORT] [--exclude EXCLUDE [EXCLUDE ...]]
[--files FILES [FILES ...]]
options:
-h, --help show this help message and exit
--verbose, -v Enable verbosity
--log, -l Create logs
--dry-run, -N Dry run mode
--force, -O Force an action without prompt
--no-color, -w Remove color into terminal
--explain-error, -x Print python traceback
Restore options:
--catalog, -C CATALOG
Folder where is catalog file
--backup-id, -i ID Backup-id of backup
--last, -L Last available backup of the same host
--user, -u USER Login name used to log into the remote host (where you're restoring)
--computer, -c HOSTNAME
Hostname or ip address to perform restore
--root-dir, -r ROOT_DIR
Root directory to perform restore
--type, -t {unix,windows,macos}
Type of operating system to perform restore
--timeout, -T TIMEOUT
I/O timeout in seconds
--mirror, -m Mirror mode
--acl, -a Preserve ACLs
--skip-error, -e Skip error
--rsync-path, -R RSYNC
Custom rsync path
--bwlimit, -b BWLIMIT
Bandwidth limit in KBPS.
--ssh-port, -P PORT Custom ssh port.
--exclude, -E EXCLUDE [EXCLUDE ...]
Exclude pattern
--files, -f FILES [FILES ...]
Restore only specified files
- Restore options
- --catalog, -C
Select the backups folder (root).
- --backup-id
Select backup id in the catalog.
- --last, -L
Select last available backup in the catalog for same hostname or ip address.
- --user, -u
User of remote machine where you want to restore files.
- --computer, -c
Select the ip or hostname where to perform restore.
- --root-dir, -r
Root directory to perform restore.
- --type, -t
Type of operating system to put under backup:
Unix -> All UNIX os (Linux, BSD, Solaris).
Windows -> Windows Vista or higher with cygwin installed.
MacOS -> MacOSX 10.8 or higher.
- --timeout, -T
Specify number of seconds of I/O timeout.
- --mirror, -m
Mirror mode. If a file or folder not exist in destination, will delete it. Overwrite files.
- --acl, -a
Preserve ACLs
- --skip-error, -e
Skip error. Quiet mode.
- --rsync-path, -R
Select a custom rsync path.
- --bwlimit, -b
Bandwidth limit in KBPS.
- --ssh-port, -P
Custom ssh port.
- --exclude, -E
Exclude pattern. Follow rsync “Exclude Pattern Rules”.
- --files, -f
Restore only specified files.
This is a few examples:
This command perform a restore on the same machine of the backup:
arthur@heartofgold$ bb restore --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0 --computer host1 --log
info: Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
info: Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc arthur@host1:/restore_2018_08_08__10_58
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* arthur@host1:/Users
Important
Without specifying the “type” flag that indicates the operating system on which the data are being retrieved, Butterfly Backup will select it directly from the catalog via the backup-id.
Now, select the last available backup on catalog; run this:
arthur@heartofgold$ bb restore --catalog /mnt/backup --last --computer host1 --log
info: Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
info: Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc arthur@host1:/restore_2018_08_08__10_59
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* arthur@host1:/Users
This example, is the same as the previous one, but restore to other machine and other operating system:
arthur@heartofgold$ bb restore --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0 --computer host2 --type Unix --log --verbose
debug: Build a rsync command
debug: Command flags are: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log
info: Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
debug: Build a rsync command
debug: Command flags are: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log
info: Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
info: Start restore on host2
debug: rsync command: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc/* arthur@host2:/etc
info: Start restore on host2
debug: rsync command: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* arthur@host2:/home
building file list ...
26633 files to consider
...
...
sent 777.53K bytes received 20 bytes 62.20K bytes/sec
total size is 6.66G speedup is 8566.41
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc/* arthur@host2:/etc
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* arthur@host2:/home
Archive
Archive operations are used to store backups by saving disk space. Backups older than n days are compressed into a zip file.
arthur@heartofgold$ bb archive --help
usage: bb archive [-h] [--verbose] [--log] [--dry-run] [--force] [--no-color] [--explain-error] --catalog CATALOG [--days DAYS] --destination DESTINATION
options:
-h, --help show this help message and exit
--verbose, -v Enable verbosity
--log, -l Create logs
--dry-run, -N Dry run mode
--force, -O Force an action without prompt
--no-color, -w Remove color into terminal
--explain-error, -x Print python traceback
Archive options:
--catalog, -C CATALOG
Folder where is catalog file
--days, -D DAYS Number of days of archive retention
--destination, -d DESTINATION
Archive destination path
- Archive options
- --catalog, -C
Select the backups folder (root).
- --days, -D
Number of days for which you want to keep your backups. Default is 30.
- --destination, -d
New destination for compress zipped backup.
Archive old backup of 3 days:
arthur@heartofgold$ bb archive --catalog /mnt/backup/ --days 3 --destination /mnt/archive/ --verbose --log
debug: Check archive this backup f65e5afe-9734-11e8-b0bb-005056a664e0. Folder /mnt/backup/host2/2018_08_08__17_50
debug: Check archive this backup 4f2b5f6e-9939-11e8-9ab6-005056a664e0. Folder /mnt/backup/host2/2018_08_04__07_26
success: Delete /mnt/backup/host2/2018_08_04__07_26 successfully.
success: Archive /mnt/backup/host2/2018_08_04__07_26 successfully.
arthur@heartofgold$ ls /mnt/archive
host1
arthur@heartofgold$ ls /mnt/archive/host1
2018_08_06__07_26.zip
The backup-id f65e5afe-9734-11e8-b0bb-005056a664e0 it is not considered, because it falls within the established time. The other, however, is zipped and deleted.
Lastly, let’s look in the catalog and see that the backup was actually archived:
arthur@heartofgold$ bb list --catalog /mnt/backup/ -i 4f2b5f6e-9939-11e8-9ab6-005056a664e0
Backup id: 4f2b5f6e-9939-11e8-9ab6-005056a664e0
Hostname or ip: host2
Type: Incremental
Timestamp: 2018-08-08 07:26:46
Start: 2018-08-08 07:26:46
Finish: 2018-08-08 08:43:45
OS: MacOS
ExitCode: 0
Path: /mnt/backup/host2/2018_08_04__07_26
Archived: True
Export
The export function is used to copy a particular backup to another path.
arthur@heartofgold$ bb export -h
usage: bb export [-h] [--verbose] [--log] [--dry-run] [--force] [--no-color] [--explain-error] --catalog CATALOG [--backup-id ID | --all] --destination DESTINATION [--mirror] [--cut] [--link PATH]
[--include INCLUDE [INCLUDE ...] | --exclude EXCLUDE [EXCLUDE ...]] [--timeout TIMEOUT] [--skip-error] [--rsync-path RSYNC] [--bwlimit BWLIMIT] [--ssh-port PORT]
options:
-h, --help show this help message and exit
--verbose, -v Enable verbosity
--log, -l Create logs
--dry-run, -N Dry run mode
--force, -O Force an action without prompt
--no-color, -w Remove color into terminal
--explain-error, -x Print python traceback
Export options:
--catalog, -C CATALOG
Folder where is catalog file
--backup-id, -i ID Backup-id of backup
--all, -A All backup
--destination, -d DESTINATION
Destination path
--mirror, -m Mirror mode
--cut, -c Cut mode. Delete source
--link, -L PATH Hard link to path
--include, -I INCLUDE [INCLUDE ...]
Include pattern
--exclude, -E EXCLUDE [EXCLUDE ...]
Exclude pattern
--timeout, -T TIMEOUT
I/O timeout in seconds
--skip-error, -e Skip error
--rsync-path, -R RSYNC
Custom rsync path
--bwlimit, -b BWLIMIT
Bandwidth limit in KBPS.
--ssh-port, -P PORT Custom ssh port.
- Export options
- --catalog, -C
Select the backups folder (root).
- --backup-id, -i
Select backup id in the catalog.
- --all, -A
All backup
- --destination, -d
Destination path.
- --mirror, -m
Mirror backup on destination.
- --cut, -c
Delete source like a move.
- --link, -L
Hard link to path.
- --include, -I
Include pattern. Follow rsync “Include Pattern Rules”
- --exclude, -E
Exclude pattern. Follow rsync “Exclude Pattern Rules”
- --timeout, -T
Specify number of seconds of I/O timeout.
- --mirror, -m
Mirror mode. If a file or folder not exist in destination, will delete it. Overwrite files.
- --skip-error, -e
Skip error. Quiet mode.
- --rsync-path, -R
Select a custom rsync path.
- --bwlimit, -b
Bandwidth limit in KBPS.
- --ssh-port, -P
Custom ssh port.
Export a backup in other directory:
arthur@heartofgold$ bb export --catalog /mnt/backup/ --backup-id f0f700e8-0435-11e9-9e78-005056a664e0 --destination /mnt/backup/export --verbose
debug: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
debug: Build a rsync command
info: Start export host1
...
debug: rsync command: rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
success: Command rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
Export all backup in other directory:
arthur@heartofgold$ bb export --catalog /mnt/backup/ --all --destination /mnt/backup/export --verbose
debug: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
debug: Build a rsync command
info: Start export host1
...
debug: rsync command: rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
success: Command rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
Export a backup with exclude pdf files:
arthur@heartofgold$ bb export --catalog /mnt/backup/ --backup-id f0f700e8-0435-11e9-9e78-005056a664e0 --destination /backup/export --verbose --exclude *.pdf
debug: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
debug: Build a rsync command
info: Start export host1
...
debug: rsync command: rsync -ah --no-links -vP --exclude=*.pdf /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
success: Command rsync -ah --no-links -vP --exclude=*.pdf /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
Donations
Donating is important. If you do not want to do it to me, do it to some companies that do not speculate. My main activity and the people of non-profit associations is to work for others, be they male or female, religious or non-religious, white or black or yellow or red, rich and poor. The only real purpose is to serve the humanity of one’s experience. Below you will find some links to do it. Thanks a lot.
For me

For Telethon
The Telethon Foundation is a non-profit organization recognized by the Ministry of University and Scientific and Technological Research. They were born in 1990 to respond to the appeal of patients suffering from rare diseases. Come today, we are organized to dare to listen to them and answers, every day of the year.