Welcome to Butterfly Backup’s documentation!

Indices and tables

Requirements

Butterfly Backup is a simple wrapper of rsync written in python; this means the first requirements is python3.8 or higher. The other requirement is a openssh and rsync (version 2.5 or higher). Ok, let’s go!

Test

If you want to try or test Butterfly Backup before installing it, run the test:

arthur@heartofgold$ git clone https://github.com/MatteoGuadrini/Butterfly-Backup.git
arthur@heartofgold$ cd Butterfly-Backup
arthur@heartofgold$ bash test_bb.py

Installation

Install Butterfly Backup is very simple; run this:

arthur@heartofgold$ git clone https://github.com/MatteoGuadrini/Butterfly-Backup.git
arthur@heartofgold$ cd Butterfly-Backup
arthur@heartofgold$ sudo python3 setup.py install -f # -f is for upgrade
arthur@heartofgold$ bb --help
arthur@heartofgold$ man bb

The upgrade is also simple; type the same commands.

Core concept

Butterfly Backup it is a server to client tool. It must be installed on a server (for example a workstation PC), from where it will contact the clients on which it will perform the backup. Also for the restore the concept is the same; the server pushes files into the client.

Backups are organized according to precise cataloguing; this is an example:

arthur@heartofgold$ tree destination/of/backup
.
├── destination
│   ├── hostname or ip of the PC under backup
│   │   ├── timestamp folder
│   │   │   ├── backup folders
│   │   │   ├── backup.log
│   │   │   └── restore.log
│   │   ├─── general.log
│   │   └─── symlink of last backup
│
├── export.log
├── backup.list
└── .catalog.cfg

This is how the operating system sees the backups on the file system:

  • destination: it is the destination of all backup of all machine under Butterfly Backup system; this is root.

  • hostname or ip of the PC under backup: it is the root of every machine under backup; contains all the backups.

  • timestamp folder: this is a single backup folder; contains all the data that have been backed up.

  • backup.log: if enabled, it is the log generated by rsync for backup operation.

  • restore.log: if enabled, it is the log generated by rsync for restore operation.

  • general.log: if enabled, it is the general log of the machine; contains all the info, warnings and errors generated during the backup.

  • symlink of last backup: this is a symbolic link of the last backup made. Isn’t supported if you use an MS-DOS file system.

  • backup.list: if enabled, it is the log generated by list operation.

  • export.log: if enabled, it is the log generated by export operation.

  • .catalog.cfg: is the log that contains the catalog of all backups. The file is the reference of Butterfly Backup to perform future backups.

Important

Be careful to not delete .catalog.cfg file. If you want to initialize the backup catalog, just run this command: bb config --init /catalog/path.

Butterfly Backup has, in its core, six main operations:

arthur@heartofgold$ bb --help
usage: bb [-h] [--verbose] [--log] [--dry-run] [--force] [--version]
          {config,backup,restore,archive,list,export} ...

Butterfly Backup

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode
  --force, -O           Force an action without prompt
  --version, -V         Print version

action:
  Valid action

  {config,backup,restore,archive,list,export}
                        Available actions
    config              Configuration options
    backup              Backup options
    restore             Restore options
    archive             Archive options
    list                List options
    export              Export options

Valid action

  • config: operations involving OpenSSH and its configurations (not mandatory).

  • backup: transactions that call rsync, in a single process or across multiple processes in case of a list.

  • restore: transactions that call rsync for pushing files to a client.

  • archive: transaction that zip old backups and move them to a new destination.

  • list: query the catalog.

  • export: export a single backup to other path.

It also has three flags that can be very useful, especially in case of error.

-h, --help

show help message and exit

--verbose, -v

Enable verbosity

--log, -l

Create a log

--dry-run, -N

Dry run mode, test your command

--force, -O

Force an action without prompt

--version, -V

Print version

Config

Configuration mode is very simple; If you’re already familiar with the exchange keys and OpenSSH, you probably won’t need it. If you don’t want to discuss the merits of exchange keys and start, then go ahead. First, you must create a configuration (rsa keys). Let’s see how to go about looking at the help:

arthur@heartofgold$ bb config --help
usage: bb config [-h] [--verbose] [--log] [--dry-run] [--force] [--version] [--new | --remove | --init CATALOG | --delete-host CATALOG HOST | --clean CATALOG | --delete-backup CATALOG ID] [--deploy DEPLOY_HOST] [--user DEPLOY_USER]

options:
-h, --help            show this help message and exit
--verbose, -v         Enable verbosity
--log, -l             Create logs
--dry-run, -N         Dry run mode
--force, -O           Force an action without prompt
--version, -V         Print version

Init configuration:
--new, -n             Generate new configuration
--remove, -r          Remove exist configuration
--init CATALOG, -i CATALOG
                      Reset CATALOG file. Specify path of backup folder.
--delete-host CATALOG HOST, -D CATALOG HOST
                      Delete all entry for a single HOST in CATALOG.
--clean CATALOG, -c CATALOG
                      Cleans the CATALOG if it is corrupt, setting default values.
--delete-backup CATALOG ID, -b CATALOG ID
                      Delete specific backup ID from CATALOG

Deploy configuration:
--deploy DEPLOY_HOST, -d DEPLOY_HOST
                      Deploy configuration to client: hostname or ip address
--user DEPLOY_USER, -u DEPLOY_USER
                      User of the remote machine

Two macro-options are available:

  • Init configuration: Generate new or remove configuration.
    --new, -n

    Generate new configuration.

    --remove, -r

    Remove exist configuration.

    --init, -i

    Reset catalog file. Specify path of backup folder.

    --delete-host, -D

    Delete all entry for a single host in catalog.

    --clean, -c

    Cleans the catalog if it is corrupt, setting default values.

    --delete-backup, -b

    Delete specific backup id from catalog.

  • Deploy configuration: Deploy configuration to client: hostname or ip address.
    --deploy, -d

    Deploy configuration to client: hostname or ip address.

    --user, -u

    User of the remote machine.

At this point, we create the configuration:

arthur@heartofgold$ bb config --new
warning: Private key ~/.ssh/id_rsa exists
If you want to use the existing key, run "bb config --deploy name_of_machine", otherwise to remove it, run "bb config --remove"

In this case, the RSA key already exists. Now try delete and create a new keys:

arthur@heartofgold$ bb config --remove
Are you sure to remove existing rsa keys? To continue [Y/N]? y
success: Removed configuration successfully!
arthur@heartofgold$ bb config --new
success: New configuration successfully created!

Once you have created the configuration, keys should be installed (copied) on the hosts you want to backup.

arthur@heartofgold$ bb config --deploy host1
Copying configuration to host1; write the password:
/usr/bin/ssh-copy-id: debug: Source of key(s) to be installed: "/home/arthur/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: debug: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: debug: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
arthur@host1's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'arthur@host1'"
and check to make sure that only the key(s) you wanted were added.

success: Configuration copied successfully on host1!

This command will try to copy the configuration with the current user. If you want to use a different user (e.g.: root), run this:

arthur@heartofgold$ bb config --deploy host1 --user root
Copying configuration to host1; write the password:
/usr/bin/ssh-copy-id: debug: Source of key(s) to be installed: "/home/arthur/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: debug: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: debug: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@host1's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'root@host1'"
and check to make sure that only the key(s) you wanted were added.

success: Configuration copied successfully on host1!

Cygwin on Windows

In order to run Butterfly Backup on Windows hosts, you need to install cygwin. Download https://cygwin.com/ cygwin and follow this instructions to install the necessary packages.

:: INSTALL SOFTWARE
:: Automates cygwin installation

SETLOCAL

:: Change to the directory of the executing batch file
CD %~dp0

:: Configure our paths
SET SITE=http://mirrors.kernel.org/sourceware/cygwin/
SET LOCALDIR=%LOCALAPPDATA%/cygwin
SET ROOTDIR=C:/cygwin

:: These are the packages we will install (in addition to the default packages)
SET PACKAGES=mintty,wget,ctags,diffutils,openssh,rsync,cygrunsrv,nano
SET PACKAGES=%PACKAGES%,gcc-core,make,automake,autoconf,readline,libncursesw-devel,libiconv,zlib-devel,gettext

:: Do it!
setup.exe -q -D -L -d -g -o -s %SITE% -l "%LOCALDIR%" -R "%ROOTDIR%" -C Base -P %PACKAGES% >nul

setx PATH "%PATH%;/cygdrive/c/cygwin/bin;C:\cygwin\bin" /m

ENDLOCAL

C:
chdir C:\cygwin\bin
mkpasswd -cl > C:\cygwin\etc\passwd
bash ssh-host-config -y
cygrunsrv -S sshd

Verify if port 22 is in LISTEN

If you want to initialize or reset the catalog file, run this:

arthur@heartofgold$ bb config --init /mnt/backup -v

Important

This command preserves existing backups on the file system. It will eliminate only those archived or deleted.

Backup

There are two backup modes: single and bulk. Let’s see how to go about looking at the help:

arthur@heartofgold$ bb backup --help
usage: bb backup [-h] [--verbose] [--log] [--dry-run] [--force] [--version] (--computer HOSTNAME | --list LIST) --destination DESTINATION [--mode {full,incremental,differential,mirror}]
               (--data {user,config,application,system,log} [{user,config,application,system,log} ...] | --custom-data CUSTOMDATA [CUSTOMDATA ...] | --file-data FILEDATA) [--user USER] --type {unix,windows,macos} [--compress]
               [--retention [DAYS [NUMBER ...]]] [--parallel PARALLEL] [--timeout TIMEOUT] [--skip-error] [--rsync-path RSYNC] [--bwlimit BWLIMIT] [--ssh-port PORT] [--exclude EXCLUDE [EXCLUDE ...]] [--start-from ID]

optional arguments:
-h, --help            show this help message and exit
--verbose, -v         Enable verbosity
--log, -l             Create logs
--dry-run, -N         Dry run mode
--force, -O           Force an action without prompt
--version, -V         Print version

Backup options:
--computer HOSTNAME, -c HOSTNAME
                        Hostname or ip address to backup
--list LIST, -L LIST  File list of computers or ip addresses to backup
--destination DESTINATION, -d DESTINATION
                        Destination path
--mode {full,incremental,differential,mirror}, -m {full,incremental,differential,mirror}
                        Backup mode
--data {user,config,application,system,log} [{user,config,application,system,log} ...], -D {user,config,application,system,log} [{user,config,application,system,log} ...]
                        Data of which you want to backup
--custom-data CUSTOMDATA [CUSTOMDATA ...], -C CUSTOMDATA [CUSTOMDATA ...]
                        Custom path of which you want to backup
--file-data FILEDATA, -F FILEDATA
                        File with custom path of which you want to backup
--user USER, -u USER  Login name used to log into the remote host (being backed up)
--type {unix,windows,macos}, -t {unix,windows,macos}
                        Type of operating system to backup
--compress, -z        Compress data
--retention [DAYS [NUMBER ...]], -r [DAYS [NUMBER ...]]
                        First argument are days of backup retention. Second argument is minimum number of backup retention
--parallel PARALLEL, -p PARALLEL
                        Number of parallel jobs
--timeout TIMEOUT, -T TIMEOUT
                        I/O timeout in seconds
--skip-error, -e      Skip error
--rsync-path RSYNC, -R RSYNC
                        Custom rsync path
--bwlimit BWLIMIT, -b BWLIMIT
                        Bandwidth limit in KBPS.
--ssh-port PORT, -P PORT
                        Custom ssh port.
--exclude EXCLUDE [EXCLUDE ...], -E EXCLUDE [EXCLUDE ...]
                        Exclude pattern
--start-from ID, -s ID
                        Backup id where start a new backup
  • Backup options
    --computer, -c

    Select the ip or hostname where to perform backup.

    --list, -l

    Select the file list of the ip or hostnames, where to perform backups.

    [File_list.txt]

    host1

    192.168.0.1

    host2

    …

    --destination, -d

    Select the destination folder (root).

    --mode, -m

    Select how rsync perform backup:

    • full:

      Complete (full) backup.

    • incremental:

      Incremental backup is based on the latest backup (the same files are linked with hard link). A Full backup is executed if this mode fails to find one.

    • differential:

      Incremental backup is based on the latest Full backup (the same files are linked with hard link). A Full backup is executed if this mode fails to find one.

    • mirror:

      Complete mirror backup. If a file in the source no longer exists, BB deletes it from the destination.

    --data, -D

    Select the type of data to put under backup: The values change depending on the type of operating system:

    • user -> folder containing the home.

    • config -> folder containing the configurations of the machine.

    • log -> folder containing the log.

    • application -> folder containing applications.

    • system -> the entire system starting from the root.

    --custom-data

    Select the absolute paths to put under backup.

    --file-data

    Select a file with the absolute paths to put under backup.

    [File_list.txt]

    /path1

    /path2/with spaces/

    “/path3/with quotes”

    …

    --user, -u

    Login name used to log into the remote host (being backed up)

    --type, -t

    Type of operating system to put under backup:

    • unix -> All UNIX os (Linux, BSD, Solaris).

    • windows -> Windows Vista or higher with cygwin installed.

    • macos -> MacOSX 10.8 or higher.

    --compress, -z

    Compresses the data transmitted.

    --retention, -r

    Number of days for which you want to keep your backups and minimum number of backup retention (optional).

    The second number is a minimum number of backup which you want keep.

    --parallel, -p

    Maximum number of concurrent rsync processes. By default is 5 jobs.

    --timeout, -T

    Specify number of seconds of I/O timeout.

    --skip-error, -e

    Skip error. Quiet mode.

    --rsync-path, -R

    Select a custom rsync path.

    --bwlimit, -b

    Bandwidth limit in KBPS.

    --ssh-port, -P

    Custom ssh port.

    --exclude, -E

    Exclude pattern. Follow rsync “Exclude Pattern Rules”

    --start-from, -s

    The new backup is based on another backup, specified by its ID.

Flowchart of the differences between Differential and Incremental backup:

+----------+   +----------+   +----------+
|          |   |          |   |          |
|   Full   | <-+   Inc.   | <-+   Inc.   |
|          |   |          |   |          |
+----------+   +----------+   +----------+

+----------+   +----------+   +----------+
|          |   |          |   |          |
|   Full   | <-+   Dif.   |   |   Dif.   |
|          |   |          |   |          |
+----+-----+   +----------+   +----+-----+
     ^                             |
     |                             |
     +-----------------------------+

Single backup

The backup of a single machine consists in taking the files and folders indicated in the command line, and put them in the cataloging structure indicated above.

This is a few examples:

arthur@heartofgold$ bb backup --computer host1 --destination /mnt/backup --data User Config --type MacOS
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
Start backup on host1
success: Command rsync -ah --no-links arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_28

Important

Without specifying the “mode” flag, Butterfly Backup by default selects Incremental mode

arthur@heartofgold$ bb backup --computer host1 --destination /mnt/backup --mode Full --data User Config --type MacOS --user root
root@host1's password:
Start backup on host1
success: Command rsync -ah --no-links root@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_30

arthur@heartofgold$ bb backup --computer host1 --destination /mnt/backup --data User Config --type MacOS --verbose --log
debug: Build a rsync command
debug: Last full is 2018-08-08 10:30:32
debug: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_28 -vP
debug: Create a folder structure for MacOS os
debug: Include this criteria: :/Users :/private/etc
debug: Destination is /mnt/backup/host1/2018_08_08__10_42
Start backup on host1
debug: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_42
receiving file list ...
39323 files to consider
Users/
...
...
sent 18.91K bytes  received 1.59M bytes  25.74K bytes/sec
total size is 6.67G  speedup is 4143.59
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_42

Note

If the backup destination is an MS-DOS fs, it will not support symlink creation. The message that will be returned is as follows: warning: MS-DOS file system doesn't support symlink file.

Bulk backup

Bulk backup follows the same logic and the same options as a single backup, but accepts a file containing the names or ips of the machine to be backed up.

This is a few examples:

arthur@heartofgold$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
error: The port 22 on host2 is closed!
error: The port 22 on host3 is closed!
Start backup on host1
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_50

Important

Port 22 (OpenSSH standard) is tested in order to verify the reachability of the machine. If the machine is not reachable, an error is generated: error: The port 22 on host2 is closed!

This example, is the same as the previous one, with the only difference being that the parallel flag is specified at 2. This means that maximum two backup jobs will run at the same time. When a first process ends, another one is started.

arthur@heartofgold$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --log
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
error: The port 22 on host2 is closed!
error: The port 22 on host3 is closed!
Start backup on host1
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58

This is the same example but specifying a retention at 3 (days). This means that in doing so we want to keep only 3 days backups.

arthur@heartofgold$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --retention 3 --log --verbose
debug: Build a rsync command
debug: Last full is 2018-08-08 10:30:32
debug: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 -vP
debug: Create a folder structure for MacOS os
debug: Include this criteria: :/Users :/private/etc
debug: Destination is /mnt/backup/host1/2018_08_08__10_30
Start backup on host1
error: The port 22 on host2 is closed!
error: The port 22 on host3 is closed!
debug: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
receiving file list ...
39323 files to consider
Users/
...
...
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
debug: Check cleanup this backup aba860b0-9944-11e8-a93f-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_28
success: Cleanup /mnt/backup/host1/2018_08_08__10_28 successfully.
debug: Check cleanup this backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30
debug: No cleanup backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30

Here we find the same example above but specifying a retention at 2 (days) and 5 backup copies. This means that at least 5 backup copies will be kept, even if they are older than two days.

arthur@heartofgold$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --retention 3 5 --log --verbose
debug: Build a rsync command
debug: Last full is 2018-08-08 10:30:32
debug: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 -vP
debug: Create a folder structure for MacOS os
debug: Include this criteria: :/Users :/private/etc
debug: Destination is /mnt/backup/host1/2018_08_08__10_30
Start backup on host1
error: The port 22 on host2 is closed!
error: The port 22 on host3 is closed!
debug: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
receiving file list ...
39323 files to consider
Users/
...
...
success: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 arthur@host1:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
debug: Check cleanup this backup aba860b0-9944-11e8-a93f-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_28
success: Cleanup /mnt/backup/host1/2018_08_08__10_28 successfully.
debug: Check cleanup this backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30
debug: No cleanup backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30

Important

If there is only one Full in our catalog, the retention policy will never delete that Full backup, but only Increments. We recommend doing more Full for better retention management.

List

When we run backup commands, a catalog is created. This serves both for future backups and all the restores that are made through Butterfly Backup. To query this catalog, the list command exists.

arthur@heartofgold$ bb list --help
usage: bb list [-h] [--verbose] [--log] [--dry-run] [--force] [--version] --catalog CATALOG
               [--backup-id ID | --archived | --cleaned | --computer HOSTNAME | --detail ID]
               [--oneline]

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode
  --force, -O           Force an action without prompt
  --version, -V         Print version

List options:
  --catalog CATALOG, -C CATALOG
                        Folder where is catalog file
  --backup-id ID, -i ID
                        Backup-id of backup
  --archived, -a        List only archived backup
  --cleaned, -c         List only cleaned backup
  --computer HOSTNAME, -H HOSTNAME
                        List only match hostname or ip
  --detail ID, -d ID    List detail of file and folder of specific backup-id
  --oneline, -o         One line output
  • List options
    --catalog, -C

    Select the backups folder (root).

    --backup-id, -i

    Select backup id in the catalog.

    --archived, -a

    List only archived backups.

    --cleaned, -c

    List only cleaned backups.

    --computer, -H

    List only match hostname or ip.

    --detail, -d

    List detail of file and folder of specific backup-id.

    --oneline, -o

    One line and concise output.

First, let’s query the catalog:

arthur@heartofgold$ bb list --catalog /mnt/backup

BUTTERFLY BACKUP CATALOG

Backup id: aba860b0-9944-11e8-a93f-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:28:12

Backup id: cc6e2744-9944-11e8-b82a-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:30:59

Backup id: dd6de2f2-9a1e-11e8-82b0-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:58:59

Press q for exit. Then we select a backup-id:

arthur@heartofgold$ bb list --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0
Backup id: dd6de2f2-9a1e-11e8-82b0-005056a664e0
Hostname or ip: host1
Type: Incremental
Timestamp: 2018-08-08 10:58:59
Start: 2018-08-08 10:58:59
Finish: 2018-08-08 11:02:49
OS: MacOS
ExitCode: 0
Path: /mnt/backup/host1/2018_08_08__10_58
List: backup.log
etc
Users

If you want to export the catalog list instead, include the log flag:

arthur@heartofgold$ bb list --catalog /mnt/backup --log
arthur@heartofgold$ cat /mnt/backup/backup.list

Instead if you want to see the details of your backup, add details flag:

arthur@heartofgold$ bb list --catalog /mnt/backup --detail dd6de2f2-9a1e-11e8-82b0-005056a664e0

Now that we have identified a backup, let’s proceed with the restore

Restore

The restore process is the exact opposite of the backup process. It takes the files from a specific backup and push it to the destination computer.

arthur@heartofgold$ bb restore --help
usage: bb restore [-h] [--verbose] [--log] [--dry-run] [--force] [--version] --catalog CATALOG (--backup-id ID | --last) [--user USER] --computer HOSTNAME [--type {Unix,Windows,MacOS}] [--timeout TIMEOUT] [--mirror] [--skip-error]
               [--rsync-path RSYNC] [--bwlimit BWLIMIT] [--ssh-port PORT] [--exclude EXCLUDE [EXCLUDE ...]] [--files FILES [FILES ...]]

options:
-h, --help            show this help message and exit
--verbose, -v         Enable verbosity
--log, -l             Create logs
--dry-run, -N         Dry run mode
--force, -O           Force an action without prompt
--version, -V         Print version

Restore options:
--catalog CATALOG, -C CATALOG
                        Folder where is catalog file
--backup-id ID, -i ID
                        Backup-id of backup
--last, -L              Last available backup
--user USER, -u USER    Login name used to log into the remote host (where you're restoring)
--computer HOSTNAME, -c HOSTNAME
                        Hostname or ip address to perform restore
--type {Unix,Windows,MacOS}, -t {Unix,Windows,MacOS}
                        Type of operating system to perform restore
--timeout TIMEOUT, -T TIMEOUT
                        I/O timeout in seconds
--mirror, -m            Mirror mode
--skip-error, -e        Skip error
--rsync-path RSYNC, -R RSYNC
                        Custom rsync path
--bwlimit BWLIMIT, -b BWLIMIT
                        Bandwidth limit in KBPS.
--ssh-port PORT, -P PORT
                        Custom ssh port.
--exclude EXCLUDE [EXCLUDE ...], -E EXCLUDE [EXCLUDE ...]
                        Exclude pattern
--files FILES [FILES ...], -f FILES [FILES ...]
                        Restore only specified files
  • Restore options
    --catalog, -C

    Select the backups folder (root).

    --backup-id

    Select backup id in the catalog.

    --last, -L

    Select last available backup in the catalog for same hostname or ip address.

    --user, -u

    User of remote machine where you want to restore files.

    --computer, -c

    Select the ip or hostname where to perform restore.

    --type, -t

    Type of operating system to put under backup:

    • Unix -> All UNIX os (Linux, BSD, Solaris).

    • Windows -> Windows Vista or higher with cygwin installed.

    • MacOS -> MacOSX 10.8 or higher.

    --timeout, -T

    Specify number of seconds of I/O timeout.

    --mirror, -m

    Mirror mode. If a file or folder not exist in destination, will delete it. Overwrite files.

    --skip-error, -e

    Skip error. Quiet mode.

    --rsync-path, -R

    Select a custom rsync path.

    --bwlimit, -b

    Bandwidth limit in KBPS.

    --ssh-port, -P

    Custom ssh port.

    --exclude, -E

    Exclude pattern. Follow rsync “Exclude Pattern Rules”.

    --files, -f

    Restore only specified files.

This is a few examples:

This command perform a restore on the same machine of the backup:

arthur@heartofgold$ bb restore --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0 --computer host1 --log
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc arthur@host1:/restore_2018_08_08__10_58
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* arthur@host1:/Users

Important

Without specifying the “type” flag that indicates the operating system on which the data are being retrieved, Butterfly Backup will select it directly from the catalog via the backup-id.

Now, select the last available backup on catalog; run this:

arthur@heartofgold$ bb restore --catalog /mnt/backup --last --computer host1 --log
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc arthur@host1:/restore_2018_08_08__10_59
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* arthur@host1:/Users

This example, is the same as the previous one, but restore to other machine and other operating system:

arthur@heartofgold$ bb restore --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0 --computer host2 --type Unix --log --verbose
debug: Build a rsync command
debug: Command flags are: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
debug: Build a rsync command
debug: Command flags are: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
Start restore on host2
debug: rsync command: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc/* arthur@host2:/etc
Start restore on host2
debug: rsync command: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* arthur@host2:/home
building file list ...
26633 files to consider
...
...
sent 777.53K bytes  received 20 bytes  62.20K bytes/sec
total size is 6.66G  speedup is 8566.41
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc/* arthur@host2:/etc
success: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* arthur@host2:/home

Archive

Archive operations are used to store backups by saving disk space. Backups older than n days are compressed into a zip file.

arthur@heartofgold$ bb archive --help
usage: bb archive [-h] [--verbose] [--log] [--dry-run] [--force] [--version] --catalog CATALOG
                  [--days DAYS] --destination DESTINATION

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode
  --force, -O           Force an action without prompt
  --version, -V         Print version

Archive options:
  --catalog CATALOG, -C CATALOG
                        Folder where is catalog file
  --days DAYS, -D DAYS  Number of days of archive retention
  --destination DESTINATION, -d DESTINATION
                        Archive destination path
  • Archive options
    --catalog, -C

    Select the backups folder (root).

    --days, -D

    Number of days for which you want to keep your backups. Default is 30.

    --destination, -d

    New destination for compress zipped backup.

Archive old backup of 3 days:

arthur@heartofgold$ bb archive --catalog /mnt/backup/ --days 3 --destination /mnt/archive/ --verbose --log
debug: Check archive this backup f65e5afe-9734-11e8-b0bb-005056a664e0. Folder /mnt/backup/host2/2018_08_08__17_50
debug: Check archive this backup 4f2b5f6e-9939-11e8-9ab6-005056a664e0. Folder /mnt/backup/host2/2018_08_04__07_26
success: Delete /mnt/backup/host2/2018_08_04__07_26 successfully.
success: Archive /mnt/backup/host2/2018_08_04__07_26 successfully.
arthur@heartofgold$ ls /mnt/archive
host1
arthur@heartofgold$ ls /mnt/archive/host1
2018_08_06__07_26.zip

The backup-id f65e5afe-9734-11e8-b0bb-005056a664e0 it is not considered, because it falls within the established time. The other, however, is zipped and deleted.

Lastly, let’s look in the catalog and see that the backup was actually archived:

arthur@heartofgold$ bb list --catalog /mnt/backup/ -i 4f2b5f6e-9939-11e8-9ab6-005056a664e0
Backup id: 4f2b5f6e-9939-11e8-9ab6-005056a664e0
Hostname or ip: host2
Type: Incremental
Timestamp: 2018-08-08 07:26:46
Start: 2018-08-08 07:26:46
Finish: 2018-08-08 08:43:45
OS: MacOS
ExitCode: 0
Path: /mnt/backup/host2/2018_08_04__07_26
Archived: True

Export

The export function is used to copy a particular backup to another path.

arthur@heartofgold$ bb export -h
usage: bb export [-h] [--verbose] [--log] [--dry-run] [--force] [--version] --catalog CATALOG
                 [--backup-id ID | --all] --destination DESTINATION [--mirror]
                 [--cut] [--include INCLUDE [INCLUDE ...] | --exclude EXCLUDE
                 [EXCLUDE ...]] [--timeout TIMEOUT] [--skip-error]
                 [--rsync-path RSYNC] [--bwlimit BWLIMIT] [--ssh-port PORT]

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode
  --force, -O           Force an action without prompt
  --version, -V         Print version

Export options:
  --catalog CATALOG, -C CATALOG
                        Folder where is catalog file
  --backup-id ID, -i ID
                        Backup-id of backup
  --all, -A             All backup
  --destination DESTINATION, -d DESTINATION
                        Destination path
  --mirror, -m          Mirror mode
  --cut, -c             Cut mode. Delete source
  --include INCLUDE [INCLUDE ...], -I INCLUDE [INCLUDE ...]
                        Include pattern
  --exclude EXCLUDE [EXCLUDE ...], -E EXCLUDE [EXCLUDE ...]
                        Exclude pattern
  --timeout TIMEOUT, -T TIMEOUT
                        I/O timeout in seconds
  --skip-error, -e      Skip error
  --rsync-path RSYNC, -R RSYNC
                        Custom rsync path
  --bwlimit BWLIMIT, -b BWLIMIT
                        Bandwidth limit in KBPS.
  --ssh-port PORT, -P PORT
                        Custom ssh port.
  • Export options
    --catalog, -C

    Select the backups folder (root).

    --backup-id, -i

    Select backup id in the catalog.

    --all, -A

    All backup

    --destination, -d

    Destination path.

    --mirror, -m

    Mirror backup on destination.

    --cut, -c

    Delete source like a move.

    --include, -I

    Include pattern. Follow rsync “Include Pattern Rules”

    --exclude, -E

    Exclude pattern. Follow rsync “Exclude Pattern Rules”

    --timeout, -T

    Specify number of seconds of I/O timeout.

    --mirror, -m

    Mirror mode. If a file or folder not exist in destination, will delete it. Overwrite files.

    --skip-error, -e

    Skip error. Quiet mode.

    --rsync-path, -R

    Select a custom rsync path.

    --bwlimit, -b

    Bandwidth limit in KBPS.

    --ssh-port, -P

    Custom ssh port.

Export a backup in other directory:

arthur@heartofgold$ bb export --catalog /mnt/backup/ --backup-id f0f700e8-0435-11e9-9e78-005056a664e0 --destination /mnt/backup/export --verbose
debug: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
debug: Build a rsync command
Start export host1
...
debug: rsync command: rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
success: Command rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1

Export all backup in other directory:

arthur@heartofgold$ bb export --catalog /mnt/backup/ --all --destination /mnt/backup/export --verbose
debug: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
debug: Build a rsync command
Start export host1
...
debug: rsync command: rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
success: Command rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1

Export a backup with exclude pdf files:

arthur@heartofgold$ bb export --catalog /mnt/backup/ --backup-id f0f700e8-0435-11e9-9e78-005056a664e0 --destination /backup/export --verbose --exclude *.pdf
debug: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
debug: Build a rsync command
Start export host1
...
debug: rsync command: rsync -ah --no-links -vP --exclude=*.pdf /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
success: Command rsync -ah --no-links -vP --exclude=*.pdf /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1

Donations

Donating is important. If you do not want to do it to me, do it to some companies that do not speculate. My main activity and the people of non-profit associations is to work for others, be they male or female, religious or non-religious, white or black or yellow or red, rich and poor. The only real purpose is to serve the humanity of one’s experience. Below you will find some links to do it. Thanks a lot.

For me

https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif

For Telethon

The Telethon Foundation is a non-profit organization recognized by the Ministry of University and Scientific and Technological Research. They were born in 1990 to respond to the appeal of patients suffering from rare diseases. Come today, we are organized to dare to listen to them and answers, every day of the year.

Adopt the future