Welcome to Butterfly Backup’s documentation!

Requirements

Butterfly Backup is a simple wrapper of rsync written in python; this means the first requirements is python3.3 or higher. The other requirement is a openssh and rsync (version 2.5 or higher). Ok, let’s go!

Installation

Install Butterfly Backup is very simple; run this:

[email protected]$ git clone https://github.com/MatteoGuadrini/Butterfly-Backup.git
[email protected]$ cd Butterfly-Backup
[email protected]$ sudo python3 setup.py
[email protected]$ bb --help
[email protected]$ man bb

The upgrade is also simple; type the same commands.

Core concept

Butterfly Backup it is a server to client tool. It must be installed on a server (for example a workstation PC), from where it will contact the clients on which it will perform the backup. Also for the restore the concept is the same; the server pushes files into the client.

Backups are organized according to precise cataloguing; this is an example:

[email protected]$ tree destination/of/backup
.
├── destination
│   ├── hostname or ip of the PC under backup
│   │   ├── timestamp folder
│   │   │   ├── backup folders
│   │   │   ├── backup.log
│   │   │   └── restore.log
│   │   ├─── general.log
│   │   └─── symlink of last backup
│
├── export.log
├── backup.list
└── .catalog.cfg

This is how the operating system sees the backups on the file system:

  • destination: it is the destination of all backup of all machine under Butterfly Backup system; this is root.
  • hostname or ip of the PC under backup: it is the root of every machine under backup; contains all the backups.
  • timestamp folder: this is a single backup folder; contains all the data that have been backed up.
  • backup.log: if enabled, it is the log generated by rsync for backup operation.
  • restore.log: if enabled, it is the log generated by rsync for restore operation.
  • general.log: if enabled, it is the general log of the machine; contains all the info, warnings and errors generated during the backup.
  • symlink of last backup: this is a symbolic link of the last backup made. Isn’t supported if you use an MS-DOS file system.
  • backup.list: if enabled, it is the log generated by list operation.
  • export.log: if enabled, it is the log generated by export operation.
  • .catalog.cfg: is the log that contains the catalog of all backups. The file is the reference of Butterfly Backup to perform future backups.

Important

Be careful to not delete .catalog.cfg file.

Butterfly Backup has, in its core, six main operations:

[email protected]$ bb --help
usage: bb [-h] [--verbose] [--log] [--dry-run] [--version]
          {config,backup,restore,archive,list,export} ...

Butterfly Backup

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode
  --version, -V         Print version

action:
  Valid action

  {config,backup,restore,archive,list,export}
                        Available actions
    config              Configuration options
    backup              Backup options
    restore             Restore options
    archive             Archive options
    list                List options
    export              Export options

Valid action

  • config: operations involving OpenSSH and its configurations (not mandatory).
  • backup: transactions that call rsync, in a single process or across multiple processes in case of a list.
  • restore: transactions that call rsync for pushing files to a client.
  • archive: transaction that zip old backups and move them to a new destination.
  • list: query the catalog.
  • export: export a single backup to other path.

It also has three flags that can be very useful, especially in case of error.

-h, --help show help message and exit
--verbose, -v Enable verbosity
--log, -l Create a log
--dry-run, -N Dry run mode, test your command
--version, -V Print version…and more

Config

Configuration mode is very simple; If you’re already familiar with the exchange keys and OpenSSH, you probably won’t need it. If you don’t want to discuss the merits of exchange keys and start, then go ahead. First, you must create a configuration (rsa keys). Let’s see how to go about looking at the help:

[email protected]$ bb config --help
usage: bb config [-h] [--verbose] [--log] [--dry-run]
                 [--new | --remove | --init INIT | --delete-host CATALOG HOST | --clean CATALOG]
                 [--deploy DEPLOY_HOST] [--user DEPLOY_USER]

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode

Init configuration:
  --new, -n             Generate new configuration
  --remove, -r          Remove exist configuration
  --init INIT, -i INIT  Reset catalog file. Specify path of backup folder.
  --delete-host CATALOG HOST, -D CATALOG HOST
                        Delete all entry for a single HOST in catalog.
  --clean CATALOG, -c CATALOG
                        Cleans the catalog if it is corrupt, setting default
                        values.

Deploy configuration:
  --deploy DEPLOY_HOST, -d DEPLOY_HOST
                        Deploy configuration to client: hostname or ip address
  --user DEPLOY_USER, -u DEPLOY_USER
                        User of the remote machine

Two macro-options are available:

  • Init configuration: Generate new or remove configuration.
    --new, -n

    Generate new configuration.

    --remove, -r

    Remove exist configuration.

    --init, -i

    Reset catalog file. Specify path of backup folder.

    --delete-host, -D
     

    Delete all entry for a single HOST in catalog.

    --clean, -c

    Cleans the catalog if it is corrupt, setting default values.

  • Deploy configuration: Deploy configuration to client: hostname or ip address.
    --deploy, -d

    Deploy configuration to client: hostname or ip address.

    --user, -u

    User of the remote machine.

At this point, we create the configuration:

[email protected]$ bb config --new
WARNING: Private key ~/.ssh/id_rsa exists
If you want to use the existing key, run "bb config --deploy name_of_machine", otherwise to remove it, run "bb config --remove"

In this case, the RSA key already exists. Now try delete and create a new keys:

[email protected]$ bb config --remove
Are you sure to remove existing rsa keys? To continue [Y/N]? y
SUCCESS: Removed configuration successfully!
[email protected]$ bb config --new
SUCCESS: New configuration successfully created!

Once you have created the configuration, keys should be installed (copied) on the hosts you want to backup.

[email protected]$ bb config --deploy host1
Copying configuration to host1; write the password:
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/arthur/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

SUCCESS: Configuration copied successfully on host1!

This command will try to copy the configuration with the current user. If you want to use a different user (e.g.: root), run this:

[email protected]$ bb config --deploy host1 --user root
Copying configuration to host1; write the password:
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/arthur/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

SUCCESS: Configuration copied successfully on host1!

Cygwin on Windows

In order to run Butterfly Backup on Windows hosts, you need to install cygwin. Download https://cygwin.com/ cygwin and follow this instructions to install the necessary packages.

:: INSTALL SOFTWARE
:: Automates cygwin installation

SETLOCAL

:: Change to the directory of the executing batch file
CD %~dp0

:: Configure our paths
SET SITE=http://mirrors.kernel.org/sourceware/cygwin/
SET LOCALDIR=%LOCALAPPDATA%/cygwin
SET ROOTDIR=C:/cygwin

:: These are the packages we will install (in addition to the default packages)
SET PACKAGES=mintty,wget,ctags,diffutils,openssh,rsync,cygrunsrv,nano
SET PACKAGES=%PACKAGES%,gcc-core,make,automake,autoconf,readline,libncursesw-devel,libiconv,zlib-devel,gettext

:: Do it!
setup.exe -q -D -L -d -g -o -s %SITE% -l "%LOCALDIR%" -R "%ROOTDIR%" -C Base -P %PACKAGES% >nul

setx PATH "%PATH%;/cygdrive/c/cygwin/bin;C:\cygwin\bin" /m

ENDLOCAL

C:
chdir C:\cygwin\bin
mkpasswd -cl > C:\cygwin\etc\passwd
bash ssh-host-config -y
cygrunsrv -S sshd

Verify if port 22 is in LISTEN

If you want to initialize or reset the catalog file, run this:

[email protected]$ bb config --init /mnt/backup -v

Important

This command preserves existing backups on the file system. It will eliminate only those archived or deleted.

Backup

There are two backup modes: single and bulk. Let’s see how to go about looking at the help:

[email protected]$ bb backup --help
usage: bb backup [-h] [--verbose] [--log] [--dry-run]
                 (--computer HOSTNAME | --list LIST) --destination DESTINATION
                 [--mode {Full,Incremental,Differential,Mirror}]
                 (--data {User,Config,Application,System,Log} [{User,Config,Application,System,Log} ...] | --custom-data CUSTOMDATA [CUSTOMDATA ...])
                 [--user USER] --type {Unix,Windows,MacOS} [--compress]
                 [--retention [DAYS [NUMBER ...]]] [--parallel PARALLEL]
                 [--timeout TIMEOUT] [--skip-error] [--rsync-path RSYNC]
                 [--bwlimit BWLIMIT] [--ssh-port PORT]
                 [--exclude EXCLUDE [EXCLUDE ...]] [--start-from ID]

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode

Backup options:
  --computer HOSTNAME, -c HOSTNAME
                        Hostname or ip address to backup
  --list LIST, -L LIST  File list of computers or ip addresses to backup
  --destination DESTINATION, -d DESTINATION
                        Destination path
  --mode {Full,Incremental,Differential,Mirror}, -m {Full,Incremental,Differential,Mirror}
                        Backup mode
  --data {User,Config,Application,System,Log} [{User,Config,Application,System,Log} ...], -D {User,Config,Application,System,Log} [{User,Config,Application,System,Log} ...]
                        Data of which you want to backup
  --custom-data CUSTOMDATA [CUSTOMDATA ...], -C CUSTOMDATA [CUSTOMDATA ...]
                        Custom path of which you want to backup
  --user USER, -u USER  Login name used to log into the remote host (being
                        backed up)
  --type {Unix,Windows,MacOS}, -t {Unix,Windows,MacOS}
                        Type of operating system to backup
  --compress, -z        Compress data
  --retention [DAYS [NUMBER ...]], -r [DAYS [NUMBER ...]]
                        First argument are days of backup retention. Second
                        argument is minimum number of backup retention
  --parallel PARALLEL, -p PARALLEL
                        Number of parallel jobs
  --timeout TIMEOUT, -T TIMEOUT
                        I/O timeout in seconds
  --skip-error, -e      Skip error
  --rsync-path RSYNC, -R RSYNC
                        Custom rsync path
  --bwlimit BWLIMIT, -b BWLIMIT
                        Bandwidth limit in KBPS.
  --ssh-port PORT, -P PORT
                        Custom ssh port.
  --exclude EXCLUDE [EXCLUDE ...], -E EXCLUDE [EXCLUDE ...]
                        Exclude pattern
  --start-from ID, -s ID
                        Backup id where start a new backup
  • Backup options
    --computer, -c

    Select the ip or hostname where to perform backup.

    --list, -l

    Select the file list of the ip or hostnames, where to perform backups.

    [File_list.txt]

    host1

    192.168.0.1

    host2

    --destination, -d
     

    Select the destination folder (root).

    --mode, -m

    Select how rsync perform backup:

    • Full:
      Complete (full) backup.
    • Incremental:
      Incremental backup is based on the latest backup (the same files are linked with hard link). A Full backup is executed if this mode fails to find one.
    • Differential:
      Incremental backup is based on the latest Full backup (the same files are linked with hard link). A Full backup is executed if this mode fails to find one.
    • Mirror:
      Complete mirror backup. If a file in the source no longer exists, BB deletes it from the destination.
    --data, -D

    Select the type of data to put under backup: The values change depending on the type of operating system:

    • User -> folder containing the home.
    • Config -> folder containing the configurations of the machine.
    • Log -> folder containing the log.
    • Application -> folder containing applications.
    • System -> the entire system starting from the root.
    --custom-data

    Select the absolute paths to put under backup.

    --user, -u

    Login name used to log into the remote host (being backed up)

    --type, -t

    Type of operating system to put under backup:

    • Unix -> All UNIX os (Linux, BSD, Solaris).
    • Windows -> Windows Vista or higher with cygwin installed.
    • MacOS -> MacOSX 10.8 or higher.
    --compress, -z

    Compresses the data transmitted.

    --retention, -r
     

    Number of days for which you want to keep your backups and minimum number of backup retention (optional).

    The second number is a minimum number of backup which you want keep.

    --parallel, -p

    Maximum number of concurrent rsync processes. By default is 5 jobs.

    --timeout, -T

    Specify number of seconds of I/O timeout.

    --skip-error, -e
     

    Skip error. Quiet mode.

    --rsync-path, -R
     

    Select a custom rsync path.

    --bwlimit, -b

    Bandwidth limit in KBPS.

    --ssh-port, -P

    Custom ssh port.

    --exclude, -E

    Exclude pattern. Follow rsync “Exclude Pattern Rules”

    --start-from, -s
     

    The new backup is based on another backup, specified by its ID.

Flowchart of the differences between Differential and Incremental backup:

+----------+   +----------+   +----------+
|          |   |          |   |          |
|   Full   | <-+   Inc.   | <-+   Inc.   |
|          |   |          |   |          |
+----------+   +----------+   +----------+

+----------+   +----------+   +----------+
|          |   |          |   |          |
|   Full   | <-+   Dif.   |   |   Dif.   |
|          |   |          |   |          |
+----+-----+   +----------+   +----+-----+
     ^                             |
     +-----------------------------+

Single backup

The backup of a single machine consists in taking the files and folders indicated in the command line, and put them in the cataloging structure indicated above.

This is a few examples:

[email protected]$ bb backup --computer host1 --destination /mnt/backup --data User Config --type MacOS
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
Start backup on host1
SUCCESS: Command rsync -ah --no-links [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_28

Important

Without specifying the “mode” flag, Butterfly Backup by default selects Incremental mode

[email protected]$ bb backup --computer host1 --destination /mnt/backup --mode Full --data User Config --type MacOS --user root
"[email protected]'s password:"
Start backup on host1
SUCCESS: Command rsync -ah --no-links [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_30
[email protected]$ bb backup --computer host1 --destination /mnt/backup --data User Config --type MacOS --verbose --log
INFO: Build a rsync command
INFO: Last full is 2018-08-08 10:30:32
INFO: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_28 -vP
INFO: Create a folder structure for MacOS os
INFO: Include this criteria: :/Users :/private/etc
INFO: Destination is /mnt/backup/host1/2018_08_08__10_42
Start backup on host1
INFO: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_42
receiving file list ...
39323 files to consider
Users/
...
...
sent 18.91K bytes  received 1.59M bytes  25.74K bytes/sec
total size is 6.67G  speedup is 4143.59
SUCCESS: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_42

Note

If the backup destination is an MS-DOS fs, it will not support symlink creation. The message that will be returned is as follows: WARNING: MS-DOS file system doesn't support symlink file.

Bulk backup

Bulk backup follows the same logic and the same options as a single backup, but accepts a file containing the names or ips of the machine to be backed up.

This is a few examples:

[email protected]$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
ERROR: The port 22 on host2 is closed!
ERROR: The port 22 on host3 is closed!
Start backup on host1
SUCCESS: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_50

Important

Port 22 (OpenSSH standard) is tested in order to verify the reachability of the machine. If the machine is not reachable, an error is generated: ERROR: The port 22 on host2 is closed!

This example, is the same as the previous one, with the only difference being that the parallel flag is specified at 2. This means that maximum two backup jobs will run at the same time. When a first process ends, another one is started.

[email protected]$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --log
# host1 SSH-2.0-OpenSSH_7.5
# host1 SSH-2.0-OpenSSH_7.5
ERROR: The port 22 on host2 is closed!
ERROR: The port 22 on host3 is closed!
Start backup on host1
SUCCESS: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58

This is the same example but specifying a retention at 3 (days). This means that in doing so we want to keep only 3 days backups.

[email protected]$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --retention 3 --log --verbose
INFO: Build a rsync command
INFO: Last full is 2018-08-08 10:30:32
INFO: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 -vP
INFO: Create a folder structure for MacOS os
INFO: Include this criteria: :/Users :/private/etc
INFO: Destination is /mnt/backup/host1/2018_08_08__10_30
Start backup on host1
ERROR: The port 22 on host2 is closed!
ERROR: The port 22 on host3 is closed!
INFO: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
receiving file list ...
39323 files to consider
Users/
...
...
SUCCESS: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
INFO: Check cleanup this backup aba860b0-9944-11e8-a93f-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_28
SUCCESS: Cleanup /mnt/backup/host1/2018_08_08__10_28 successfully.
INFO: Check cleanup this backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30
INFO: No cleanup backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30

Here we find the same example above but specifying a retention at 2 (days) and 5 backup copies. This means that at least 5 backup copies will be kept, even if they are older than two days.

[email protected]$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type MacOS --parallel 2 --retention 3 5 --log --verbose
INFO: Build a rsync command
INFO: Last full is 2018-08-08 10:30:32
INFO: Command flags are: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 -vP
INFO: Create a folder structure for MacOS os
INFO: Include this criteria: :/Users :/private/etc
INFO: Destination is /mnt/backup/host1/2018_08_08__10_30
Start backup on host1
ERROR: The port 22 on host2 is closed!
ERROR: The port 22 on host3 is closed!
INFO: rsync command: rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
receiving file list ...
39323 files to consider
Users/
...
...
SUCCESS: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2018_08_08__10_30 [email protected]:/Users :/private/etc /mnt/backup/host1/2018_08_08__10_58
INFO: Check cleanup this backup aba860b0-9944-11e8-a93f-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_28
SUCCESS: Cleanup /mnt/backup/host1/2018_08_08__10_28 successfully.
INFO: Check cleanup this backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30
INFO: No cleanup backup cc6e2744-9944-11e8-b82a-005056a664e0. Folder /mnt/backup/host1/2018_08_08__10_30

Important

If there is only one Full in our catalog, the retention policy will never delete that Full backup, but only Increments. We recommend doing more Full for better retention management.

List

When we run backup commands, a catalog is created. This serves both for future backups and all the restores that are made through Butterfly Backup. To query this catalog, the list command exists.

[email protected]$ bb list --help
usage: bb list [-h] [--verbose] [--log] [--dry-run] --catalog CATALOG
               [--backup-id ID | --archived | --cleaned | --computer HOSTNAME | --detail ID]
               [--oneline]

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode

List options:
  --catalog CATALOG, -C CATALOG
                        Folder where is catalog file
  --backup-id ID, -i ID
                        Backup-id of backup
  --archived, -a        List only archived backup
  --cleaned, -c         List only cleaned backup
  --computer HOSTNAME, -H HOSTNAME
                        List only match hostname or ip
  --detail ID, -d ID    List detail of file and folder of specific backup-id
  --oneline, -o         One line output
  • List options
    --catalog, -C

    Select the backups folder (root).

    --backup-id, -i
     

    Select backup id in the catalog.

    --archived, -a

    List only archived backups.

    --cleaned, -c

    List only cleaned backups.

    --computer, -H

    List only match hostname or ip.

    --detail, -d

    List detail of file and folder of specific backup-id.

    --oneline, -o

    One line and concise output.

First, let’s query the catalog:

[email protected]$ bb list --catalog /mnt/backup

BUTTERFLY BACKUP CATALOG

Backup id: aba860b0-9944-11e8-a93f-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:28:12

Backup id: cc6e2744-9944-11e8-b82a-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:30:59

Backup id: dd6de2f2-9a1e-11e8-82b0-005056a664e0
Hostname or ip: host1
Timestamp: 2018-08-08 10:58:59

Press q for exit. Then we select a backup-id:

[email protected]$ bb list --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0
Backup id: dd6de2f2-9a1e-11e8-82b0-005056a664e0
Hostname or ip: host1
Type: Incremental
Timestamp: 2018-08-08 10:58:59
Start: 2018-08-08 10:58:59
Finish: 2018-08-08 11:02:49
OS: MacOS
ExitCode: 0
Path: /mnt/backup/host1/2018_08_08__10_58
List: backup.log
etc
Users

If you want to export the catalog list instead, include the log flag:

[email protected]$ bb list --catalog /mnt/backup --log
[email protected]$ cat /mnt/backup/backup.list

Instead if you want to see the details of your backup, add details flag:

[email protected]$ bb list --catalog /mnt/backup --detail dd6de2f2-9a1e-11e8-82b0-005056a664e0

Now that we have identified a backup, let’s proceed with the restore

Restore

The restore process is the exact opposite of the backup process. It takes the files from a specific backup and push it to the destination computer.

[email protected]$ bb restore --help
usage: bb restore [-h] [--verbose] [--log] [--dry-run] --catalog CATALOG
                  (--backup-id ID | --last) [--user USER] --computer HOSTNAME
                  [--type {Unix,Windows,MacOS}] [--timeout TIMEOUT] [--mirror]
                  [--skip-error] [--rsync-path RSYNC] [--bwlimit BWLIMIT]
                  [--ssh-port PORT] [--exclude EXCLUDE [EXCLUDE ...]]

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode

Restore options:
  --catalog CATALOG, -C CATALOG
                        Folder where is catalog file
  --backup-id ID, -i ID
                        Backup-id of backup
  --last, -L            Last available backup
  --user USER, -u USER  Login name used to log into the remote host (where
                        you're restoring)
  --computer HOSTNAME, -c HOSTNAME
                        Hostname or ip address to perform restore
  --type {Unix,Windows,MacOS}, -t {Unix,Windows,MacOS}
                        Type of operating system to perform restore
  --timeout TIMEOUT, -T TIMEOUT
                        I/O timeout in seconds
  --mirror, -m          Mirror mode
  --skip-error, -e      Skip error
  --rsync-path RSYNC, -R RSYNC
                        Custom rsync path
  --bwlimit BWLIMIT, -b BWLIMIT
                        Bandwidth limit in KBPS.
  --ssh-port PORT, -P PORT
                        Custom ssh port.
  --exclude EXCLUDE [EXCLUDE ...], -E EXCLUDE [EXCLUDE ...]
                        Exclude pattern
  • Restore options
    --catalog, -C

    Select the backups folder (root).

    --backup-id

    Select backup id in the catalog.

    --last, -L

    Select last available backup in the catalog for same hostname or ip address.

    --user, -u

    User of remote machine where you want to restore files.

    --computer, -c

    Select the ip or hostname where to perform restore.

    --type, -t

    Type of operating system to put under backup:

    • Unix -> All UNIX os (Linux, BSD, Solaris).
    • Windows -> Windows Vista or higher with cygwin installed.
    • MacOS -> MacOSX 10.8 or higher.
    --timeout, -T

    Specify number of seconds of I/O timeout.

    --mirror, -m

    Mirror mode. If a file or folder not exist in destination, will delete it. Overwrite files.

    --skip-error, -e
     

    Skip error. Quiet mode.

    --rsync-path, -R
     

    Select a custom rsync path.

    --bwlimit, -b

    Bandwidth limit in KBPS.

    --ssh-port, -P

    Custom ssh port.

    --exclude, -E

    Exclude pattern. Follow rsync “Exclude Pattern Rules”

This is a few examples:

This command perform a restore on the same machine of the backup:

[email protected]$ bb restore --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0 --computer host1 --log
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
SUCCESS: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc [email protected]:/restore_2018_08_08__10_58
SUCCESS: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* [email protected]:/Users

Important

Without specifying the “type” flag that indicates the operating system on which the data are being retrieved, Butterfly Backup will select it directly from the catalog via the backup-id.

Now, select the last available backup on catalog; run this:

[email protected]$ bb restore --catalog /mnt/backup --last --computer host1 --log
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
SUCCESS: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc [email protected]:/restore_2018_08_08__10_59
SUCCESS: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* [email protected]:/Users

This example, is the same as the previous one, but restore to other machine and other operating system:

[email protected]$ bb restore --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0 --computer host2 --type Unix --log --verbose
INFO: Build a rsync command
INFO: Command flags are: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/etc? To continue [Y/N]? y
INFO: Build a rsync command
INFO: Command flags are: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log
Want to do restore path /mnt/backup/host1/2018_08_08__10_58/Users? To continue [Y/N]? y
Start restore on host2
INFO: rsync command: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc/* [email protected]:/etc
Start restore on host2
INFO: rsync command: rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* [email protected]:/home
building file list ...
26633 files to consider
...
...
sent 777.53K bytes  received 20 bytes  62.20K bytes/sec
total size is 6.66G  speedup is 8566.41
SUCCESS: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/etc/* [email protected]:/etc
SUCCESS: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2018_08_08__10_58/restore.log /mnt/backup/host1/2018_08_08__10_58/Users/* [email protected]:/home

Archive

Archive operations are used to store backups by saving disk space. Backups older than n days are compressed into a zip file.

[email protected]$ bb archive --help
usage: bb archive [-h] [--verbose] [--log] [--dry-run] --catalog CATALOG
                  [--days DAYS] --destination DESTINATION

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode

Archive options:
  --catalog CATALOG, -C CATALOG
                        Folder where is catalog file
  --days DAYS, -D DAYS  Number of days of archive retention
  --destination DESTINATION, -d DESTINATION
                        Archive destination path
  • Archive options
    --catalog, -C

    Select the backups folder (root).

    --days, -D

    Number of days for which you want to keep your backups. Default is 30.

    --destination, -d
     

    New destination for compress zipped backup.

Archive old backup of 3 days:

[email protected]$ bb archive --catalog /mnt/backup/ --days 3 --destination /mnt/archive/ --verbose --log
INFO: Check archive this backup f65e5afe-9734-11e8-b0bb-005056a664e0. Folder /mnt/backup/host2/2018_08_08__17_50
INFO: Check archive this backup 4f2b5f6e-9939-11e8-9ab6-005056a664e0. Folder /mnt/backup/host2/2018_08_04__07_26
SUCCESS: Delete /mnt/backup/host2/2018_08_04__07_26 successfully.
SUCCESS: Archive /mnt/backup/host2/2018_08_04__07_26 successfully.
[email protected]$ ls /mnt/archive
host1
[email protected]$ ls /mnt/archive/host1
2018_08_06__07_26.zip

The backup-id f65e5afe-9734-11e8-b0bb-005056a664e0 it is not considered, because it falls within the established time. The other, however, is zipped and deleted.

Lastly, let’s look in the catalog and see that the backup was actually archived:

[email protected]$ bb list --catalog /mnt/backup/ -i 4f2b5f6e-9939-11e8-9ab6-005056a664e0
Backup id: 4f2b5f6e-9939-11e8-9ab6-005056a664e0
Hostname or ip: host2
Type: Incremental
Timestamp: 2018-08-08 07:26:46
Start: 2018-08-08 07:26:46
Finish: 2018-08-08 08:43:45
OS: MacOS
ExitCode: 0
Path: /mnt/backup/host2/2018_08_04__07_26
Archived: True

Export

The export function is used to copy a particular backup to another path.

[email protected]$ bb export -h
usage: bb export [-h] [--verbose] [--log] [--dry-run] --catalog CATALOG
                 [--backup-id ID | --all] --destination DESTINATION [--mirror]
                 [--cut] [--include INCLUDE [INCLUDE ...] | --exclude EXCLUDE
                 [EXCLUDE ...]] [--timeout TIMEOUT] [--skip-error]
                 [--rsync-path RSYNC] [--bwlimit BWLIMIT] [--ssh-port PORT]

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Enable verbosity
  --log, -l             Create a log
  --dry-run, -N         Dry run mode

Export options:
  --catalog CATALOG, -C CATALOG
                        Folder where is catalog file
  --backup-id ID, -i ID
                        Backup-id of backup
  --all, -A             All backup
  --destination DESTINATION, -d DESTINATION
                        Destination path
  --mirror, -m          Mirror mode
  --cut, -c             Cut mode. Delete source
  --include INCLUDE [INCLUDE ...], -I INCLUDE [INCLUDE ...]
                        Include pattern
  --exclude EXCLUDE [EXCLUDE ...], -E EXCLUDE [EXCLUDE ...]
                        Exclude pattern
  --timeout TIMEOUT, -T TIMEOUT
                        I/O timeout in seconds
  --skip-error, -e      Skip error
  --rsync-path RSYNC, -R RSYNC
                        Custom rsync path
  --bwlimit BWLIMIT, -b BWLIMIT
                        Bandwidth limit in KBPS.
  --ssh-port PORT, -P PORT
                        Custom ssh port.
  • Export options
    --catalog, -C

    Select the backups folder (root).

    --backup-id, -i
     

    Select backup id in the catalog.

    --all, -A

    All backup

    --destination, -d
     

    Destination path.

    --mirror, -m

    Mirror backup on destination.

    --cut, -c

    Delete source like a move.

    --include, -I

    Include pattern. Follow rsync “Include Pattern Rules”

    --exclude, -E

    Exclude pattern. Follow rsync “Exclude Pattern Rules”

    --timeout, -T

    Specify number of seconds of I/O timeout.

    --mirror, -m

    Mirror mode. If a file or folder not exist in destination, will delete it. Overwrite files.

    --skip-error, -e
     

    Skip error. Quiet mode.

    --rsync-path, -R
     

    Select a custom rsync path.

    --bwlimit, -b

    Bandwidth limit in KBPS.

    --ssh-port, -P

    Custom ssh port.

Export a backup in other directory:

[email protected]$ bb export --catalog /mnt/backup/ --backup-id f0f700e8-0435-11e9-9e78-005056a664e0 --destination /mnt/backup/export --verbose
INFO: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
INFO: Build a rsync command
Start export host1
...
INFO: rsync command: rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
SUCCESS: Command rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1

Export all backup in other directory:

[email protected]$ bb export --catalog /mnt/backup/ --all --destination /mnt/backup/export --verbose
INFO: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
INFO: Build a rsync command
Start export host1
...
INFO: rsync command: rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
SUCCESS: Command rsync -ah --no-links -vP /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1

Export a backup with exclude pdf files:

[email protected]$ bb export --catalog /mnt/backup/ --backup-id f0f700e8-0435-11e9-9e78-005056a664e0 --destination /backup/export --verbose --exclude *.pdf
INFO: Export backup with id f0f700e8-0435-11e9-9e78-005056a664e0
INFO: Build a rsync command
Start export host1
...
INFO: rsync command: rsync -ah --no-links -vP --exclude=*.pdf /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1
SUCCESS: Command rsync -ah --no-links -vP --exclude=*.pdf /mnt/backup/host1/2018_12_20__10_02 /mnt/backup/export/host1

Donations

Donating is important. If you do not want to do it to me, do it to some companies that do not speculate. My main activity and the people of non-profit associations is to work for others, be they male or female, religious or non-religious, white or black or yellow or red, rich and poor. The only real purpose is to serve the humanity of one’s experience. Below you will find some links to do it. Thanks a lot.

For me

https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif

For Telethon

The Telethon Foundation is a non-profit organization recognized by the Ministry of University and Scientific and Technological Research. They were born in 1990 to respond to the appeal of patients suffering from rare diseases. Come today, we are organized to dare to listen to them and answers, every day of the year.

http://www.telethon.it/sites/all/themes/telethon/images/svg/logo.svg

Adopt the future