Frequent asked questions about raspiBackup. Every new user of raspiBackup should read all questions and answers.

0) Where does raspiBackup come from?

I run three Raspberries at home. Two of them run 7/24 all the time as LAN servers. It's common practice to backup a server on a regular base just in case there are any failures. In particular the SD card of the Raspberry tends to fail after some time. That's why I wrote a small script which created a dd backup. Then a tar backup was possible to create because the dd backup saves the whole SD card all the time even there is only a subset of the space used. Finally a rsync backup was introduced to use hardlinks and save backupspace. raspiBackup helped me multiple times to revover from a broken SD card so I thought the script may also be very helpful for other people running Raspberries and decided to publish raspiBackup.

1) Does it work to create a backup from a running system? Shouldn't the system be stopped first?

The most reliable backup is a backup from a stopped system. Unfortunately this cannot be done on a regular base with cron automatically. If you stop all active services as mysql, samba, nfs, seafile, Owncloud, Webserver and all other active services before starting the backup in order to keep all the data consistent the backup can be used to restore the system. If these services are not stopped there is a high probability that the backup will become inconsistent. Parameter -a and -o help to achieve for this. See also FAQ 18. As an alternative a sample wrapper script can be used and extended .

2) How can I restore a backup?

raspiBackup can restore all created backups (See this page for details). raspiBackup runs on Linux only. Windows user can use windisk32imager to restore a dd backup. For all other backup types a Linux system is required.

Fortunately your Raspberry can be used to restore the backup with raspiBackup. Install a Linux OS (e.g. raspbian) on a second SD card, start it and install raspiBackup.  Now connect an external SD card reader and the drive with the backup and use raspiBackup to restore the backup to the external SD card. Shutdown the system, insert the external SD card with the restored backup and start your Raspberry again.

3) What will be saved and restored by raspiBackup?

raspiBackup can save two partitionens in the normal backup mode: The boot and the rootpartition of the SD card. If the rootpartition was moved to an external partition this external partition is also saved. An unlimited number of partitions on the SD card will be saved in the partition oriented mode. Additional external partitions will not be saved in this mode. This mode is usefull for NOOBs images.

4) Which Linux methods to create backups can be used?

You can use dd, tar and rsync to backup the data.  dd and tar backups can be zipped also. See the following page for pros and cons of the different backup methods.

5) Can the backup created by raspiBackup be used to restore the backup manually?

That was a requirement for raspiBackup: It should be possible to restore the backup with Linux means only.

raspiBackup creates readable files of the output of the Linux commands sfdisk, blkid and fdisk of the SD card. This allows to restore the partition layout of the original SD card with Linux means. Then the partition backups can be restored into these partition using the corresponding linux backup tools.

6) Can a backup be restored on a smaller or larger SD card than the original SD card?

That works perfectly for larger SD cards. But the additional space will not be used. To use the additional space a Linux repartitioning tool has to be used when the backup was restored.

Starting with version 0.6.1.3 this works perfectly for smaller and larger SD cards. The root partition size is adjusted accordingly. It's either truncated if the SD card is smaller than the original SD card or expanded to use the whole SD card size if it's larger. If it's expanded no repartitioning is needed any more. This works for tar and rsync backup in the normal backup mode.

If the existing partitioning of the backup should be used for the restore the option -1 (one) disables this automatic repartitioning. Option -0 (zero) allows to use the existing partitioning of the target SD card.

7) How can I control the partitioning of the target SD card?

There exist two options which give you full control of the way the new SD card is partitioned. Option -1 (one) forces raspiBackup to create the same partitioning on the target SD card even the partitions are smaller or larger than the size of the target SD card. Option -0 (zero) disables partitioning by raspBackup. This allows to create the partitioning and formating on the target SD card upfront and these paritions are used as is by raspiBackup during restore.

8) On which devices can raspiBackup save the backup?

In general on every device which can be mounted on Linux:

  • External USB Stick
  • External USB disk
  • Synology
  • cifs/samba networking disk
  • nfs networking disk
  • sshfs networking disk
  • webdav networking disk
  • ftpfs networking disk

9) There is some data on my system which raspiBackup doesn't save. How can I include this data in the backup?

Use one of the following approaches:

a) Use a wrapper script  (see here) and write your own code which is executed before and after raspiBackup and saves additional data

b) Use extension points which are called before and after the backup function to save the data. There exist two sample extensions (see here) which report the CPU temperature before and after a backup run and the used memory. An eMailExtension can be used to plug in your own eMailClient.

10) Which eMailClients are supported by raspiBackup?

mailx, sendEmail and ssmtp are supported. The eMailExtension plugpoint can be used to plugin every other eMailClient (Details see here).

11) My mailProgram is not supported by raspiBackup. How can I now get eMails from raspiBackup?

Use the eMalExtension plugpoint to write a small extension which will be used to send the emails from raspiBackup. The existing extensionsamples include one sample code for the mailx eMailClient.

12) Ich have a question about raspiBackup. How do I get an answer?

At the bottom of every webspage you can create comments. Just ask your question there. To eliminate spam the comments are checked manually and that's why it usually takes one day until the comment is published and will be answered. The eMailAdress which is optional, will not be published. It's required to get update notifications about new comments and may also be used to get in contact offline.

13) I found a bug in raspiBackup. How can I report the bug and when do I get a fix?

nichtEvery software can have bugs and thus it's possible for raspiBackup also. Report the bug in a comment on the webpage and add your eMailAdress. Now you will be either asked in followon comments to provide additional details about the bugor you will be contacted via eMail and asked to create a detailed log and send it via eMail for problemanalysis. Usually you will get a fix quite fast sent in an eMail and should verify the fix. If the fix was verified successfully it will be published.

14) Do I get any notification if there is a new version available?

raspiBackup checks in every invocation whether there exist a new version. A message will be written and the notification eMail will have a smiley ;-) in it's mail title. Now you can check on this page what's new in the new version and update raspiBackup with parameter -U.

15) How can I revert to the previous raspiBackup version if I detect the new version doesn't work as expected when I upgraded?

raspiBackup creates a backup copy every time an upgrade is done with parameter -U. Option -V can be used to revert to a previous version. A list of all saved raspiBackup versions will be presented and the version to restore can be selected.

16) I have a 32GB SD card but need only 8GB. The dd backup uses 32GB instead of 8GB.

dd backup backups the whole SD card Starting with version 0.6.1.3 there is a configuration parameter

DEFAULT_DD_BACKUP_SAVE_USED_PARTITIONS_ONLY which causes raspiBackup to save only the space used by defined partitions. So you just have to resize the 32GB partition to 8GB with gparted or other repartitiontools and the backup will only use 8GB. Current partiton sizes can be checked with lsblk command.

17) How can I check whether rsync backup uses hardlinks and backup space is saved?

From time to time raspiBackup users report they cannot use rsync backup because raspiBackup complaints the backupspace doesn't support hardlinks. Hardlinks are used successfully by raspiBackup with rsync if a local attached USB stick or a local attached USB disk formatted with ext3/ext4 or a remote disk mounted via nfs is used. samba and sshfs don't support hardlinks.

Command du -sh * displays the used backup space and du -shl * displays how much space would be used if no hardlinks are used.

Example:

root@raspberrypi:/media/nas-backup/raspberrypi# du -sh *
4,5G raspberrypi-rsync-backup-20160822-184617
4,5M raspberrypi-rsync-backup-20160822-190436
5,2M raspberrypi-rsync-backup-20160822-195250
5,7M raspberrypi-rsync-backup-20160822-201610

root@raspberrypi:/media/nas-backup/raspberrypi# du -shl *
4,7G raspberrypi-rsync-backup-20160822-184617
4,7G raspberrypi-rsync-backup-20160822-190436
4,7G raspberrypi-rsync-backup-20160822-195250
4,7G raspberrypi-rsync-backup-20160822-201610

18) Which services have to be stopped before starting the backup and started again afterwards?

All services which keep any states in databases or in memory or on filesystemhave to be stopped. Otherwise inconsistent data will be saved and this will be detected when the backup is restored and the backup is unusable. raspiBackup offers option -o to stop services before starting the backup and option -a to start the services afterwards.

Following services have to be stopped with option -o in any case:

Service Stop command
nfs service nfs-kernel-serv er stop
Samba service samba stop
Pilight service pilight stop
Cups service cups stop
Minidlna service minidlna stop
Apache service apache2 stop
Wordpress service wordpress stop
nginx service nginx stop
seafile service seafile stop
Owncloud see Apache
mysql service mysql stop
FHEM /etc/init.d/fhem stop
cron service cron stop

 

To start the services again use option -a and use the reverse service sequence used with -o. If you are dead certain you don't need to stop and start services use ":" as parameter for option -a and -o.

19) Which format should be used on the backup partition?

In general every filesystem can be used which can be mounted on LInux. But there are following constraints:

- rsync backup reuqires supprot for hardlinks which is available with ext3/4. If an ext4 filesystem is exported via samba there is no support for hardlinks. NFS is an alternative.

- FAT32 can store files with max 4GB only. A dd backup will become as large as the SD card is (Configuration option DEFAULT_DD_BACKUP_SAVE_USED_PARTITIONS_ONLY can reduce the size) and is in genaral larger than 4GB. It's similar for tar backups which become larger than 4GB quite quickly. An alternative may be NTFS on windows.

General suggestion: Use ext3/4 if possible. On Linux use NFS for remote drives. On windows NTFS for exported samba network drives. Use FAT32 only if you are sure the backup files will not become larger than 4GB,

20) I have issues to save my backups on Synology. How can I fix them?

There are various raspiBackup users who successfully save their backups on Synology via nfs. Unfortunately I don't have a Synology and cannot help much on this. But there exists a dedicated page for Synology where one raspiBackup user told me what he configured on his Synology in order to save his backups successfully.

21) The bootpartition doesn't change. Why is the bootpartition nevertheless saved in every backup again?

In about 98% situations this is true. But a firmware update can change the bootpartition. There is a configuration parameter DEFAULT_LINK_BOOTPARTITIONFILES which causes raspiBackup to link the bootpartition backups with hardlinks if they are supported. That way every backup is 60MB smaller. But the bootpartition is nevertheless saved all the time to check whether there are changes to the previous backup and then replaced with a hardlink. This option is useful only if you use a very small backup device.

 

Add comment

Spam Kommentare sind sinnlos !
Vor der Publizierung wird jeder Beitrag auf Spam geprüft. Leider dauert es deswegen bis ein Kommentar sichtbar wird. Dafür erhält aber kein Subscriber Spam eMails.
Die eMail ist optional und nicht öffentlich sichtbar. Sie ist notwendig um eMail Benachrichtigungen zu Antworten auf Kommentare zu erhalten und wird u.U. genutzt um offline zu kommunizieren.

Spam comments are useless !
Any comment will be reviewed first and checked for spam. Unfortunately this delays the publishing of comments but will protect subscribers from spam.
eMail is optional and hidden and is required get update notifications for any comments and may be used to contact you offline


Comments   

0 #6 framp 2016-10-29 20:06
Hello Alfred,

Quoting Alfred:
...Your test now returns a "2" and I conclude that the mount is operational. All is not well however. After a few attempts I concluded that the previous backups [made with an SSHFS mount] had to be deleted.
Sounds reasonable to delete old backups. I never tested a mixture of sshfs backups and nfs backups.
Quoting Alfred:
...1. On the first partition I get a large number of errors like:

"rsync: chown "/mnt/nas/araminta/araminta-rsync-backup-20161029-190948/mmcblk0p1/overlays/.w1-gpio-overlay.dtb.ansSC4" failed: Operation not permitted (1)"
These are probably caused by the fact that the script is trying to backup a device tree. I did not succeed in excluding this part of the files system.

These errors are created by rsync which is used by raspiBackup to backup your Raspi. I did some research and it sounds you don't have full write access as root on your Synology via nfs. I unfortunately don't have a Synology and don't know how to configure Synology for this. But I just translated an existing page written in German where I quoted a raspiBackup user who successfully managed to get raspiBackup to save the backup via nfs on a Synology. Maybe you find some useful hints on this page.

Quoting Alfred:
...2. A more serious problem is this: after approx. 148Mb the backup seems to stall [it should be approx. 8Gb] and never completes. I killed the script with CTRL-C after 75min. There are no hints in the logfile.

This may be related to the first issue you faced. Please fix the first issue first. If the second issue still persist then please add options -l 1 -m 1 when calling raspiBackup and send the log to me for further analysis. Please use my eMail listed on the contact page.

Hope this answers your questions.

Cu framp
Quote
0 #5 Alfred 2016-10-29 19:26
Hi framp,

Thanks for getting back to me.
I ran the test you suggested and got back a "1" and therefore abandoned SSHFS.

After doing some research in order to allow my system to execute an NFS mount, I succeeded after entering the following commands:

sudo update-rc.d rpcbind enable
sudo update-rc.d nfs-common enable
sudo service rpcbind restart

Your test now returns a "2" and I conclude that the mount is operational. All is not well however. After a few attempts I concluded that the previous backups [made with an SSHFS mount] had to be deleted.

The backup does not run to completion and I encounter the following 2 problems:

1. On the first partition I get a large number of errors like:

"rsync: chown "/mnt/nas/arami nta/araminta-rs ync-backup-2016 1029-190948/mmc blk0p1/overlays /.w1-gpio-overl ay.dtb.ansSC4" failed: Operation not permitted (1)"
These are probably caused by the fact that the script is trying to backup a device tree. I did not succeed in excluding this part of the files system.

2. A more serious problem is this: after approx. 148Mb the backup seems to stall [it should be approx. 8Gb] and never completes. I killed the script with CTRL-C after 75min. There are no hints in the logfile.

I don't know if the 2 problems are related. Please let me know what additional tests I can perform.
Quote
0 #4 framp 2016-10-28 20:13
Hello Alfred,

raspiBackup verifies whether hardlinks can be used by creating a file and try to create a hardlink to this file on the backup target directory.

For sure hardlinks are supported with local disks using ext3/4 and nfs attached remote space. samba doesn't work even there is ext3/4 used. I found pages in the net which state hardlinks are not supported by sshfs. On the other side I found pages stating it should work.
The raspiBackup hardlink test fails for you and because you disabled the test you get full backup copies all the time.
I suggest to use nfs which is supported by Synology instead of sshfs. Then you will be able to use hardlinks.
Otherwise you have to search in the net for an exact description how to use hardlinks with sshfs.
You can execute the test manually also on your backup directory:

touch test.hlinkfile
cp -l test.hlinkfile test.hlinklink
echo $(ls -la test.hlinkfile | cut -f 2 -d ' ')

You will get 2 if hardlinks are working.

Hope this answers your questions.

CU framp
Quote
0 #3 Alfred 2016-10-28 15:00
I installed and ran the package. Everything seems to go smoothly once I used the -5 parameter. It won't run without even though I'm using a hardlink compatible version of rsync:

-------------------------------------------------------------------------------
pi@dosadi:~ $ rsync --version
rsync version 3.1.1 protocol version 31
Copyright (C) 1996-2014 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, prealloc

-------------------------------------------------------------------------------

The backup volume is an SSHFS-mounted Synology NAS with an ext4 filesystem mounted like this:

sudo sshfs admin@diskstati on:/RPI/rpi-ful l-backups /mnt/nas -o IdentityFile=~/.ssh/id_rsa,idmap=user,reconnect,uid=1000,gid=1000,allow_other

However, it seems that hardlinks are not actually used and each time it makes a full backup, taking approx. 1 hr. to run:

admin@DiskStation:/volume1/RPI/rpi-full-backups/dosadi$ du -sh *
2.2G dosadi-rsync-backup-20161027-190900
2.2G dosadi-rsync-backup-20161027-204745
2.2G dosadi-rsync-backup-20161028-122931
2.2G dosadi-rsync-backup-20161028-135330
admin@DiskStation:/volume1/RPI/rpi-full-backups/dosadi$ du -shl *
2.2G dosadi-rsync-backup-20161027-190900
2.2G dosadi-rsync-backup-20161027-204745
2.2G dosadi-rsync-backup-20161028-122931
2.2G dosadi-rsync-backup-20161028-135330

Any idea how this can be fixed?
Quote
0 #2 framp 2016-08-13 09:41
Hi Clark,

I tried to answer your question in FAQ #3. Looks like there is some information missing :oops:

raspiBackups initial purpose was to backup the SD card only. At some point in time a user asked whether it's possible to enhance raspiBackup to save an external root partition also (You modify /boot/cmdline.t xt to use an external SD card or use raspiSD2USB.py https://www.linux-tips-and-tricks.de/en/raspberry/475-how-to-move-a-root-partition-of-a-raspberry-pi-to-an-external-partition). I found this request to be a common usecase and added this functionality.
The partition oriented mode which allows to backup more than 2 partitions just saves the SD card partitions.
If you mount and use in addition other external partitions on your raspi as data disks they are not backuped. But you can use the wrapper script I provide (https://www.linux-tips-and-tricks.de/de/raspberry/23-pi-erstellt-automatisch-backups-von-sich-selbst-pi-creates-automatic-backups-of-itself#wrapper) to backup and restore them when raspiBackup finishes it's backup processing.

Hope this answers your question.

Cu framp
Quote
0 #1 Clark 2016-08-12 23:53
Is there a way to backup data from an external hard drive instead of the (root) partition(s) of the raspberry´s sd card?
Quote