Backups
Moderators: MattKingUSA, khz
- lilith
- Established Member
- Posts: 1706
- Joined: Fri May 27, 2016 11:41 pm
- Location: bLACK fOREST
- Has thanked: 122 times
- Been thanked: 57 times
- Contact:
Backups
Wonder how you are making backups and what do you usually backup. I have two 250 GB SSDs in my computer (internal and external). The external SSD contains my music projects and I regularly make a backup of this folder to my internal SSD via rsync (grsync).
On my internal SSD I have a folder which contains all my private docs, plugin licences, etc. This I also backup to the external SSD via grsync. So, I have the docs and my music stuff on two SSDs always. From time to time I copy it also to an external HDD.
I'm not doing any backups yet of my plugins, plugin folders or even of the whole system. I did this in the past with windows, but I never made use of it as I always made a new install. Is there any program that you can recommend? I'm on DEBIAN stable.
On my internal SSD I have a folder which contains all my private docs, plugin licences, etc. This I also backup to the external SSD via grsync. So, I have the docs and my music stuff on two SSDs always. From time to time I copy it also to an external HDD.
I'm not doing any backups yet of my plugins, plugin folders or even of the whole system. I did this in the past with windows, but I never made use of it as I always made a new install. Is there any program that you can recommend? I'm on DEBIAN stable.
- khz
- Established Member
- Posts: 1648
- Joined: Thu Apr 17, 2008 6:29 am
- Location: German
- Has thanked: 42 times
- Been thanked: 92 times
Re: Backups
I'm just backing up (c/p or rsync) my "/home" directory.
I reinstall the system if necessary, very rarely the case with Debian stable, because it is fast and I prefer to redo all my optimizations. This time hopefully less / better: learn.
I reinstall the system if necessary, very rarely the case with Debian stable, because it is fast and I prefer to redo all my optimizations. This time hopefully less / better: learn.
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
- I don't care about the freedom of speech because I have nothing to say.
- lilith
- Established Member
- Posts: 1706
- Joined: Fri May 27, 2016 11:41 pm
- Location: bLACK fOREST
- Has thanked: 122 times
- Been thanked: 57 times
- Contact:
- sysrqer
- Established Member
- Posts: 2527
- Joined: Thu Nov 14, 2013 11:47 pm
- Has thanked: 320 times
- Been thanked: 153 times
- Contact:
Re: Backups
Oh did you mean system directories then? Like the packages you have installed?
I think there's a way to back that up with apt/dpkg and be able to restore them
Never tried this though.
I think there's a way to back that up with apt/dpkg and be able to restore them
Code: Select all
dpkg --get-selections > package_list
sudo dpkg --clear-selections && sudo dpkg --set-selections < package_list
sudo apt-get update && sudo apt-get dselect-upgrade
sudo apt-get autoremove
- lilith
- Established Member
- Posts: 1706
- Joined: Fri May 27, 2016 11:41 pm
- Location: bLACK fOREST
- Has thanked: 122 times
- Been thanked: 57 times
- Contact:
Re: Backups
This is what was my idea, yes. But, I also never used it (in windows I used acronis).sysrqer wrote:Oh did you mean system directories then? Like the packages you have installed?
I think there's a way to back that up with apt/dpkg and be able to restore them
Never tried this though.Code: Select all
dpkg --get-selections > package_list sudo dpkg --clear-selections && sudo dpkg --set-selections < package_list sudo apt-get update && sudo apt-get dselect-upgrade sudo apt-get autoremove
The question is also how good it works if such a backup is really needed and how good you can rely on that.
-
- Established Member
- Posts: 1392
- Joined: Thu Oct 11, 2018 4:13 pm
- Has thanked: 168 times
- Been thanked: 247 times
Re: Backups
I backup drives because it's drives, not files, that fail.
It might not be the most efficient way to do it, but here's my approach :
You buy a drive the same size as the drive you want to backup. Then use a live USB, I use Knoppix, to boot up. Then use dd to copy the whole drive onto the new one. dd creates a clone, so all the UUIDs are identical. You can't then use both drives in the same system, so you put the backup somewhere safe. Because it isn't powered up my thinking is it's extremely unlikely to fail. If anything happens to your working drive, the backup is a drop-in replacement.
It might not be the most efficient way to do it, but here's my approach :
You buy a drive the same size as the drive you want to backup. Then use a live USB, I use Knoppix, to boot up. Then use dd to copy the whole drive onto the new one. dd creates a clone, so all the UUIDs are identical. You can't then use both drives in the same system, so you put the backup somewhere safe. Because it isn't powered up my thinking is it's extremely unlikely to fail. If anything happens to your working drive, the backup is a drop-in replacement.
Re: Backups
That is my approach too, though a little bit different.merlyn wrote:If anything happens to your working drive, the backup is a drop-in replacement.
I divide the disk into partitions. One for the system and others for the data. I have multiple data partitions for historical reasons. There were times when I had to use multiple old disks for backup so I had to keep manageable partitions. But one single data partition will probably do for most people.
Ideally, I should have two disks of the exact same size. Rather than cloning the disk with dd, I like to take a snapshot of the OS file system with xfscopy. It's good because xfscopy saves a lot of space. I only do that once or twice a year because it requires rebooting with a boot CD and I hate that. I do that so I can perfectly restore my entire system to some previous state. That's light-years ahead of formatting and reinstalling the entire circus show. You just reboot, restore it, reboot and everything is impeccably fine. Note that xfscopy snapshots can also be mounted and browsed just like ISO images. That system snapshot is then kept in a data partition, which can always be copied with rsync along with everything else while the system is running and I'm doing other things. Everything else is copied frequently to the external disk with rsync.
However...
I don't know how "unlikely" that is. I've had disks fail after not running for too long. In fact, one of my previous backup HDs got corrupted not too long ago after sitting idly on a shelf for some time. It wouldn't mount and xfs_repair couldn't recover everything. So I had to buy a new disk in an emergency and "clone" (as described above) the production disk immediately.merlyn wrote:You can't then use both drives in the same system, so you put the backup somewhere safe. Because it isn't powered up my thinking is it's extremely unlikely to fail.
Maybe I should note that I used to live far, very far from the ocean, but then I moved to another city where I live VERY close to the beach now, and I can tell some of my electronic stuff is rotting. I try to keep them in boxes with moisture absorbers, but I'm not sure it's working. I don't really know what to do about that.
Beware of SSDs. They die very suddently. SMART can't do much to detect their deaths beforehand.
- khz
- Established Member
- Posts: 1648
- Joined: Thu Apr 17, 2008 6:29 am
- Location: German
- Has thanked: 42 times
- Been thanked: 92 times
Re: Backups
- The Debian Administrator's Handbook - 9.10. Backup https://debian-handbook.info/browse/sta ... ackup.html (German: Das Debian Administrationshandbuch - 9.10. Datensicherung https://debian-handbook.info/browse/de- ... ackup.html, (https://debiananwenderhandbuch.de/))
- 10.2. Backup and recovery https://www.debian.org/doc/manuals/debi ... d_recovery
- BackupAndRecovery https://wiki.debian.org/BackupAndRecovery
- (GNU/Linux Debian Users' Manuals >> https://www.debian.org/doc/user-manuals)
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
- I don't care about the freedom of speech because I have nothing to say.
- lilith
- Established Member
- Posts: 1706
- Joined: Fri May 27, 2016 11:41 pm
- Location: bLACK fOREST
- Has thanked: 122 times
- Been thanked: 57 times
- Contact:
Re: Backups
Thanks for the commands. I think just backuping the /home folder should do the job for me. Since yesterday I'm struggling with rsync or even cp. When rsyncing my /home to an external USB 2.0 HDD which is connected to an USB 3.0 port, the system slows down and I get drop outs in the audio. It's not a big issue, but I want to understand the reason for this. I read about the nice command to set the priority for processes lower, but even with
I get drop outs, etc. The external drive is NTFS formatted and a Western Digital test told me it's fine.
Code: Select all
nice -n 19 rsync -r -t -v --progress --delete -l -s /home /media/marco/Elements/homebackup_test
Re: Backups
Are you writing to an NTFS file system from Linux? Is that considered stable now? I really don't know. Sounds dangerous.
- lilith
- Established Member
- Posts: 1706
- Joined: Fri May 27, 2016 11:41 pm
- Location: bLACK fOREST
- Has thanked: 122 times
- Been thanked: 57 times
- Contact:
Re: Backups
öhm.. yes. Was it considered to be unstable?Luc wrote:Are you writing to an NTFS file system from Linux? Is that considered stable now? I really don't know. Sounds dangerous.
edit: Hmm.. might be a problem with the user rights when backuping /home
- khz
- Established Member
- Posts: 1648
- Joined: Thu Apr 17, 2008 6:29 am
- Location: German
- Has thanked: 42 times
- Been thanked: 92 times
Re: Backups
https://wiki.debian.org/NTFS Should work. IMHO Do you use ntfs because you still have Windows? Otherwise I would use a Linux https://wiki.debian.org/FileSystem
or fat in case of need.
or fat in case of need.
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
- I don't care about the freedom of speech because I have nothing to say.
- lilith
- Established Member
- Posts: 1706
- Joined: Fri May 27, 2016 11:41 pm
- Location: bLACK fOREST
- Has thanked: 122 times
- Been thanked: 57 times
- Contact:
Re: Backups
I could reformat the device and will do so. I don't have any windows anymore, but the device is from 2010.
This is nice and clear:
https://www.youtube.com/watch?v=H7tIN9SdByQ
This is nice and clear:
https://www.youtube.com/watch?v=H7tIN9SdByQ
- sysrqer
- Established Member
- Posts: 2527
- Joined: Thu Nov 14, 2013 11:47 pm
- Has thanked: 320 times
- Been thanked: 153 times
- Contact:
Re: Backups
Writing to ntfs is not unstable but (I believe) you lose the permissions which could be an issue potentially. More of a concern is performance which you have noticed. I struggled with this for a long time and the only explanation I ever found was the fact that ntfs uses journaling and doesn't write in the same way as linux formats, even though linux formats do have journaling as well, I guess it's more aggressive than ntfs's implementation. I researched this a lot and tried many different attempts to fix it including mountpoint and copying options but never really found much to make a huge difference but I know that it can bring your system to a halt sometimes. I have come to the conclusion that it is just the way that linux deals with that filesystem rather than an inherent problem as windows doesn't have such a bad performance with it, predictably, although it can be problematic there too. There are a number of things you can try, depending on how deep you want to go, but unless you need to it is better just to backup to a similar or same filesystem you are using.lilith wrote:öhm.. yes. Was it considered to be unstable?Luc wrote:Are you writing to an NTFS file system from Linux? Is that considered stable now? I really don't know. Sounds dangerous.
edit: Hmm.. might be a problem with the user rights when backuping /home