Backups

What other apps and distros do you use to round out your studio?

Moderators: MattKingUSA, khz

User avatar
lilith
Established Member
Posts: 1698
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Has thanked: 117 times
Been thanked: 57 times
Contact:

Backups

Post by lilith »

Wonder how you are making backups and what do you usually backup. I have two 250 GB SSDs in my computer (internal and external). The external SSD contains my music projects and I regularly make a backup of this folder to my internal SSD via rsync (grsync).

On my internal SSD I have a folder which contains all my private docs, plugin licences, etc. This I also backup to the external SSD via grsync. So, I have the docs and my music stuff on two SSDs always. From time to time I copy it also to an external HDD.

I'm not doing any backups yet of my plugins, plugin folders or even of the whole system. I did this in the past with windows, but I never made use of it as I always made a new install. Is there any program that you can recommend? I'm on DEBIAN stable.
User avatar
sysrqer
Established Member
Posts: 2519
Joined: Thu Nov 14, 2013 11:47 pm
Has thanked: 319 times
Been thanked: 148 times
Contact:

Re: Backups

Post by sysrqer »

I just rsync everything from the internal drives to an external one.
User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

Re: Backups

Post by khz »

I'm just backing up (c/p or rsync) my "/home" directory.
I reinstall the system if necessary, very rarely the case with Debian stable, because it is fast and I prefer to redo all my optimizations. This time hopefully less / better: learn.
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
User avatar
lilith
Established Member
Posts: 1698
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Has thanked: 117 times
Been thanked: 57 times
Contact:

Re: Backups

Post by lilith »

rsyncing /home to an external drive is what I'm currently doing :D
User avatar
sysrqer
Established Member
Posts: 2519
Joined: Thu Nov 14, 2013 11:47 pm
Has thanked: 319 times
Been thanked: 148 times
Contact:

Re: Backups

Post by sysrqer »

Oh did you mean system directories then? Like the packages you have installed?
I think there's a way to back that up with apt/dpkg and be able to restore them

Code: Select all

dpkg --get-selections > package_list
sudo dpkg --clear-selections && sudo dpkg --set-selections < package_list
sudo apt-get update && sudo apt-get dselect-upgrade
sudo apt-get autoremove
Never tried this though.
User avatar
lilith
Established Member
Posts: 1698
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Has thanked: 117 times
Been thanked: 57 times
Contact:

Re: Backups

Post by lilith »

sysrqer wrote:Oh did you mean system directories then? Like the packages you have installed?
I think there's a way to back that up with apt/dpkg and be able to restore them

Code: Select all

dpkg --get-selections > package_list
sudo dpkg --clear-selections && sudo dpkg --set-selections < package_list
sudo apt-get update && sudo apt-get dselect-upgrade
sudo apt-get autoremove
Never tried this though.
This is what was my idea, yes. But, I also never used it (in windows I used acronis).
The question is also how good it works if such a backup is really needed and how good you can rely on that.
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: Backups

Post by merlyn »

I backup drives because it's drives, not files, that fail.

It might not be the most efficient way to do it, but here's my approach :

You buy a drive the same size as the drive you want to backup. Then use a live USB, I use Knoppix, to boot up. Then use dd to copy the whole drive onto the new one. dd creates a clone, so all the UUIDs are identical. You can't then use both drives in the same system, so you put the backup somewhere safe. Because it isn't powered up my thinking is it's extremely unlikely to fail. If anything happens to your working drive, the backup is a drop-in replacement.
Luc
Established Member
Posts: 741
Joined: Fri Mar 27, 2015 1:04 pm
Been thanked: 1 time

Re: Backups

Post by Luc »

merlyn wrote:If anything happens to your working drive, the backup is a drop-in replacement.
That is my approach too, though a little bit different.

I divide the disk into partitions. One for the system and others for the data. I have multiple data partitions for historical reasons. There were times when I had to use multiple old disks for backup so I had to keep manageable partitions. But one single data partition will probably do for most people.

Ideally, I should have two disks of the exact same size. Rather than cloning the disk with dd, I like to take a snapshot of the OS file system with xfscopy. It's good because xfscopy saves a lot of space. I only do that once or twice a year because it requires rebooting with a boot CD and I hate that. I do that so I can perfectly restore my entire system to some previous state. That's light-years ahead of formatting and reinstalling the entire circus show. You just reboot, restore it, reboot and everything is impeccably fine. Note that xfscopy snapshots can also be mounted and browsed just like ISO images. That system snapshot is then kept in a data partition, which can always be copied with rsync along with everything else while the system is running and I'm doing other things. Everything else is copied frequently to the external disk with rsync.

However...
merlyn wrote:You can't then use both drives in the same system, so you put the backup somewhere safe. Because it isn't powered up my thinking is it's extremely unlikely to fail.
I don't know how "unlikely" that is. I've had disks fail after not running for too long. In fact, one of my previous backup HDs got corrupted not too long ago after sitting idly on a shelf for some time. It wouldn't mount and xfs_repair couldn't recover everything. So I had to buy a new disk in an emergency and "clone" (as described above) the production disk immediately.

Maybe I should note that I used to live far, very far from the ocean, but then I moved to another city where I live VERY close to the beach now, and I can tell some of my electronic stuff is rotting. I try to keep them in boxes with moisture absorbers, but I'm not sure it's working. I don't really know what to do about that.

Beware of SSDs. They die very suddently. SMART can't do much to detect their deaths beforehand.
User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

Re: Backups

Post by khz »

. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
User avatar
lilith
Established Member
Posts: 1698
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Has thanked: 117 times
Been thanked: 57 times
Contact:

Re: Backups

Post by lilith »

Thanks for the commands. I think just backuping the /home folder should do the job for me. Since yesterday I'm struggling with rsync or even cp. When rsyncing my /home to an external USB 2.0 HDD which is connected to an USB 3.0 port, the system slows down and I get drop outs in the audio. It's not a big issue, but I want to understand the reason for this. I read about the nice command to set the priority for processes lower, but even with

Code: Select all

nice -n 19 rsync -r -t -v --progress --delete -l -s /home /media/marco/Elements/homebackup_test
I get drop outs, etc. The external drive is NTFS formatted and a Western Digital test told me it's fine.
Luc
Established Member
Posts: 741
Joined: Fri Mar 27, 2015 1:04 pm
Been thanked: 1 time

Re: Backups

Post by Luc »

Are you writing to an NTFS file system from Linux? Is that considered stable now? I really don't know. Sounds dangerous.
User avatar
lilith
Established Member
Posts: 1698
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Has thanked: 117 times
Been thanked: 57 times
Contact:

Re: Backups

Post by lilith »

Luc wrote:Are you writing to an NTFS file system from Linux? Is that considered stable now? I really don't know. Sounds dangerous.
öhm.. yes. Was it considered to be unstable?

edit: Hmm.. might be a problem with the user rights when backuping /home
User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

Re: Backups

Post by khz »

https://wiki.debian.org/NTFS Should work. IMHO Do you use ntfs because you still have Windows? Otherwise I would use a Linux https://wiki.debian.org/FileSystem
or fat in case of need.
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
User avatar
lilith
Established Member
Posts: 1698
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Has thanked: 117 times
Been thanked: 57 times
Contact:

Re: Backups

Post by lilith »

I could reformat the device and will do so. I don't have any windows anymore, but the device is from 2010.

This is nice and clear:

https://www.youtube.com/watch?v=H7tIN9SdByQ
User avatar
sysrqer
Established Member
Posts: 2519
Joined: Thu Nov 14, 2013 11:47 pm
Has thanked: 319 times
Been thanked: 148 times
Contact:

Re: Backups

Post by sysrqer »

lilith wrote:
Luc wrote:Are you writing to an NTFS file system from Linux? Is that considered stable now? I really don't know. Sounds dangerous.
öhm.. yes. Was it considered to be unstable?

edit: Hmm.. might be a problem with the user rights when backuping /home
Writing to ntfs is not unstable but (I believe) you lose the permissions which could be an issue potentially. More of a concern is performance which you have noticed. I struggled with this for a long time and the only explanation I ever found was the fact that ntfs uses journaling and doesn't write in the same way as linux formats, even though linux formats do have journaling as well, I guess it's more aggressive than ntfs's implementation. I researched this a lot and tried many different attempts to fix it including mountpoint and copying options but never really found much to make a huge difference but I know that it can bring your system to a halt sometimes. I have come to the conclusion that it is just the way that linux deals with that filesystem rather than an inherent problem as windows doesn't have such a bad performance with it, predictably, although it can be problematic there too. There are a number of things you can try, depending on how deep you want to go, but unless you need to it is better just to backup to a similar or same filesystem you are using.
Post Reply