LinuxMusicians

check if you have quotas and snapshots on your /home

https://unix.stackexchange.com/question ... pshot-size

Largos wrote: ↑Wed Jan 04, 2023 11:56 pm

Are you sure you've really deleted all the snapshots? How are you making them?

I think you may have missed some of my edits on my previous few posts. I've also updated the start of the opening post.

Anyway, this issue is continuing to happen now after no snapshots are being created. I was using 'Btrfs assistant' to create them but I deleted them all with the app and disabled them. I then double checked the folder where it was creating them (admin:///.snapshots/) and there is nothing there. The only way I can see this being to do with snapshots is if there is something else creating them and putting them somewhere esle but I cannot think what that would be..

I've been testing just now and found these things:

1 . If I add files to the drive, the filesystem size increases and free space decreases proportionately. If I remove these same files, the inverse happens.

2 . If I delete any existing files from the file system, the file system shrinks but no space is freed up. Because of this, I've lost 100s of GBs.

Kott wrote: ↑Wed Jan 04, 2023 11:59 pm

check if you have quotas and snapshots on your /home

https://unix.stackexchange.com/question ... pshot-size

So I'm supposed to run

sudo btrfs quota enable /home

which will let me check..?

Edit: I've figured out that quotas are not enabled by running

sudo btrfs qgroup show /

From what I can tell, I'm best of not enabling it either.

Did you run

sudo btrfs subvolume show /home/

Were there any snapshots listed at the bottom?

Largos wrote: ↑Thu Jan 05, 2023 12:56 am

Did you run

Code: Select all

sudo btrfs subvolume show /home/

Were there any snapshots listed at the bottom?

[z@fedora ~]$ sudo btrfs subvolume show /home/
[sudo] password for z: 
ERROR: Not a Btrfs subvolume: Invalid argument

Cheers.

Try

sudo btrfs subvolume show /

[z@fedora ~]$ sudo btrfs subvolume show /
[sudo] password for z: 
root
	Name: 			root
	UUID: 			[Redacted]
	Parent UUID: 		[Redacted]
	Received UUID: 		-
	Creation time: 		2022-12-23 19:37:10 +0000
	Subvolume ID: 		896
	Generation: 		972263
	Gen at creation: 	930846
	Parent ID: 		5
	Top level ID: 		5
	Flags: 			-
	Send transid: 		0
	Send time: 		2022-12-23 19:37:10 +0000
	Receive transid: 	0
	Receive time: 		-
	Snapshot(s):

Any use?

Cheers.

No, it shows there are no snapshots associated with with your root directory. The behaviour you're getting is consistent with still having a snapshot somewhere though. One last command I can think of for clues is

df

If there is anything else mounted that is a btrfs subvolume then run btrfs subvolume show command on that mountpoint to check if it's still associated with a snapshot.

Ok. So first of all

[z@fedora ~]$ df
Filesystem      1K-blocks       Used  Available Use% Mounted on
devtmpfs             4096          0       4096   0% /dev
tmpfs            16387004      22540   16364464   1% /dev/shm
tmpfs             6554804       2328    6552476   1% /run
/dev/sda3       975097856  901492848   69912688  93% /
tmpfs            16387004         20   16386984   1% /tmp
/dev/sda2          996780     325440     602528  36% /boot
/dev/sda1          613160      17720     595440   3% /boot/efi
/dev/sdb1      2930265088 1785756580 1142061596  61% /run/timeshift/backup
tmpfs             3277400        332    3277068   1% /run/user/1000
/dev/sdc1         3642928    3642928          0 100% /run/media/z/Nobara-36
/dev/sdd1       960302096  682670044  228777628  75% /run/media/z/Photo HDD
/dev/sde1      4883769344   67137696 4815922128   2% /run/media/z/Backup HDD 5TB

Snapshots were never set on any other drive unless it was by complete accident without me knowing. I've now got new Timeshift, Rsync (never did BTRFS snapshots with it) backups running on my internal HDD and you can also see a couple of external HDDs on the end there which are currently only plugged in due to running backups on them with backintime (Nothing to do with BTRFS snapshots either).

Now on the internal HDD which is also BTRFS

sudo btrfs subvolume show /
[sudo] password for z: 
root
	Name: 			root
	UUID: 			[Redacted]
	Parent UUID: 		[Redacted]
	Received UUID: 		-
	Creation time: 		2022-12-23 19:37:10 +0000
	Subvolume ID: 		896
	Generation: 		973939
	Gen at creation: 	930846
	Parent ID: 		5
	Top level ID: 		5
	Flags: 			-
	Send transid: 		0
	Send time: 		2022-12-23 19:37:10 +0000
	Receive transid: 	0
	Receive time: 		-
	Snapshot(s):

By the way, I forced Trim to run earlier which trimmed about 71GB. I then rebooted and am now running another full balance without filters as before. So far, my free space has not changed; It's still around 75GB.

This is so weird. If there's no snapshots going on then it's really hard to make sense of! I've looked at so many other posts online and it always turns out to be snapshots or that a balance was needed..

I left a search of the entire Nobara/Fedora partition running overnight for anything named 'snapshot'. I've checked through the results and there's nothing of significance that I can see, nothing taking up much space at all. Unless there's some secret, invisible snapshots somewhere..

I also rebooted after that full balance but the free space has not changed.

I really do not know what's happening here. If anyone has any ideas then please let me know because I'm starting to think I might just reinstall the lot and use EXT4 instead but that'd be days of work I'd rather avoid if possible...

Cheers!

Ok I think I've found something..

I was going to try making some snapshots with Btrfs Assistant again just to see what happened and so when going to make a new configuration I noticed one of the default backup directories you could select from was '/run.BtrfsAssistant'. This directory was not ever selected before at any point, but I thought it was worth a look in there just incase.. Well, I noticed in there it had a folder with the name of my SSD then inside there, a folder simply called 'root' & 3 folders with the name 'root_backup_' then a bunch of numbers on the ends of the names which looks to represent time & date. All 3 of them look to be from the same day in December. I'm not sure why these folders even exist. There was a separate snapshots folder which was in use when I was using snapshots...

But anyway, what I'm thinking of doing is deleting them all with 'sudo rm -r', or maybe just delete the ones that have 'backup' in the name. When I look in any of these folders it appears to be my root directory. If these are remnants of old snapshots then although deleting them may free up some space, I'm not sure how it would solve the issue I've got where deleting existing files doesn't free up any space but it's somewhere to start and see what happens. I'm unsure of how to proceed and don't want to bork my system by accidentally deleting root files though so I'll try and hold off until you guys can give me some advice! Any ideas then?

Cheers.

Edit: I've just checked my Timeshift, Rsync backup of root which is backed up onto a separate btrfs drive and it does not include this 'BtrfsAssistant' directory. For some reason it just ignores it. I'm hoping that suggests it's ok to just delete all of those folders I just mentioned above..?

@Death

If these are remnants of old snapshots then although deleting them may free up some space, I'm not sure how it would solve the issue I've got where deleting existing files doesn't free up any space but it's somewhere to start and see what happens.

First, a disclaimer: I don't know anything about BTRFS nor about how the snapshots might be managed.

But, the behavior you are seeing (removing a huge file doesn't seem to free up space) would fit a mechanism in which the snapshots contain hard links to the same files: "removing" the file in its primary location would then only update that directory, but the inodes making up the file would not be freed, because they would still be linked in one or more snapshots. I do know that rsnapshot uses a strategy like this for backups, because it means that multiple backups of your main directory don't take N * X space: the hard links are also shared between the backup directories, up to the point that a file changes.

Well, I have a funny story...

So it turns out that deleting those 'root_backup' folders freed up the space I was missing. My free space went from 75GB to about 430GB. After doing that, I decided to run a backup and then delete the other folder in the same directory called 'root' which did not have 'backup' in its name. As I watched my desktop environment fall apart in front of me, I realised what I'd just done.. Yes, I'd deleted my root directory. For the first time in all my years of Linux use, I'd actually done this

But luckily, I had my Timeshift backups, right? Well, I had trouble getting Timeshift to mount the drive so that turned into a research session, but, I got it sorted in the end. After restoring from my backup I was then greeted with the Grub command line. This turned into more research to figure out how to get the system to boot and I think I almost did it but not quite.

So then I decided to call it a night and just do a fresh install of the system today and here I am!

So the question is now, do I bother continuing to use these Btrfs snapshots or do I just stick with Timeshift's Rsync ones

Depends on how bad things would be if you lost data I suppose, but probably one or the other would suffice. I've never used BTRFS so I don't know how it's backup snapshots work. I like Timeshift, but I've never had to restore a complete system either. Regardless, make sure the location that you stash your backups to is excluded from what gets backed up other wise HD space quickly spirals out of control.

Timeshift always wants to back up root only and not home. Personally I don't really care if root gets lost. I can reinstall a system pretty quickly. The important data, the stuff I would really miss is in /home.

bluzee wrote: ↑Fri Jan 06, 2023 11:39 pm

Depends on how bad things would be if you lost data I suppose, but probably one or the other would suffice. I've never used BTRFS so I don't know how it's backup snapshots work. I like Timeshift, but I've never had to restore a complete system either. Regardless, make sure the location that you stash your backups to is excluded from what gets backed up other wise HD space quickly spirals out of control.

Timeshift always wants to back up root only and not home. Personally I don't really care if root gets lost. I can reinstall a system pretty quickly. The important data, the stuff I would really miss is in /home.

Btrfs snapshots are less robust as far as I understand. What I really like about them though is how instant they are and they can be very small in size as they don't make copies of files that already exist on the system, not even for the sake of the snapshot, only when they change or are deleted. That's how i think it works anyway.. I find them convenient for quickly rolling back changes. In short, the speed & relatively small size of the backups is a pro, for sure. However, I'm not a fan of the crap I just went through haha.. But then again, that seems like an issue with the software I used rather than the filesystem itself.

Timeshift will ignore the home folder by default and I like that as I have separate backup software for the home folder. I treat system backups as a different thing to my data backups. I'm sure you can set Timeshift to backup your home folder if you like though..