How to create holes in files ?

[Karli Sjöberg]

On fre, 2017-09-29 at 08:26 +0200, Ben RUBSON wrote:
> > 
> > On 29 Sep 2017, at 07:54, Karli Sjöberg <karli at inparadise.se>
> > wrote:
> > 
> > On tor, 2017-09-28 at 22:16 +0200, Ben RUBSON wrote:
> > > 
> > > > 
> > > > 
> > > > On 28 Sep 2017, at 20:48, Karli Sjöberg <karli at inparadise.se>
> > > > wrote:
> > > > 
> > > > Den 28 sep. 2017 6:47 em skrev Ben RUBSON <ben.rubson at gmail.com
> > > > >:
> > > > > 
> > > > > 
> > > > > On 28 Sep 2017, at 18:34, Bob Eager wrote:
> > > > > 
> > > > > On Thu, 28 Sep 2017 17:26:09 +0200
> > > > > Fabian Keil wrote:
> > > > > 
> > > > > > 
> > > > > > 
> > > > > > Ben RUBSON wrote:
> > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > I'm trying to make holes in files in C.
> > > > > > > Goal is to deallocate huge files on ZFS while (randomly)
> > > > > > > reading
> > > > > > > them. 
> > > > > > My interpretation of the above is that you want to create
> > > > > > holes
> > > > > > without changing the file size and without affecting data
> > > > > > that
> > > > > > is located before or after the holes that you want to
> > > > > > create.
> > > > > > 
> > > > > > Otherwise you could simply "deallocate" the content with
> > > > > > truncate(1).
> > > > > If he doesn't mind copying the files, dd(1) will do the job.
> > > > > However, I
> > > > > expect that doesn't meet his criteria.
> > > > Thank you Bob for your suggestion.
> > > > You're right goal is to avoid copying data : free space would
> > > > not
> > > > necessarily
> > > > allow this, and as the files I'm working on are some hundreds
> > > > of GB
> > > > in size,
> > > > it would really be a counterproductive long stressing storage
> > > > operation.
> > > > 
> > > > Well, correct me if I'm wrong, but wouldn't dd with "seek"
> > > > mitigate
> > > > the issue of writing out all of the data from beginning to end.
> > > > If
> > > > you seek from the beginning of the file to the point you want
> > > > to
> > > > start writing from and use bs to specify how large of a "hole"
> > > > you
> > > > want, the operation wouldn't take long at all. You would, in my
> > > > opinion, achieve exactly what you want, to "create holes in
> > > > files".
> > > > Am I wrong?
> > > I would have liked to do this in C.
> > Yepp, I understand that, I was just bringing up the idea. But hey,
> > if
> > dd can do it, just look in the source of how it does it?
> Of course yes, you're right :)
> 
> > 
> > > 
> > > And the holes I need to create are not necessarily at the
> > > beginning
> > > of a file.
> > No, you _do not_ have to create holes in the beginning of a file,
> > you
> > can do it anywhere in the file you like. E.g. it´s the trick I use
> > to
> > "clean" hardrives to make them look empty, by only erasing the
> > first
> > and last 10 MB of the drives, a whole JBOD only takes seconds to
> > clean.
> > Say you have a 10 MB large file and you want to make a 1 MB large
> > hole
> > somewhere in the middle of it, you do it like this:
> > 
> > # dd if=/dev/zero of=/foo/bar.bin bs=1M seek=7
> > 
> > It´ll make the file look like this inside (hope the ASCII gods are
> > with
> > me):
> >  ____________
> > > 
> > > _______|_|__|
> Unfortunately here storage blocks will be set/written to 0,
> they will not be freed/recovered, a file hole will not be created.
> Below are some commands which demonstrate this.
> The last 3/4 commands show a _newly_ created file with a hole.
> 
> # zfs userspace home
> POSIX User  root  36.5K   none
> 
> # dd if=/dev/random of=f bs=1M count=4
> 4+0 records out
> 
> # zfs userspace home
> POSIX User  root  4.04M   none
> 
> # dd if=/dev/zero of=f bs=1M count=2 seek=2
> 2+0 records out
> 
> # zfs userspace home
> POSIX User  root  4.04M   none
> 
> # rm f
> 
> # dd if=/dev/zero of=f bs=1M count=2 seek=2
> 2+0 records out
> 
> # zfs userspace home
> POSIX User  root  2.04M   none
> 
> # ls -lh f
> -rw-------  1 root  wheel   4.0M 29 Sep 08:19 f

Ah, I see, nice demonstration! But wouldn´t TRIM take of that though
(if that´s available)?

/K

> 
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"