svn commit: r264027 - in head: release share/man/man7

[Nikolai Lifanov]

On 04/02/14 12:33, Glen Barber wrote:
> On Wed, Apr 02, 2014 at 12:23:46PM -0400, Nikolai Lifanov wrote:
>> On 04/02/14 12:06, Glen Barber wrote:
>>> On Wed, Apr 02, 2014 at 11:55:33AM -0400, Nikolai Lifanov wrote:
>>>> On 04/02/14 11:51, Glen Barber wrote:
>>>>> On Wed, Apr 02, 2014 at 10:40:22AM -0500, Brooks Davis wrote:
>>>>>> On Tue, Apr 01, 2014 at 10:41:27PM +0000, Glen Barber wrote:
>>>>>>> Author: gjb
>>>>>>> Date: Tue Apr  1 22:41:26 2014
>>>>>>> New Revision: 264027
>>>>>>> URL: http://svnweb.freebsd.org/changeset/base/264027
>>>>>>>
>>>>>>> Log:
>>>>>>>   Add a new release build variable, WITH_COMPRESSED_IMAGES.
>>>>>>>   
>>>>>>>   When set to a non-empty value, the installation medium is
>>>>>>>   compressed with gzip(1) as part of the 'install' target in
>>>>>>>   the release/ directory.
>>>>>>>   
>>>>>>>   With gzip(1) compression, downloadable image are reduced in
>>>>>>>   size quite significantly.  Build test against head at 263927
>>>>>>>   shows the following:
>>>>>>>   
>>>>>>>    bootonly.iso:		64% smaller
>>>>>>>    disc1.iso:		44% smaller
>>>>>>>    memstick.img:		47% smaller
>>>>>>>    mini-memstick.img:	65% smaller
>>>>>>>    dvd1.iso:		untested
>>>>>>>   
>>>>>>>   This option is off by default, I would eventually like to
>>>>>>>   turn it on by default, and remove the '-k' flag to gzip(1)
>>>>>>>   so only compressed images are published on FTP.
>>>>>>
>>>>>> I'd recommend testing xz compression as well.  With UFS images of a full
>>>>>> world the savings vs gzip are significant (more than 30% IIRC, but it's
>>>>>> need more than a year since I checked so I'm a bit unsure of the exact
>>>>>> numbers).
>>>>>>
>>>>>
>>>>> delphij also brought this up.
>>>>>
>>>>> I have concerns with xz(1), since there was mention in IRC that Windows
>>>>> users may have problems decompressing xz-compressed images.  So, gzip(1)
>>>>> is used because it seems to be the more commonly-supported archive
>>>>> mechanisms.
>>>>>
>>>>> The benefit of xz(1) over gzip(1) was only 50M-ish.
>>>>>
>>>>>   -rw-r--r--  1 root  wheel   601M Mar 28 20:18 disc1.iso
>>>>>   -rw-r--r--  1 root  wheel   381M Mar 28 20:18 disc1.iso.bz2
>>>>>   -rw-r--r--  1 root  wheel   392M Mar 28 20:18 disc1.iso.gz
>>>>>   -rw-r--r--  1 root  wheel   348M Mar 28 20:18 disc1.iso.xz
>>>>>
>>>>> Glen
>>>>>
>>>>
>>>> How about 7zip (Windows program, not file format)? What would a Windows
>>>> user use that can decompress gzip and not xz? It was a problem around
>>>> ~2007, but xz support is no longer rare or exotic.
>>>>
>>>
>>> I don't know, to be honest.  I have no Windows machines to test, so
>>> I can only go by what I am told.
>>>
>>> Glen
>>>
>>
>> I just verified it with 7zip for Windows version 9.22. It extracts
>> .tar.xz archives and decompresses .xz images.
>>
> 
> Great, thank you for confirming.
> 
> So, the question I have now is, considering the time it takes to
> compress the image with xz(1) compared to gzip(1), is that worth saving
> the 50MB space?
> 
> I do not object to using xz(1), but the compression time difference was
> quite significant.  (I do not have numbers off-hand, but can run the
> compression again.)
> 
> Glen
> 

Compression is much more expensive. There are different levels too, so
xz -9e my16Gfile.img on a 4 core i7 can take up to 8 hours. Other levels
are cheaper. De-compression (for the end user) is as cheap or cheaper
than gzip. Depending on the chosen compression level, the minimum amount
of RAM required for decompression can also grow. The xz(1) explains
these properties.

- Nikolai Lifanov