svn commit: r320156 - in head: cddl/contrib/opensolaris/cmd/zdb cddl/contrib/opensolaris/cmd/ztest cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contri...

[Justin Hibbits]

Hi Andriy,

On Tuesday, June 20, 2017, Andriy Gapon <avg at freebsd.org> wrote:

> Author: avg
> Date: Tue Jun 20 17:39:24 2017
> New Revision: 320156
> URL: https://svnweb.freebsd.org/changeset/base/320156
>
> Log:
>   MFV r318946: 8021 ARC buf data scatter-ization
>
>   illumos/illumos-gate at 770499e185d15678ccb0be57ebc626ad18d93383
>   https://github.com/illumos/illumos-gate/commit/
> 770499e185d15678ccb0be57ebc626ad18d93383
>
>   https://www.illumos.org/issues/8021
>     The ARC buf data project (known simply as "ABD" since its genesis in
> the ZoL
>     community) changes the way the ARC allocates `b_pdata` memory from
> using linear
>     `void *` buffers to using scatter/gather lists of fixed-size 1KB
> chunks. This
>     improves ZFS's performance by helping to defragment the address space
> occupied
>     by the ARC, in particular for cases where compressed ARC is enabled.
> It could
>     also ease future work to allocate pages directly from `segkpm` for
> minimal-
>     overhead memory allocations, bypassing the `kmem` subsystem.
>     This is essentially the same change as the one which recently landed
> in ZFS on
>     Linux, although they made some platform-specific changes while
> adapting this
>     work to their codebase:
>     1. Implemented the equivalent of the `segkpm` suggestion for future
> work
>     mentioned above to bypass issues that they've had with the Linux
> kernel memory
>     allocator.
>     2. Changed the internal representation of the ABD's scatter/gather
> list so it
>     could be used to pass I/O directly into Linux block device drivers.
> (This
>     feature is not available in the illumos block device interface yet.)
>
>   FreeBSD notes:
>   - the actual (default) chunk size is 4KB (despite the text above saying
> 1KB)
>   - we can try to reimplement ABDs, so that they are not permanently
>     mapped into the KVA unless explicitly requested, especially on
>     platforms with scarce KVA
>   - we can try to use unmapped I/O and avoid intermediate allocation of a
>     linear, virtual memory mapped buffer
>   - we can try to avoid extra data copying by referring to chunks / pages
>     in the original ABD
>
>   Reviewed by: Matthew Ahrens <mahrens at delphix.com>
>   Reviewed by: George Wilson <george.wilson at delphix.com>
>   Reviewed by: Paul Dagnelie <pcd at delphix.com>
>   Reviewed by: John Kennedy <john.kennedy at delphix.com>
>   Reviewed by: Prakash Surya <prakash.surya at delphix.com>
>   Reviewed by: Prashanth Sreenivasa <pks at delphix.com>
>   Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
>   Reviewed by: Chris Williamson <chris.williamson at delphix.com>
>   Approved by: Richard Lowe <richlowe at richlowe.net>
>   Author: Dan Kimmel <dan.kimmel at delphix.com>
>
>   MFC after:    3 weeks
>
> Added:
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c
>      - copied, changed from r318946, vendor-sys/illumos/dist/uts/
> common/fs/zfs/abd.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/abd.h
>      - copied, changed from r318946, vendor-sys/illumos/dist/uts/
> common/fs/zfs/sys/abd.h
> Modified:
>   head/cddl/contrib/opensolaris/cmd/zdb/zdb.c
>   head/cddl/contrib/opensolaris/cmd/zdb/zdb_il.c
>   head/cddl/contrib/opensolaris/cmd/ztest/ztest.c
>   head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c
>   head/sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c
>   head/sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.h
>   head/sys/cddl/contrib/opensolaris/uts/common/Makefile.files
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/blkptr.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/edonr_zfs.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lz4.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/skein_zfs.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/ddt.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_checksum.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_cache.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_checksum.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c
>   head/sys/conf/files
> Directory Properties:
>   head/cddl/contrib/opensolaris/   (props changed)
>   head/cddl/contrib/opensolaris/cmd/zdb/   (props changed)
>   head/cddl/contrib/opensolaris/lib/libzfs/   (props changed)
>   head/sys/cddl/contrib/opensolaris/   (props changed)
>
>

This commit breaks gcc builds with new warnings introduced:

/home/chmeee/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
In function 'arc_kmem_reap_now':
/home/chmeee/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:4364:
warning: nested extern declaration of 'abd_chunk_cache'
[-Wnested-externs]
/home/chmeee/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c:
In function 'vdev_label_write_pad2':

/home/chmeee/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
In function 'arc_kmem_reap_now':
/home/chmeee/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:4364:
warning: nested extern declaration of 'abd_chunk_cache'
[-Wnested-externs]

/home/chmeee/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
In function 'arc_kmem_reap_now':

/home/chmeee/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:4364:
warning: nested extern declaration of 'abd_chunk_cache' [-Wnested-externs]
/home/chmeee/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c:902:
warning: passing argument 2 of 'abd_copy_from_buf' discards qualifiers from
pointer target type

- Justin