Author: bjk
Date: Mon Jan 18 20:13:00 2016
New Revision: 48055
URL: https://svnweb.freebsd.org/changeset/doc/48055
Log:
Add sendfile entry from glebius
Modified:
head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml
Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml
==============================================================================
--- head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml Mon Jan 18 18:42:46 2016 (r48054)
+++ head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml Mon Jan 18 20:13:00 2016 (r48055)
@@ -3631,4 +3631,114 @@
</task>
</help>
</project>
+
+ <project cat='kern'>
+ <title>Sendfile(2) Improvements</title>
+
+ <contact>
+ <person>
+ <name>
+ <given>Gleb</given>
+ <common>Smirnoff</common>
+ </name>
+ <email>glebius at FreeBSD.org</email>
+ </person>
+ </contact>
+
+ <links>
+ <url href="https://svnweb.FreeBSD.org/base?view=revision&revision=293439">Commit to Head</url>
+ <url href="http://www.slideshare.net/facepalmtarbz2/new-sendfile-in-english">Slides</url>
+ <url href="https://events.yandex.ru/lib/talks/2682/">Presentation (in Russian)</url>
+ </links>
+
+ <body>
+ <p>The <tt>sendfile(2)</tt> system call was introduced in
+ 1998 as an alternative to a traditional
+ <tt>read(2)</tt>/<tt>write(2)</tt> loop, speeding up server
+ performance by a factor of ten at the time. Since it was adopted
+ by all major operating systems, it is now used by any serious web
+ server software. Wherever there is high traffic, there is
+ <tt>sendfile(2)</tt> under the hood.</p>
+
+ <p>Now, with &os; 11, we are making the next revolutinary step
+ in serving traffic. <tt>sendfile(2)</tt> no longer blocks waiting
+ on disk I/O. Instead, it immediately returns control to the
+ application, performing the necessary I/O in the background. The
+ original <tt>sendfile(2)</tt> waited for the disk read operation
+ to complete and then put the data that was read into the socket,
+ then returned to userspace. If a web server serves thousands of
+ clients, with thousands of requests, in order to avoid stalls it
+ needed to spawn extra contexts from which to run
+ <tt>sendfile(2)</tt>. Alternatively, it could use special tricks
+ like the <tt>SF_NODISKIO</tt> flag that forces
+ <tt>sendfile(2)</tt> to serve only content that is cached in
+ memory. Now, these tricks are in the past, and a web server can
+ simply use <tt>sendfile(2)</tt> as it would use <tt>write(2)</tt>,
+ withouth any extra care. The new sendfile cuts out the overhead
+ of extra contexts, short writes, and extra syscalls to prepopulate
+ the cache, bringing performance to new level.</p>
+
+ <p>The new syscall is built on top of two new features
+ introduced in the kernel. The first one is an asynchronous VM
+ pager interface, and the corresponding
+ <tt>VOP_GETPAGES_ASYNC()</tt> file system method for UFS. The
+ second one is the concept of "not ready" data in
+ sockets. When <tt>sendfile(2)</tt> is called, first
+ VOP_GETPAGES_ASYNC() is called, which dispatches I/O requests for
+ completion. Then, buffers with pages to be populated are put into
+ the socket buffer, but flagged as not-yet-ready. Then control
+ immediately returns to the application. When the I/O is finished,
+ the buffers are marked as ready, and the socket is activated to
+ continue transmission.</p>
+
+ <p>Additional features of the new sendfile are new flags that
+ provide an application with extra control over the transmitted
+ content. Now it is possible to prevent caching of content in
+ memory, which is useful when it is known that the content is
+ unlikely to be reused any time soon. In such cases, it is better
+ to let the associated storage be freed, rather than puting the
+ data in cache. It is also possible to specify a readahead with
+ every syscall, if the application can predict client behavior.</p>
+
+ <p>The new <tt>sendfile(2)</tt> is a drop-in replacement, API
+ and ABI compatible with old one. Applications do not even need to
+ recompile in order to benefit from the new implementation.</p>
+
+ <p>This work is a joint effort between two companies: NGINX,
+ Inc., and Netflix. There were many people involved in the
+ project. At its initial stage, when no code was yet written, the
+ idea of such an asynchronous drop-in replacement was discussed
+ amongst &a.glebius;, &a.scottl;, &a.kib;, &a.adrian;, and Igor
+ Sysoev. The initial prototype was coded by Gleb under the
+ supervision of Kostik on the VM parts of patch, and under constant
+ pressure from Igor, who demanded that nginx be capable of running
+ with the new <tt>sendfile(2)</tt> with no modifications. The
+ prototype demonstrated good performance and stability and quickly
+ went into Netflix production in late 2014. During 2015, the code
+ matured and continued serving production traffic at Netflix.
+ &a.scottl;, &a.rrs;, &a.emax;, and &a.gallatin; added their
+ contributions to the code.</p>
+
+ <p>Now we are releasing the code behind our success to the
+ &os; community, making it available to all &os; users
+ worldwide!</p>
+ </body>
+
+ <sponsor>
+ Netflix
+ </sponsor>
+
+ <sponsor>
+ NGINX, Inc.
+ </sponsor>
+
+ <help>
+ <task>
+ <p><tt>SSL_sendfile()</tt> — an extension to the new
+ <tt>sendfile(2)</tt> that allows uploading session keys to the
+ kernel, and then using <tt>sendfile(2)</tt> on an SSL-enabled
+ socket.</p>
+ </task>
+ </help>
+ </project>
</report>