svn commit: r48055 - head/en_US.ISO8859-1/htdocs/news/status

Mon Jan 18 20:13:02 UTC 2016

Author: bjk
Date: Mon Jan 18 20:13:00 2016
New Revision: 48055
URL: https://svnweb.freebsd.org/changeset/doc/48055

Log:
  Add sendfile entry from glebius

Modified:
  head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml

Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml
==============================================================================

--- head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml	Mon Jan 18 18:42:46 2016	(r48054)
+++ head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml	Mon Jan 18 20:13:00 2016	(r48055)
@@ -3631,4 +3631,114 @@
       </task>
     </help>
   </project>
+
+  <project cat='kern'>
+    <title>Sendfile(2) Improvements</title>
+
+    <contact>
+      <person>
+	<name>
+	  <given>Gleb</given>
+	  <common>Smirnoff</common>
+	</name>
+	<email>glebius at FreeBSD.org</email>
+      </person>
+    </contact>
+
+    <links>
+      <url href="https://svnweb.FreeBSD.org/base?view=revision&revision=293439">Commit to Head</url>
+      <url href="http://www.slideshare.net/facepalmtarbz2/new-sendfile-in-english">Slides</url>
+      <url href="https://events.yandex.ru/lib/talks/2682/">Presentation (in Russian)</url>
+    </links>
+
+    <body>
+      <p>The <tt>sendfile(2)</tt> system call was introduced in
+	1998 as an alternative to a traditional
+	<tt>read(2)</tt>/<tt>write(2)</tt> loop, speeding up server
+	performance by a factor of ten at the time.  Since it was adopted
+	by all major operating systems, it is now used by any serious web
+	server software.  Wherever there is high traffic, there is
+	<tt>sendfile(2)</tt> under the hood.</p>
+
+      <p>Now, with &os; 11, we are making the next revolutinary step
+	in serving traffic.  <tt>sendfile(2)</tt> no longer blocks waiting
+	on disk I/O.  Instead, it immediately returns control to the
+	application, performing the necessary I/O in the background.  The
+	original <tt>sendfile(2)</tt> waited for the disk read operation
+	to complete and then put the data that was read into the socket,
+	then returned to userspace.  If a web server serves thousands of
+	clients, with thousands of requests, in order to avoid stalls it
+	needed to spawn extra contexts from which to run
+	<tt>sendfile(2)</tt>.  Alternatively, it could use special tricks
+	like the <tt>SF_NODISKIO</tt> flag that forces
+	<tt>sendfile(2)</tt> to serve only content that is cached in
+	memory.  Now, these tricks are in the past, and a web server can
+	simply use <tt>sendfile(2)</tt> as it would use <tt>write(2)</tt>,
+	withouth any extra care.  The new sendfile cuts out the overhead
+	of extra contexts, short writes, and extra syscalls to prepopulate
+	the cache, bringing performance to new level.</p>
+
+      <p>The new syscall is built on top of two new features
+	introduced in the kernel.  The first one is an asynchronous VM
+	pager interface, and the corresponding
+	<tt>VOP_GETPAGES_ASYNC()</tt> file system method for UFS.  The
+	second one is the concept of "not ready" data in
+	sockets.  When <tt>sendfile(2)</tt> is called, first
+	VOP_GETPAGES_ASYNC() is called, which dispatches I/O requests for
+	completion.  Then, buffers with pages to be populated are put into
+	the socket buffer, but flagged as not-yet-ready.  Then control
+	immediately returns to the application.  When the I/O is finished,
+	the buffers are marked as ready, and the socket is activated to
+	continue transmission.</p>
+
+      <p>Additional features of the new sendfile are new flags that
+	provide an application with extra control over the transmitted
+	content.  Now it is possible to prevent caching of content in
+	memory, which is useful when it is known that the content is
+	unlikely to be reused any time soon. In such cases, it is better
+	to let the associated storage be freed, rather than puting the
+	data in cache.  It is also possible to specify a readahead with
+	every syscall, if the application can predict client behavior.</p>
+
+      <p>The new <tt>sendfile(2)</tt> is a drop-in replacement, API
+	and ABI compatible with old one.  Applications do not even need to
+	recompile in order to benefit from the new implementation.</p>
+
+      <p>This work is a joint effort between two companies: NGINX,
+	Inc., and Netflix.  There were many people involved in the
+	project.  At its initial stage, when no code was yet written, the
+	idea of such an asynchronous drop-in replacement was discussed
+	amongst &a.glebius;, &a.scottl;, &a.kib;, &a.adrian;, and Igor
+	Sysoev.  The initial prototype was coded by Gleb under the
+	supervision of Kostik on the VM parts of patch, and under constant
+	pressure from Igor, who demanded that nginx be capable of running
+	with the new <tt>sendfile(2)</tt> with no modifications.  The
+	prototype demonstrated good performance and stability and quickly
+	went into Netflix production in late 2014.  During 2015, the code
+	matured and continued serving production traffic at Netflix.
+	&a.scottl;, &a.rrs;, &a.emax;, and &a.gallatin; added their
+	contributions to the code.</p>
+
+      <p>Now we are releasing the code behind our success to the
+	&os; community, making it available to all &os; users
+	worldwide!</p>
+    </body>
+
+    <sponsor>
+      Netflix
+    </sponsor>
+
+    <sponsor>
+      NGINX, Inc.
+    </sponsor>
+
+    <help>
+      <task>
+	<p><tt>SSL_sendfile()</tt> — an extension to the new
+	  <tt>sendfile(2)</tt> that allows uploading session keys to the
+	  kernel, and then using <tt>sendfile(2)</tt> on an SSL-enabled
+	  socket.</p>
+      </task>
+    </help>
+  </project>
 </report>
