[Bug 202637] www/lighttpd: 1.4.36 has header corruption, and breaks under load

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202637

--- Comment #3 from Peter Wemm <peter at FreeBSD.org> ---
All that's left:

# egrep -v '(^#|^$)' lighttpd.conf conf.d/*.conf
lighttpd.conf:var.log_root    = "/logs"
lighttpd.conf:var.server_root = "/data"
lighttpd.conf:var.state_dir   = "/var/run"
lighttpd.conf:var.conf_dir    = "/etc"
lighttpd.conf:server.chroot   = "/usr/local/www"
lighttpd.conf:var.vhosts_dir  = server_root + "/vhosts"
lighttpd.conf:var.cache_dir   = "/var/cache/lighttpd"
lighttpd.conf:var.socket_dir  = "/sockets"
lighttpd.conf:include "modules.conf"
lighttpd.conf:server.port = 80
lighttpd.conf:server.bind = "149.20.53.26"
lighttpd.conf:$SERVER["socket"] == "[2001:4f8:3:ffe0:406a:0:16:1a]:80" { } 
lighttpd.conf:server.username  = "www"
lighttpd.conf:server.groupname = "www"
lighttpd.conf:server.document-root = "/data/"
lighttpd.conf:server.tag = "lighttpd"
lighttpd.conf:server.pid-file = state_dir + "/lighttpd.pid"
lighttpd.conf:server.errorlog             = log_root + "/error.log"
lighttpd.conf:include "conf.d/access_log.conf"
lighttpd.conf:include "conf.d/debug.conf"
lighttpd.conf:server.event-handler = "freebsd-kqueue"
lighttpd.conf:server.network-backend = "write"
lighttpd.conf:server.max-fds = 40960
lighttpd.conf:server.stat-cache-engine = "simple"
lighttpd.conf:server.max-connections = 20480
lighttpd.conf:server.max-keep-alive-idle = 5
lighttpd.conf:server.max-keep-alive-requests = 1024
lighttpd.conf:server.max-request-size = 512
lighttpd.conf:index-file.names += (
lighttpd.conf:  "index.xhtml", "index.html", "index.htm", "default.htm",
"index.php"
lighttpd.conf:)
lighttpd.conf:url.access-deny             = ( "~", ".inc", ".swp" )
lighttpd.conf:$HTTP["url"] =~ "\.pdf$" {
lighttpd.conf:  server.range-requests = "disable"
lighttpd.conf:}
lighttpd.conf:static-file.exclude-extensions = ( ".php", ".pl", ".fcgi",
".scgi" )
lighttpd.conf:include "conf.d/mime.conf"
lighttpd.conf:include "conf.d/dirlisting.conf"
lighttpd.conf:server.follow-symlink = "disable"
lighttpd.conf:server.upload-dirs = ( "/var/tmp/" )
conf.d/dirlisting.conf:dir-listing.activate      = "enable"
conf.d/dirlisting.conf:dir-listing.hide-dotfiles = "disable" 
conf.d/dirlisting.conf:dir-listing.exclude       = ( "~$" )
conf.d/dirlisting.conf:dir-listing.encoding = "UTF-8"
conf.d/dirlisting.conf:dir-listing.hide-header-file = "disable"
conf.d/dirlisting.conf:dir-listing.show-header = "disable"
conf.d/dirlisting.conf:dir-listing.hide-readme-file = "disable"
conf.d/dirlisting.conf:dir-listing.show-readme = "disable"

The the conf files not shown here were deleted by 'pkg delete' as they were
stock / unmodified.

We saw it on the freebsd-update and portsnap servers.  They typically have a
very very large number of files with long file names.  The file names are
usually sha256 of the file itself.  The easiest way to trigger it was:
# find 10.2-RELEASE/amd64/f -type f >/tmp/flist
(this generates about 66000 files)
somewhere else:
xargs -P8 /usr/libexec/phttpget servername.freebsd.org < /tmp/flist
and watch the sporadic '400 bad request'.  With 1.4.35_05, it runs without the
error 400s.

/usr/libexec/phttpget is src/usr.sbin/portsnap/phttpget/* - it does pipelined
http fetches.

If you have debug.conf log failed requests, you can see it reporting what
appears to be truncated request headers.

-- 
You are receiving this mail because:
You are the assignee for the bug.