website statistics

[Murray Stokely]

On Tue, May 09, 2006 at 01:20:26PM +0200, Pav Lucistnik wrote:
> Simon Nielsen was kind enough to provide me with log files from
> www.FreeBSD.org, nine days between Friday 28 April and Saturday 6 May.
> 
> BASIC STATISTICS
> 
> All data are cleared from search engine crawlers, RSS clients and
> automated downloads of files for ports infrastructure.

Very interesting.  Thanks for doing this analysis.

> My take: Ports are immensely popular, I think they should get their own
> entry on the horizontal grey navbar. On the other hand, Community is
> rarely visited and could be collapsed into Support.

I think that is a good idea well supported by your findings.  We
should make sure to remove the 'Ports' from the 'Shortcuts' list when
it is added to the gray top navbar.

> Couldn't find /smp/ referenced from /projects/ page, should it be there?
> Couldn't find a link to CVSweb interface, where is it referenced?

cvsweb : www.freebsd.org -> developers -> cvs -> web interface
Seems easy enough to find.

The SMP page is out of date, so I'm not particularly worried about it
being referenced more.

> SEARCHBOTS
> 
> On top of the numbers above, search engine crawlers generated another
> 43,778 hits/day. Googlebot alone is responsible for 7875 hits/day.

Google claims to have 1,680,000 documents from www.freebsd.org indexed
(+site:www.freebsd.org query), and 3 times as much if you count
people.freebsd.org and the various ftpN and other domains.  If that is
anywhere near correct, then 7875 hits / day means the average page is
only refreshed every 213 days.  I hope it crawls faster than 7875 hits
/ day on other days.  Does Apache log a page hit if it was not
returned because of an If-Modified-Since header?

The current cached version of www.freebsd.org/index.html on Google is
from May 8, just before our new logo update, while the cached version
on Yahoo is from mid April, and on MSN is from May 2.

   - Murray