FreeBSD on Amazon AWS EC2 long standing performance problems

[Gunther Schadow]

Gordon Bergling wrote:
> Can you verify your feelings by numbers?

Yes, like I said

>> Not by a few % points, but by factors if not an order of magnitude!

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253261

Do this:

dd if=/dev/zero of=/dev/nvd2 bs=100M status=progress

and you see that it's writing with the "whopping" speed of 70 MB/s.

That used to be good, but it is no longer good. Compare Amazon Linux doing
the same thing at 300 MB/s.

Now, when you put a file system over it, zfs or ufs, then instantly the
performance gets better:

newfs /dev/nvd2
mount /dev/nvd2 /mnt
dd if=/dev/zero of=/mnt/test bs=100M status=progress

now that works at about 250 MB/s. Decent. So, problem solved?

No! It turns out if I create a PostgreSQL database over this setup, then
again there is massive delay on the read and write and throughput will drop
to even worse than 70 MB/s. Creating one index takes 10 times as long as
that same on the Linux system.

PS: no need to point out that Linux uses buffer cache for direct write to
device and BSD doesn't. Those effects will not make a difference when you
write (or read) more than the buffer cache size (e.g., a few GBs).