increasing swap activity

[Michael R. Hines]

Ok....Here's the system's design:

What I have created is a 3-part distributed memory system:
1. a client machine setup as a diskless client has a page fault or wants to 
swap out a page. The request goes through NFS to a machine.
2. The machine that receives the request is a linux machine running a 
kernel module. The module does the following:
         - acts as a purely in-memory NFS server
         - parses and interprets the RPC/NFS packet
         - re-schedules the request (and it's transmitted page data) to 
ANOTHER machine which either holds the requested page or stores the swapped 
out page
3. This third machine is also a linux machine running a second kernel 
module that acts as pure memory storage. It answers the 2nd machine's 
request by sending back the data or storing the data and alerts the second 
machine.
4. finally, the second machine creates and NFS/RPC reply to send back to 
the client machine, and it can continue processing.

So you're prolly asking yourself: Why in the world would someone do 
something this crazy?

Well:
- The system only operates within a gigabit-switched cluster of machines.
- The system is supposed to scale: i.e. LOTS of client machines, one single 
second machine (which I call an engine) and LOTS of 3rd machines, called 
backend-servers.
- All of this is multiplexed and demultiplexed by one central engine.
- It creates a pool of memory that is virtualized by the engine. i.e. any 
client page could be stored on any back-end server machine at any time.
- There are no disks whatsoever involved in the whole processes.

Disk-swap page-fault latencies range between: 7 to 11 milliseconds
My system?: 650 MICROseconds =)

So, the system isn't the bottle-neck. It's the FreeBSD machine's swapper, 
which is being used a client.

So, I'd like to drastically boost the activity of the FreeBSD swapper.

Make sense?

At 08:51 AM 2/6/2005, Nagilum wrote:
>Michael Hines wrote:
>
>>
>>I've written a remote-memory system for a thesis of mine. As a result, 
>>I'm able to setup a diskless client that swaps to remote memory instead 
>>of a remote disk. The specifics, reasons, and design of the system are a 
>>long story....
>>
>>However - this system allows for much faster page-fault latencies (on the 
>>order of 10-20 times faster than using a disk for swap space). The 
>>problem is: I cannot get the swapper to page-out or page-in data any faster.
>>
>>My end question is: how would one DRASTICALLY increase the rate at which 
>>the system does its paging in freebsd?
>>
>>Everthing I find on the net says "don't mess with freebsd's VM system or 
>>you'll die and go to hell."
>>
>>However, I do in fact need to drastically increase the paging bandwidth.
>>
>>Anybody know how? Preferably during runtime?
>>
>>Thanks a lot.
>>
>>/*********************************/
>>Michael R. Hines
>>Grad Student, Florida State
>>Dept. Computer Science
>>http://www.cs.fsu.edu/~mhines/
>>Jusqu'a ce que le futur vienne...
>>/*********************************/
>
>
>I think the first thing to do is determine the bottleneck you're fighting.
>Maybe using several machines as swap-slaves would help? (the swapspace 
>will be used interleaved)
>Also it would help to explain a bit more what you have done so far. Have 
>you written your own swapfs?
>Kind regards,
>Alex.
>
>_______________________________________________
>freebsd-questions at freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"

/*********************************/
Michael R. Hines
Grad Student, Florida State
Dept. Computer Science
http://www.cs.fsu.edu/~mhines/
Jusqu'à ce que le futur vienne...
/*********************************/