Multiqueue support for bpf

[Takuya ASADA]

Hi all,

I'd like to propose multiqueue support for bpf.
It's result of GSoC'11, and proposed on freebsd-net at Aug.2011 but not yet
merged:
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/syuu1228/1
http://lists.freebsd.org/pipermail/freebsd-net/2011-August/029585.html

The objectives of the patch is to support multiqueue NICs on BPF, and
provide interfaces for multithreaded packet processing using BPF.
To get optimal performance and to reduce lock contention, multiqueue BPF
provides a feature to specify hardware queues.
Following is basic usage of multiqueue BPF:

void *bpf_thread(void *arg) {
    cpu = (int)arg;
    CPU_ZERO(&cpuset);
    CPU_SET(cpu, &cpuset);
    cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_TID, -1, sizeof(cpuset),
&cpuset);
    fd = open("/dev/bpf0", O_RDWR);
    ioctl(fd, BIOCSTRXQMASK, &cpu);
    ioctl(fd, BIOCSTTXQMASK, &cpu);
    /* actual works */
}

int main(void)
{
    for (i = 0; i < maxcpus; i++)
        pthread_create(&threads[i], NULL, bpf_thread, (void *)i);
    for (i = 0; i < maxcpus; i++)
        pthread_join(&threads[i], NULL);
}

In this example, threads[n] bind to CPUn, receives packets from rxqueue n
and txqueue n.

To implement it, the patch modifies bpf_*tap*() and extends struct ifnet,
struct mbuf to notify hardware queue information.

The changes are in this branch:
http://svnweb.freebsd.org/base/user/syuu/mq_bpf/

API descriptions and benchmark results are on previous post:
http://lists.freebsd.org/pipermail/freebsd-net/2011-August/029585.html

Benchmark program is on this repository:
https://github.com/syuu1228/mq_bpf_test