A cookie for anyone who can solve this C/macro problem...

[Lawrence Stewart]

On 04/15/10 23:17, Lawrence Stewart wrote:
> Hi All,
>
> I've hit a road block that I'm hoping people with better C/macro fu can
> help me with. It also may be the case that what I'm trying to do can't
> be done, so I welcome alternative suggestions as well.
>
> The problem:
>
> I'm working on a KPI to modularise congestion control support in the
> kernel (overview of KPI is visible in [1]). So far, I've been focused
> solely on TCP, but I want to have the framework generalise to be able to
> share congestion control algorithms between other congestion aware
> transports e.g. SCTP, DCCP.
>
> To achieve this, I need to pass congestion control information into the
> hook functions in a protocol agnostic way. The current KPI passes the
> tcpcb ptr and this works nicely, but obviously doesn't help.
>
> The cleanest option as I see it is to embed a new CC specific struct in
> the appropriate TCP and SCTP structs, and pass this in to the KPI. I
> started down this road and quickly stopped - the amount of code churn
> this will cause in the TCP and SCTP stacks is to my mind far to
> intrusive, at least for an initial import of the framework.
>
> So the next idea I've been trying to prototype was to pass a union of
> the various protocol struct types into the KPI and then use macro magic
> to allow the CC algorithms to access commonly named variables across the
> different transport struct types. It would require vars to have the same
> name, which would still mean some code churn, but less than the first
> option. Here are the basic bits I'm currently working with:
>
> struct cc_var {
> uint16_t type;
> union {
> struct tcpcb *tcp;
> struct sctp_nets *sctp;
> } ccvc;
> };
>
> All function ptrs in struct cc_algo [1] are modified to accept a "struct
> cc_var *ccv".
>
> In an algorithm implementation e.g. NewReno [2], I then want to be able
> to do things like this with the fictitious CCVC() macro:
>
> void
> newreno_ack_received(struct cc_var *ccv)
> {
> u_int cw = CCVC(ccv)->snd_cwnd;
> ...
> CCVC(ccv)->snd_cwnd = blah;
> }
>
>
> So far I haven't found a way to achieve the above, and the best I've
> come up (with help) is this:
>
> #define CCV_DO(ccv, what) \
> ( \
> (ccv)->type == IPPROTO_TCP ? (ccv)->ccvc.tcp->what : \
> (ccv)->ccvc.sctp->what \
> )
>
> which can be used like this:
>
> void
> newreno_ack_received(struct cc_var *ccv)
> {
> u_int cw = CCV_DO(ccv, snd_cwnd);
> ...
> CCVC(ccv, snd_cwnd = blah);
> }
>
> Of course, this falls apart if you try do this for example:
>
> CCVC(ccv, snd_cwnd = min(blah, bleh));
>
>
> So... I'm sure there are some good ideas out there and would really
> appreciate hearing about them.

Thanks to those who replied on and off list. The cookie reward goes to 
Hans and Daniel. The key insight I was missing is that the C ternary 
statement doesn't return something suitable for use as an lvalue. 
Dereferencing a ptr returned by the ternary does though, and this 
provides me with a fairly neat way of dealing with the problem.

For posterity's sake, relevant snippets of working code that I'm now 
using are as follows:

struct cc_var {
         union ccv_container {
                 struct tcpcb *tcp;
                 struct sctp_nets *sctp;
         } ccvc;
         int type;
};

#define CCV(ccv, what)                                                 \
(*(                                                                    \
         (ccv)->type == IPPROTO_TCP ?    &(ccv)->ccvc.tcp->what :       \
                                         &(ccv)->ccvc.sctp->what        \
))

In future, the CCV macro could also grow support for DCCP using a nested 
ternary e.g. (untested)

#define CCV(ccv, what)                                                 \
(*(                                                                    \
         (ccv)->type == IPPROTO_TCP   ?  &(ccv)->ccvc.tcp->what :       \
	((ccv)->type == IPPROTO_SCTP ?	&(ccv)->ccvc.sctp->what :      \
				        &(ccv)->ccvc.dccp->what)       \
))


And finally, an example use of the current code in the NewReno CC module:

void
newreno_ack_received(struct cc_var *ccv)
{
         u_int cw = CCV(ccv, snd_cwnd);
         u_int incr = CCV(ccv, t_maxseg);
         if (cw > CCV(ccv, snd_ssthresh))
                 incr = max((incr * incr / cw), 1);

         CCV(ccv, snd_cwnd) = min(cw+incr, TCP_MAXWIN <<
	    CCV(ccv, snd_scale));
}

Whilst we're here and on an educational roll, does anyone have thoughts 
on what the compiler actually does with "*(blah ? &1 : &2)"? As this TCP 
code is called in the fast path, I'm curious to know if there is a 
runtime penalty for taking the ref and then immediately derefing it like 
this, or if the only overhead is the test+branch and the compiler is 
smart enough to collapse *(&1 or &2) to (1 or 2). I don't have a good 
mental model of how this stuff works under the hood...

Thanks again.

Cheers,
Lawrence