Magic symlinks redux

[Luigi Rizzo]

On Fri, Aug 22, 2008 at 02:13:12PM -0600, M. Warner Losh wrote:
> In message: <20080822193657.GB63527 at onelab2.iet.unipi.it>
>             Luigi Rizzo <rizzo at iet.unipi.it> writes:
> : On Fri, Aug 22, 2008 at 12:40:19PM -0600, M. Warner Losh wrote:
> : > In message: <20080822162259.GA61694 at onelab2.iet.unipi.it>
> : >             Luigi Rizzo <rizzo at iet.unipi.it> writes:
> : > : Also, we do have a need to push tables of info in the kernel
> : > : (e.g. list of PCI/USB ids, quirks tables and the like),
> : > : maybe we can take this chance to make the varsym subsystem useful
> : > : also within device drivers ?
> : > 
> : > No.  what problem would this solve?
> : 
> : take e.g. uscanner (or several other devices, e.g. if_rl)
> : where the only way to tell whether a device
> : is supported or not is looking up a table of usb vendor/id
> : (the same happens for many pci devices). in the simple cases
> : you just need the id - a more complex one would use 
> 
> How is this related?  I guess was my question.

background:

    the initial topic was variant symlinks, for which is desirable to
    have system- or user-definable variables that can be used during
    the translation -- so one could play tricks such as setting
	    /tmp -> /tmpdir/${uid}
    and have private temp directories, and the like.
    To implement those variables you need a storage subsystem that is accessible
    from kernel, outside the process space, and supports multiple
    instances (per-user or per-process or perhaps both; the 'multiple
    instance' thing can be easily implemented with something with a
    hierarchical structure). It probably needs to be support very
    fast reads as it is accessed every time we need to translate a
    pathname involving variant symlinks. It doesn't need to be persistent.

    Dragonfly has implemented this storage subsystem with the 'varsym'
    command/subsystem.

the connection:

    At least as i understand it, we already have plenty of storage
    subsystems in the OS that fit most of the above requirements (except
    perhaps the read cost) including the sysctl tree, the kenv tree, the /proc
    filesystem -- so i think it would be great to have a unified
    solution [at least as a backend] for all these purposes.

    I haven't looked at the internals of /proc or varsym so i don't know how
    suitable are them for our purposes.

> Also, this problem isn't just replacing a table in the kernel.  The
> problem is "map this ID to that ID" because many drivers do special
> things for different IDs, and you have to specify the ID that it is
> compatible with.


what's wrong with something like this

	driverparms.if_re.devid.0x11864300 = "0x04000000 D-Link DGE-528(T) Gigabit Ethernet Adapter"
	driverparms.if_re.devid.0x11868139 = "0x74800000 RealTek 8139c..."
	...

The subtree under if_re only needs to be known to the device, though
some standardization would help of course -- e.g. we could have generic
code to parse device ids of variable size using wildcards, and for
the value field we could use xml encoding (if something more trivial
does not fit -- but a barebone xml parser such as the one we need here
fits in 5-10k of object code, i did that a few months ago , see

	http://info.iet.unipi.it/~luigi/FreeBSD/minixmlrpc/

Hooking a write callback on a node, e.g. to driverparms.
if_re.devid would allow the driver to update its own internal
table of descriptors upon a change, and retain the current match
code (and speed) essentially unchanged in all drivers.

> : linux has a way (forget what the command name is) to add entries
> : to the table at runtime, whereas on freebsd
> : we need to patch&rebuild the module.
> : 
> : if make this 'object store' thing (varsym) able to store arrays we
> : could have device drivers scan the arrays (e.g.  uscanner_id or
> : if_rl_pciids etc.) to find out if it has a suitable string.
> 
> That's the wrong way to solve the problem.  In FreeBSD there's no
> universal table on any bus except for PC Card.  Until we have that,
> this solution can't happen.  Each driver has its own ad-hoc way of
> doing this.  Even in PC Card land, the size of the table isn't
> exported.
> 
> The way that drivers are written today in FreeBSD, the bus has to
> provide this translation layer.  There's really no other viable
> solution.  If someone goes through and fixes all the important busses,
> then maybe this would be a needed service.

perhaps you are pointing to an ideal solution, which however would
still require significant work on each driver to adapt the current,
ad-hoc tables to the solution supplied by the bus.

The approach i suggest above allows incremental deployment and i believe
it still scales well.

	cheers
	luigi