NVIDIA and TLS

[Daniel Eischen]

On Sat, 14 Jun 2003, Andy Ritger wrote:
> I'd like to add a few comments to what has recently been said here
> and on freebsd-current to clarify what the %gs register is used
> for by the NVIDIA driver and what ideally would be done to allow
> the NVIDIA driver to coexist with FreeBSD threading implementations.
> 
> The NVIDIA driver does not need %gs to maintain internal state in the
> kernel; rather, it uses it to maintain fast thread local data for the
> OpenGL implementation, where it is critically important to have fast
> (single instruction) access to such data.
> 
> When the NVIDIA driver was initially ported to FreeBSD, two 
> threading implementations were available (linux-threads, libc_r),
> but neither of them supported static thread local storage or made
> use of it internally.
> 
> The situation was similar to what it had been on Linux for some
> time.  On Linux, the NVIDIA driver today deals with three possible and
> incompatible configurations: the original pthreads without fast TLS
> support, pthreads with internal TLS support, and current implentations
> which use static TLS internally and provide a mechanism to allow 
> both applications and libraries to store their own thread local data.
> 
> The linux_sysvec.c change which Christian Zander made available 
> on his website is related to the mechanism built into the NVIDIA
> Linux driver that decides which configuration is used; without this
> change, this mechanism fails from within the Linux ABI compatibility
> environment.  This is not related to the native NVIDIA FreeBSD
> OpenGL implementation.
> 
> The current NVIDIA FreeBSD driver only supports one threading library
> on FreeBSD for thread-safe, multi-threaded OpenGL applications: the
> FreeBSD port of linux-threads. The NVIDIA FreeBSD OpenGL driver uses
> both the i386_set_ldt system call and %gs to support high performance
> native OpenGL applications.
> 
> This obviously conflicts with new FreeBSD threading implementation(s)
> if they are also making use of %gs to store and access thread local
> data.  To solve this problem, ideally support for static TLS similar
> to that recently added to glibc would be implemented by FreeBSD
> threading libraries and supported by the NVIDIA FreeBSD driver.
> 
> A good reference and source for ideas is Ulrich Drepper's ELF 
> TLS paper:
> 
>     http://people.redhat.com/drepper/tls.pdf.
> 
> If this or a comparable model were implemented on FreeBSD, both 
> applications and libraries could be ported more easily between 
> Linux and FreeBSD.  Admittedly, I am not yet familiar with libkse
> or libthr... perhaps such a mechanism already exists? (where could
> I learn more about libkse and libthr?)

http://www.freebsd.org/kse/

mailing list: threads at freebsd.org
http://lists.FreeBSD.org/mailman/listinfo/freebsd-threads

> If you are interested, I can provide more details on why fast thread 
> local storage is so important to an OpenGL implementation.

I don't see anything like this happening any time soon[*].  It's
not a standard, and we have enough work ahead of us to make
libkse and libthr stable as well as implementing standards
that we don't yet currently support.

The code for pthread_getspecific is basically:

	pthread = _get_curthread();
	if (pthread->specific != NULL && key < PTHREAD_KEYS_MAX) {
		if (key_table[key].allocated &&
		    (pthread->specific[key].seqno == key_table[key].seqno)) {
			/* Return the value: */
			data = (void *) pthread->specific[key].data;
		} else {
			/*
			 * This key has not been used before, so return NULL
			 * instead: 
			 */
			data = NULL;
		}
	} else
		/* No specific data has been created, so just return NULL: */
		data = NULL;
	return (data);

_get_curthread() is function, but it's only one instruction.  It will
can be optimized (made a macro) later.

[*] Others are welcome to work on it, but it's not on _my_ TODO list.

-- 
Dan Eischen