libthr: ia64 port: trapframe issue

[Marcel Moolenaar]

Gang,

After implementing cpu_thread_setup() as Mike pointed out I hit
upon another snafu. In kern_thr.c:thr_create() we copy the
trapframe over from the currently running thread (td) to the
newly created thread (td0). This however cannot be done on ia64:

On ia64 we can have a nested fault when trying to contruct a
trapframe. A nested fault is icky, because the CPU does not save
any state (it would otherwise clobber the state of the original
trap/interrupt). This also means that we have to know exactly
what can cause a nested fault so that we know what the faulting
address is and where we have to jump to when we inserted a TLB.
This means that we have to make sure that a trapframe does not
cross a page boundary, because otherwise we can have a nested
fault any place we write into the trapframe. To solve this, we
align the trapframe on a 1KB boundary.

Now the crux: if we align the trapframe, we don't know how much
the delta is from the previous stack pointer to the bottom of
the trapframe and hence the new stack pointer. This itself is not
really a problem, but we need to know this delta if we want to
restore the stack pointer to the value it had prior to the trap
(we're talking kernel stack pointer). Therefore, we save the length
of the trapframe (ie the delta) in the trapframe itself.

By copying trapframes, we also copy the length of the trapframe,
which is generally bad. The new trapframe is not 1KB aligned (we
know up front it will be contained in a single page) and thus
will have a fixed length (sizeof(struct trapframe)). However the
trapframe we copy from has been 1KB aligned and is generally
larger than sizeof(trapframe).

Anyway: the end result is that if we enter userland with a thread
created by thr_create, we clobber our PCB because we incorrectly
updated the stack pointer.

The following solutions exist:

1. quick and dirty:
-------------------

We know that the new trapframe is sizeof(struct trapframe) long, so
we simply correct the length after copying:

Index: kern_thr.c
===================================================================
RCS file: /home/ncvs/src/sys/kern/kern_thr.c,v
retrieving revision 1.8
diff -u -r1.8 kern_thr.c
--- kern_thr.c  16 May 2003 21:26:42 -0000      1.8
+++ kern_thr.c  1 Jun 2003 22:02:46 -0000
@@ -157,6 +157,9 @@
        td0->td_sigmask = td->td_sigmask;
        PROC_UNLOCK(td->td_proc);
        bcopy(td->td_frame, td0->td_frame, sizeof(struct trapframe));
+#ifdef __ia64__
+       td0->td_frame->tf_length = sizeof(struct trapframe);
+#endif
        td0->td_ucred = crhold(td->td_ucred);
  
        /* Initialize our kse structure. */

2. cpu__set_upcall() change:
----------------------------

We have all the MD specifics in cpu_set_upcall, so we move the
copying of the trapframe to cpu_set_upcall(). This means we
need to change its prototype to either
	void cpu_set_upcall(struct thread *new, struct thread *old)
or
	void cpu_set_upcall(struct thread *new, struct pcb *pcb,
	    struct trapframe *tf)

I don't know if we can simply remove the copying of the trapframe
and have it happen after crhold() and kse_alloc() or that we need
to move the call to cpu_set_upcall() up to before kse_alloc() and/or
crhold(). In other words: I don't know if we have a phase ordering
problem if we delay copying the trapframe until we call cpu_set_upcall()

Thought?

-- 
 Marcel Moolenaar	  USPA: A-39004		 marcel at xcllnt.net