svn commit: r437215 - in head/graphics: gbm libEGL libGL libglapi

[Matthew Rezny]

On Thursday 30 March 2017 13:57:15 Johannes M Dieterich wrote:
> On 2017-03-30 12:40, Matthew Rezny wrote:
> > On Thursday 30 March 2017 11:39:11 Johannes M Dieterich wrote:
> >> Dear Matthew,
> >> CC: Jan, Koop, Steve
> >> 
> >> I will unify your two mails below.
> >> 
> >> On 2017-03-30 01:26, Matthew Rezny wrote:
> >> > On Thursday 30 March 2017 05:36:03 Jan Beich wrote:
> >> >> Matthew Rezny <rezny at freebsd.org> writes:
> >> >> > On Wednesday 29 March 2017 19:51:55 Jan Beich wrote:
> >> >> >> Matthew Rezny <rezny at FreeBSD.org> writes:
> >> >> >> > -	@${REINPLACE_CMD} -e 's|x86_64|amd64|' \
> >> >> >> > +	@${REINPLACE_CMD} -e 's|x86_64|amd64|' -e 's|\\S\*//|[:space:]*
> >> > 
> >> > //|'
> >> > 
> >> >> >> > \
> >> >> >> 
> >> >> >> [:space:] is invalid character class thus treated as a list of
> >> >> >> characters.
> >> >> >> \S corresponds to [^[:space:]], while \s to [[:space:]].
> >> >> >> 
> >> >> >>   $ man pcrepattern | col -b | fgrep -m1 \\S
> >> >> >>   
> >> >> >>            \S     any character that is not a white space character
> >> >> >> 
> >> >> >> This may break build given -march, etc. are no longer stripped.
> >> >> > 
> >> >> > I wish that information had been presented when I said "I guess it
> >> >> > should
> >> >> > have been [:space:] instead of [:graph:] in the replacement." after
> >> >> > you
> >> >> > stated [:graph:] was plain incorrect, although it is what had been
> >> >> > previously suggested to me and did seem to be working.
> >> >> 
> >> >> I didn't focus on pointing out every mistake with the existing hack
> >> >> because it was soon going away. Now that devel/libclc depends on
> >> >> llvm40
> >> >> the original motivation to hold out on 17.* (bug 217016) before 2017Q2
> >> >> has been weakened.
> >> > 
> >> > Pointing out something is wrong without giving the correct solution is
> >> > not
> >> > helpful. The upstream change in the 17.1-dev branch was not directly
> >> > applicable to 13.0.x, so the the 'hack' would remain unless I was going
> >> > to
> >> > change to change the configure.ac and patch-configure myself, which is
> >> > certainly
> >> > more work that to edit the post-patch and it would have been the same
> >> > changes
> >> > in either place lacking clearer input.
> >> > 
> >> > Nobody said anything to me about committing an update to libclc at this
> >> > moment. I do not believe that has tested in combination with the rest
> >> > of the
> >> > graphics stack at the current versions in ports, the mix of LLVM
> >> > versions
> >> > almost certainly will be a problem, and it's a day before the quarterly
> >> > branch. WTF? I've been holding off Mesa 17.x and LLVM4.0 for after that
> >> > branch
> >> > while attempting to get Xorg 1.19 ready in time for that. The latter
> >> > won't
> >> > happen because it took over a week to even start an exp-run on the
> >> > bsd.xorg.mk
> >> > changes. I just explained the reason for holding of an update of libclc
> >> > yesterday in a PR that proposed a more recent snapshot for the
> >> > transition to
> >> > dependence on llvm40. I had not even gotten everything onto llvm39 yet;
> >> > pocl
> >> > 0.14 should be compatible past 3.8, but it is not yet released and I
> >> > had
> >> > difficulties with rc1 as they switched to cmake so the build system
> >> > needs to be
> >> > redone. While I'm sure Mesa 17.0.x will be ok with llvm40 after
> >> > appropriate
> >> > patching (I had to add several patches for clover in Mesa 13), I cannot
> >> > say
> >> > the same for all the OpenCL ports, i.e. beignet which was only recently
> >> > made
> >> > compatible past llvm37 with the 1.3.0 update that allowed it to use
> >> > llvm39.
> >> > So I expect r438268 to be reverted, or someone else to handle all the
> >> > fallout
> >> > this causes on the quarterly branch during Q2. I realize multiple
> >> > people want
> >> > to help, but we need some coordination or else we are just wasting each
> >> > other's time. Sorry to be brunt, but that was an ill timed commit.
> >> 
> >> This patch was committed by me, after
> >> https://reviews.freebsd.org/D9394
> >> got accepted by swills, reviewed by kwm (with x11 hat on) in the
> >> beginning of February. No objections were raised since then. I did not
> >> commit it earlier to wait for stable llvm40 to hit the tree. I again
> >> checked with swills yesterday before committing.
> > 
> > I was not aware of the review until the commit. I do not know why
> > nobody
> > bothered to tell me other than there is serious lack of communication.
> 
> There is not from our side. See below.
> 
Oh yes it is...

> >> Hence, I will back this out if either kwm or swills tell me to.
> >> Steve/Koop: opinions? I will also personally say that I believe we
> >> should only support one version of llvm (the latest stable) across
> >> Mesa
> >> ports if possible, which is another reason why my patch to
> >> graphics/libGL bumps to LLVM4.0. I do not see any added value despite
> >> more maintenance work in supporting older versions of LLVM for Mesa.
> >> The
> >> PR you referenced was also explicitly looking for support of a newer
> >> LLVM, not an older one.
> > 
> > I do not advocate sticking to older LLVM for long, and just made it
> > easier for
> > people who want to try moving ahead. I merely want to wait until after
> > the
> > quarterly branch to switch to llvm40 because I expect some fallout.
> > Also, I do
> > want to be sure all the OpenCL ports can use that version. I had wanted
> > to get
> > Xorg 1.19 before the quarterly branch, then after the branch do the
> > rest, but
> > it'll all have to wait a bit to not have a broken quarterly.
> 
> OK, to be very blunt: the current HEAD is just bad for accelerated
> OpenGL. Nothing newer than Ivybridge is supported unless you go with the
> NVIDIA BLOB (which I have zero experience with). OpenCL is even more
> horrible: clover is a joke (I am actually using it for work on carrizo)
> and you'll be hard pressed to find any HW around that would be able to
> use with it unless you are on drm-next. Beignet I worked on porting over
> back in the day with kwm, it is an even bigger joke b/c even with
> drm-next you only get single precision (I am unfortunately still on
> their mailing list and they now are slowly adding DP support it seems).
> An NVIDIA rep told us at an XDC directly that they will not support
> either CUDA or OpenCL on BSD unless we can show a customer demanding
> this and buying a few thousand GPUs. pocl is nothing that you would
> actually want to use unless you want to test drive OpenCL code (which I
> figure I am one of the few people on FreeBSD that actually does this).
> 
> But if you have other insights into OpenCL that my experience with it
> and daily use of has not revealed: please do educate me!
> 
> So, the situation is as follows: you need (very) old HW to use
> OpenGL/OpenCL on anything including CURRENT. For anything newer,
> however, you need drm-next and a newer Mesa with llvm40. So this
> quarterly just doesn't have that big of an impact for any decision as
> people will either be on drm-next, use software rendering, or eventually
> jump to the kms ports.
> 
To be very blunt, the situation is crap and I've been wanting to fix it for 
years. Support is split across three segments. There's the DRI1/UMS hardware 
for which support has almost been killed in ports and the kernel status is 
questionable, this I need to fix but there's still some blockers artificial). 
There were excuses of nobody to work on the older stuff while I was 
volunteering to do it and being ignored. There's the DRI2/KMS hardware 
supported by the kernel, which works well with what is in ports and is what I 
am trying to support for the users as well as myself. It is all we have been 
able to use reasonable for some time. I had tried to get involved working on 
this years earlier, but my attempts were met with "no, we've got it covered", 
so I sat back, watched, trying to prod it along further. And finally there is 
the drm-next work to use hardware in generations beyond, so again no overlap 
with the support. This is work that started in response to an absolute 
stagnation of the graphics stack during the last year. It does not make sense 
to bring stuff into FreeBSD ports that does not work with any FreeBSD kernel. I 
am concerned with OpenGL first and OpenCL second. I have no real use for 
OpenCL, and haven't been able to see it work, so I've just tried to make it no 
worse.

> >> Concerning your changes to graphics/libGL: I appreciate you wanting to
> >> help and improve things. The Mesa ports are in dire need of that. I
> >> however would also appreciate if you would acknowledge the work other
> >> people (Mark Johnston, Johannes Lundberg, Kip Macy, Koop Mast, Pete
> >> Wright, and myself) have put into this over the last 6+ months. For
> >> some
> > 
> >> selected time line:
> > I appreciate Koop's past work and Macy's efforts throughout the past
> > year. I
> > actually don't know what the rest of you have been up to since there is
> > little
> > discussion. kwm and dumbbell are no longer active with x11 as far as I
> > know
> > and rarely active on IRC. Macy used to be active on IRC but not lately,
> > and
> > I'm not sure I've seen you on the xorg channel but I don't know what
> > nick you
> > might use. As far as I know, I am the only one attending to the
> > graphics ports
> > in FreeBSD for the past quarter. Macy mentioned trying to get more
> > people
> > involved a few times, but since nobody else ever showed up on IRC, I
> > had no
> > idea who might be up to what, at least until I saw your commits.
> 
> Sorry, but that's simply not true. There have been multiple status
> reports where our efforts were discussed, there are relatively regular
> graphics meetings organized by Ed Maste, everything happens in public
> githubs, we are responsive to github issues, there are reviews on phab,
> all of us have email, and more. This may predate your involvement with X
> but please do not mis-construct this as a lack of communication from our
> side just b/c we are not as active on the one communication channel you
> prefer. It would be good for you to contact Ed (emaste@) and get onto
> the regular graphics meeting list to coordinate your efforts with us.
> 
My involvement in X in terms of trying to help goes back at least 4 years. I 
contributed many patches to the x11 repo when it was svn and for a while after 
it moved to github. Eventually kwm stopped updating that repo, it stagnated 
and only caused confusion while I maintained my ports tree and shared patches 
on occasion. Finally I was let to commit my work because there was absolutely 
nobody else that showed any interest in doing so.

I know there were and are other github repos, but they are just that, external 
repos where individual work happens. I'm working on FreeBSD and what is in the 
official subversion repo is what I see. As I have been working alone as far as I 
knew, I felt no need to setup yet another repo, but if people would like to 
work with me, then returning to svn would be my first suggestion. I value my 
work too much to throw it down the toilet by using git.

I was aware that there was some sort of regularly scheduled conference call 
going on just before dumbbell and kwm essentially ceased x11 activity. I asked 
if I could be involved, but I was told no; I was just a lowly outsider. Since 
they became inactive, nobody else has mentioned calls on IRC, so I assumed 
those stopped last fall when things ground to a halt. If those call are still 
going on, they are secret meeting to which I have not been invited! I have not 
been contact by emaste, though I did just throw him a bone on IRC regarding 
the libclc then currently (now formerly) in ports not being compatible with 
LLVM 4.0.

There was a status report last quarter for FreeBSDDesktop, which I read as a 
separate effort from anything the x11 team is involved in.

> I am honestly extremely disappointed by the fact that the hours we spent
> on this are now at least partially wasted and I need to waste even more
> hours to rebase and test this patch again.
> 
Without coordination, it's all wasted. How do you think I feel getting told 
that there's a bunch of duplicate work I should have known about, or that 
there's now other people who are seriously interested in working on FreeBSD, 
after having sat and watched things deteriorate while being denied entry until 
nobody could deny the situation was dire and the only option was to let the 
"new guy" come take a swing?

> >> * Johannes Lundberg enabled wayland and dri3 on the public
> >> FreeBSDDesktop github in last autumn
> >> * Kip and I changed to using libudev-devd there in December
> >> * I updated to Mesa 17.0 rc in January there
> >> * I bumped to llvm40 in the end of January there as that was needed to
> >> support Polaris and Carrizo (better)
> >> * before that part goes amiss: compilation with the same llvm as is
> >> linked against is also needed for amdgpu, we had crashes otherwise
> > 
> > Work in external repos doesn't help until it is merged. Nothing was
> > pointed
> > out to me as merge ready. I know there is plenty of work going on
> > there, but
> > as far as I knew, the focus is on the kernel modules and any changes to
> > userland/ports were just temporary support until the kernel parts will
> > be
> > merged. Nobody said there was anything mergeable from the ports and I
> > haven't
> > time to pick through it at to figure out what is ready. I have looked
> > at some
> > of it, what I've stumbled on really, and not much looks like it was
> > done with
> > an intent for clean merge so it requires more work to use on stock
> > FreeBSD.
> 
> The merging part is exactly what I am doing. There are also open reviews
> for the kernel modules I opened, which are blocked by linuxkpi changes
> that haven't made it upstream yet.
> 
I've not been alerted to any of these reviews. The one review for Mesa 17, 
when it was premature, is the only one that was ever mentioned to me, and as I 
said it did not look like effort had been made to ready if for FreeBSD ports.

> >> * throughout this time kwm (with x11 hat on) was involved in these
> >> updates
> > 
> > Which time frame are you talking about? I haven't seen kwm active in
> > x11
> > affairs in at least 4 months. I had opened the mass of PRs for Xorg
> > 1.18 when I
> > started my 1.19 work in order to try to kick him, or anyone else into
> > committing, and the result was I got a bit to commit those bapt hadn't
> > already
> > gotten around to. Since then I've been working mostly alone, not that I
> > want
> > it that way.
> 
> kwm has been reasonable active the last year from what I can testify to.
> 
Active on what? I had xorg and mesa patches stacking up that I couldn't get 
into his github or into the ports tree. He said he was going to do a CFT for 
Xorg 1.18, even mentioned it in a status report I believe, and then another 
quart goes by until bapt finally does the CFT just in time for he and I to 
commit all that work, which by that point had superseded everything in various 
external repos as far as I knew. It's been at least half a year since I saw 
him active on x11, and even longer that I've seen the gtk/gnome stuff 
stagnating. There was a gtk+ 3.20 update in progress middle of last year that 
still hasn't been done (I have a theme update PR blocked on the gtk3 update 
PR) while 3.22 has been out for months. So when he said something about not 
having time for FreeBSD anymore, I figured that was finally the self-admission 
that someone else needs to fill the role. I have asked questions on IRC about 
why things are the way they are but don't get answers. I see dumbbell still 
says good morning sometimes but I haven't had anything I needed to ask.

> >> * I uploaded the first version of the Mesa 17.0 patch to reviews on
> >> Feb
> >> 7 (which you were apparently aware of)
> > 
> > It was pointed out to me when I was committing Mesa 13, at which time
> > it was
> > definitely premature, so something to come back to but after some
> > number of
> > times without seeing any change I assumed it dead, whoops. I think it
> > was
> > still an RC, or it was 17.0.0 but llvm40 was at rc1 was time I looked.
> > 
> >> * I've since updated the patch multiple times for minor releases and
> >> address comments by mat and kwm (with x11 hat on)
> >> * again, no comments/objections were raised there by anybody else
> > 
> > I suppose I should have raised concerns earlier, but it looked too raw
> > (too
> > much stuff specific to drm-next) for me to think it anywhere close to
> > being
> > committed, or to think to search for something like an update to
> > libclc. I was
> > also kinda hoping you'd join #freebsd-xorg at some point.
> > 
> >> The first Mesa 17 patch in PR217016 dates from March 6, a month after
> >> my
> >> first Mesa 17 patch on reviews. Apparently you were aware of (at
> >> least)
> >> the last two items on the list above. So yes, coordination is
> >> important,
> >> which is also why we have an x11 hat (Koop). As you can see from the
> >> above, Koop was involved in all of this from the onset.
> > 
> > I find it odd that Koop is barely responsive on IRC but active on some
> > reviews.
> > Whatever, just part of the coordination and communication issues. Last
> > time I
> > talked to kwm at any length he said he didn't really have time for
> > FreeBSD, so
> > I counted him out since then, only bothering to ask the occasional
> > question
> > without reply.
> 
> kwm has the x11 hat and I consider him therefore the only authority on
> this. If he feels like he can't do that anymore, I am certain he'll let
> us know and step down. Assumptions are not necessary and as we've seen
> above not helpful.
> 
It sounds like there's some misunderstanding about hats. There hat isn't a 
crown and there isn't a single person who runs the x11 team; x11 is a group of 
peers.  When someone on the team does something where they represent the team, 
e.g. commit ports maintained by x11@, they are wearing the x11 hat for the 
duration of that action. The hat is an indicator like a jersey for a sports 
team. I might be one more than one team (i.e. KDE, sorta) and I switch hats as 
needed.

Unless someone rage-quits, which has happened, people tend to dissipate. Their 
priorities shift and they come around less, but it often takes time to say 
they are done. I don't think kwm has to step down for anyone else to get 
involved, but if so, then I'd say that already happened with the 
aforementioned statement on IRC.

According to https://wiki.freebsd.org/Graphics/ the x11 team is:
Baptiste Daroussin (active, but busy with many other things)
Eitan Adler (not seen active with x11 in at least a year)
Jean-Sébastien Pédron (x11 inactive since last summer, active on other ports)
Jung-uk Kim (not seen active with x11, active elsewhere)
Koop Mast (inactive for at least half a year)
Niclas Zeising (not seen active in at least a year)
MatthewRezny ("they new guy")

Now I know that page isn't really up to date because I have not been able to 
edit it (I need to bug someone about wiki permissions) to state the current 
status, but I as I don't see Mark Johnston, Johannes Lundberg, Kip Macy, Pete 
Wright, or Johannes Dieterich on that list, I wouldn't count y'all as members 
of x11 at . You might all be doing some great work in your FreeBSDDesktop 
project, but that is separate from the FreeBSD project. If you would like to 
volunteer for team x11@ then I encourage you to get active on our IRC channel 
and ask to be added to the team roster.

> >> > I had not noticed that your review was updated recently as it sat idle
> >> > for
> >> > quite a while after the update to 13.0.x landed in ports. I'm curious
> >> > about
> >> > the comment regarding a need for libudev. As far as I know, Mesa
> >> > dropped the
> >> > dependence on udev in v13 and relies upon libdrm for all the direct
> >> > hardware
> >> > interaction. Initially, I had some hope to use libudev-devd as a single
> >> > translation layer, but after reading through the libdrm code it became
> >> > obvious
> >> > that would not be the case as the conditional udev code in libdrm is
> >> > withing
> >> > Linux-specific code paths so no help to us. Thus, I implemented support
> >> > for our
> >> > platform directly in libdrm in order to drop dependence on libdevq,
> >> > which was
> >> > needed for no other purpose. Could you explain why Mesa 17 would need
> >> > libudev-
> >> > devd for AMDGPU?
> >> 
> >> We used libudev for loader related functionality in Mesa. I am happy
> >> if
> >> we do not need it anymore, at the time Kip and I just wanted to get
> >> rid
> >> of libdevq -- a FreeBSD-ism. libdrm did not support the linuxkpi
> >> amdgpu
> >> and Mesa would not load radeonsi properly.
> > 
> > Is libdrm missing some functionality I should implement, or was it just
> > a
> > matter that the version with libdevq hacked in was only partially
> > working? I
> > have tried to address the shortcomings I knew of in libdrm and cater to
> > the
> > differences I was aware of in drm-next, so it would be good to know if
> > libdrm
> > sans libdevq works for what you need. I've only had test reports from
> > users of
> > drm-next that have replaced libdrm and removed libdevq, but may still
> > have a
> > modified version of Mesa. I do not know the extent of changes in the
> > the
> > external ports collection.
> 
> It didn't work back then for amdgpu with libdevq. I believe i915 worked,
> so you may well just get those reports now as the number of people with
> amdgpu HW is probably tiny.
>
Please test the new libdrm and let me know the situation. "It didn't work back 
then" doesn't tell me anything meaningful about the prior deficiencies.
 
> >> Concerning the libdrm udev: as you may have seen from the
> >> FreeBSDDesktop
> >> github, I tried to do that before Christmas w/o success.
> > 
> > I have not trawled through GitHub nor was I planning to. The work there
> > is
> > focused on newer hardware than I own and I have enough else to keep me
> > occupied supporting the hardware I do own, which includes stuff I must
> > revive
> > support for after it feel by the wayside over the last several years.
> > If there
> > is stuff worth looking at, please point it out to me.
> 
> I believe bapt has committed all the wayland ports from Johannes
> Lundberg, there are some alpha ports for ROCm in there but those are
> blocked on amdkfd at the moment.
> 
> This github will be deleted once Mesa 17 hits the tree and replaced with
> a fresh one (the svn->git force push has broken this one)
> 
One more reason I have no interest in using github.

> >> >> > To be sure there is no misunderstanding now, would you consider this
> >> >> > correct? @${REINPLACE_CMD} -e 's|x86_64|amd64|' -e
> >> >> > 's|\\S\*//|[^[:space:]]* //|' \
> >> >> 
> >> >> Not quite, adding extra space after [] is unnecessary.
> >> >> 
> >> >>   -e 's|\\S\*|[^[:space:]]*|' \
> >> > 
> >> > It is interesting how many variations appear to produce the same
> >> > result. You
> >> > mention the space is not needed, so 's|\\S\*//|[^[:space:]]*//|' but
> >> > then the
> >> > example has the trailing // removed. Any reason why?
> 
> I will also at this point say that further discussions are moot until
> Steve/Koop step in for libclc and you had a chance to catch up with us
> on a graphics meeting.

I'll ask Steve as my mentor about reverting the commit to libclc that came 
from outside the x11 team. I would don the x11 hat and do it immediately if 
not a mentee. I will be surprised if Koop steps in, but he did already say the 
same thing in the review that I did earlier, "This should probably wait until 
the next mesa to keep the llvm versions in sync used by libclc/mesa/beignet." 
I did not invest all the time I did over this past quarter only to have a 
broken graphics stack in the quarterly branch.