Documentation testing and textproc/igor

[Li-Wen Hsu]

On Sun, Nov 23, 2014 at 16:58:30 -0700, Warren Block wrote:
> As far as I know, igor checkstyle output works now.  I would like to 
> expand on that later, but it can wait for the basic Jenkins doc test to 
> be completed.  

If I understand this paragraph correctly, you want to have a basic
jenkins doc test (checking if building is fine), which does exist:

    https://jenkins.freebsd.org/jenkins/job/FreeBSD_DOC/

and this jobs sends mail to freebsd-doc@ list.

FreeBSD_Doc-igor job does a bit more, as a "downstream" job for
FreeBSD_DOC, after each FreeBSD_DOC job finished successfully, it
launches igor and check if the doc files are well-written. Then
maintains a graph for tracking the "bad smell" counter.

Do you guys think it's time to send those "updates" to freebsd-doc@
list?  I thought it can send mail such as ""

   Checkstyle: 190 warnings from one analysis.
   17 new warnings
   13 fixed warnings

Just like we see in:

    https://jenkins.freebsd.org/jenkins/job/FreeBSD_Doc-igor/20/

I would try if this can be done by job configuration.

> If changes to igor are needed, possibly to silence those 
> checkstyle warnings, please let me know.  Beyond that, the Jenkins setup 
> is out of my hands.
> 
> I should point out that nothing formal was agreed at BSDCan.  We saw a 
> prototype of the Jenkins page using igor output and agreed that it would 
> be useful.

In BSDCan, I asked Warren if ci team can do anything for doc team, he
suggested runing igor over the doc tree to produce the report.  The
first part (running igor) is easy as just one line of build script.  The
second part is somehow difficult to me, since it needs a plugin knows
igor output, just like Jenkins treats JUnit output.

And I thought of while developing Java project in my $job, we use
checkstyle and its jenkins plugin for tracking the bad style in our
code.  So I did some quick hack in igor to make it generates checkstyle
output.  I also had some worries about the mesage:

    "No description available.  Please upgrade to latest checkstyle version."

And Warren encourages me that the idea worked and we can improve that later.

That message is because checkstyle is designed for Java and of course if
doesn't have rule descriptions for igor.  That's why the warning message
shown like this.  I don't have a easy solution at this point, it seems we
need to have build our own jenkins plugin, which might take checksytle
plugin as base.  That takes time, though.  Possibly, there is a magic
syntax in checkstyle output and can suppress that useless message.  It
also takes some (shorter) time for searching that.  Or, there is another
wonderful plugin we can use for generating igor report, we should change
to that.

So, what we have now is a proof of concept on jenkins.freebsd.org, just
as have we had in BSDCan.  I would like to listen to suggestions and
hopefully receive some help from more people. :)

Li-Wen

-- 
Li-Wen Hsu <lwhsu at FreeBSD.org>
http://lwhsu.org