portaudit wishlist

[Oliver Eikemeier]

Jacques A. Vidrine wrote:

> On Sun, Aug 22, 2004 at 11:39:47PM +0200, Oliver Eikemeier wrote:
>> It should be trivial to deal with the <alternate> tag in XSLT, the same
>> should be possible with <optional>, and for entering them into 
>> databases
>> you have to render the stuff anyway.
>
> One of your concerns was bloat, and I think using
> <alternate>/<optional> like this would be just as much bloat.  The
> only redeeming quality of the csh-style syntax was brevity (but as I
> said, I don't think that's a sufficient reason to use it).

Bloat as in `linear search'. It's easy to render <alternate> to 
csh-style braces.

>> Readability of the XML code is a
>> non-issue, since it is designed to be machine-readable, not
>> human-readable.
>
> No, that's really not correct.  XML is not necessarily user-readable,
> but it may certainly be human-readable.  Or do you think that all our
> doc committers that work on the web site or hand book and so forth
> work in "write-only" mode? (rhetorical question)

Users should have tools. I may write a `make entry' when we settled on a 
format suitable for our purposes.

>> portaudit is designed to be lightweight and work without
>> a database, so it does linear searches on all systems. I might change
>> that, but that's the way things currently are, so what's the problem
>> here?
>
> I don't know, what *is* the problem here?  What problem are you trying
> to solve with this new syntax?  It doesn't seem to be readability
> (the csh syntax and <alternate>/<optional> elements are both worse
> for readability).  You mentioned space, but I don't believe that's an
> issue, and you seem to not either.

I don't care about the size of the vuxml file, but about the rendered 
version. I'm fine with openldap-{client,server}, but you had concerns 
that this isn't unsable for xml tools, so I suggested an alternative in 
xml.

>> [...]
>>> So basically I think we should introduce this in VuXML 1.2.  I'd like
>>> to hear some comments from others about it, especially from the point
>>> of view of the user reading content generated from VuXML.
>>
>> Uhm, you don't need vuxml 1.2 for that, this is perfectly legal in 
>> vuxml
>> since 1.0.
>
> No, it is not.  You can of course have a *valid* VuXML document with
> `1.*' or even `@#$@% mary had a little lamb' as content for the <range>
> child elements, but that doesn't make it a legal or proper VuXML
> document.  The DTD is only part of the specification.  The other part is
> described in the DTD comments.

I can only find `The version ranges given must not overlap.' Other 
pointers? I still think this is a proper vuxml range specification. How 
a platform defines (and sorts) version numbers isn't handled in the 
vuxml specification.

>>>> [...]
>> I can't really follow your rationale here. You claim that because it
>> can't be done perfectly, it shouldn't be done at all.
>
> I don't think you read and understood what I wrote.  Let me try to
> be clearer: the security team does not currently assign severity to
> security advisories or VuXML entries, and has no plans to do so.
> Rather, we believe that all of these security notices should be acted
> upon.

Theory and practice. There are seven unfixed vulnerabilities on freefall:
   ssh <login>@freefall.freebsd.org pkg_info | cut -f 1 -d ' ' | xargs 
portaudit

It is useful to have the ability to mark vulnerabilities as `minor', 
like for example CAN-2004-0777. I did make an entry because we can't 
currently rank them. And insisting that this is not useful for `the 
security team' supports the theory that portaudit and vuxml need 
different databases.

>> I would find it
>> incredibly useful to get some categorization into `dangerous',
>> `important' and `minor', even when it's wrong sometimes.  As discussed
>> before: You usually have a pretty good idea whether a vulnerability is
>> severe or not, you just don't want to tell anybody.
>>
>> I consider this valuable to give users a chance to customize the
>> notification scheme of portaudit, and why shouldn't this information
>> find its place in the vuxml database? Make this tag optional, so you
>> don't have to fill in anything when you on't feel like it.
>
> I think that any severity rating system should be seperate from the
> VuXML documentation to allow for multiple such systems and policies,
> particularly as practices in this area evolve.

I don't think so. Either we have a common view on vulnerabilities, or we 
don't. I guess we should make vuxml the official database format, 
portaudit the official tool and be set. What are `multiple systems and 
policies' in this context? a severity rating is advisory only, and could 
be used or ignored however the system thinks is fit. What's the point 
when FreshPorts has an other severity level that portaudit? Or should it 
use the portaudit severity ranking? I can't understand your problems 
here.

>>>> - add a classification into remote/local exploitable
>>>
>>> On the one hand, I feel this should be handled in the narrative
>>> description and that it is otherwise just another facet of the
>>> severity rating.  On the other hand, I can imagine someone deciding
>>> that it was OK e.g. to install any ports that did not have a
>>> *remotely* exploitable vulnerability.
>>>
>>> My instinct tells me this too should be adjunct, at least
>>> for now.  Why would we include remotely/locally, but not
>>> authenticated/unauthenticated, application privileges/system
>>> privileges, user privileges/superuser privileges, or other such
>>> aspects?  If you have a server with a vulnerability that lets someone
>>> who has a local account to get some other access, would you record
>>> that as local or remote?
>>
>> umh, local of course?
>
> Funny you should say `of course'.  I would have classified it as
> remote.  I picked this example from some correspondence involving
> several colleagues, and guess what: the answers were mixed.  Which is
> `of course' my point.

You have border cases in every system. Just denying to add useful 
features because some entries could be erroneous isn't helpful. Users 
could use this information, even when it's not 100% correct. Even the 
<range> entries in vuxml are not 100% correct, so what?

>>> Yes, I think it is misleading to apply such tags which a user might
>>> take as an absolute judgement when in fact they just need to read the
>>> description.
>>
>> Not everyone has the time to review every description.  Besides, the
>> description might be as wrong or misleading as the tags mentioned. If
>> you say "users have to understand the system fully or they shouldn't 
>> run
>> the software" you basically state "FreeBSD is only for experts". I'm
>> just trying to make some often asked questions machine readable.  For
>> example when I run portaudit on a server with no users, I might decide
>> to care for local exploitable vulnerabilities only ever friday, while I
>> have to handle remote exploitable vulnerabilities immediately. This
>> system is not perfect, but usable. You give users basically no way to
>> filter the information, which would be a valuable feature. One one hand
>> you state users have to be knowledgeable to run a system, one the other
>> you claim they might take tags `as an absolute judgement'. In this case
>> reading the (possibly wrong) description might not improve anything.
>
> Your ``reasoning'' makes me dizzy.
>
> Look Oliver, knock yourself out: come up with your own severity rating
> scheme and implement it.  Stop bugging the security team to do it,
> I've already explained that we will not at this time.

Ok, back to my own database specification then? We have just a different 
view on our user base, and I think you fail to address some needs. Not 
everybody is a purist here, some `just want to have the job done', even 
when this means to err once or twice.

>>>> - add a `fixed' field that lists a version where the vulnerability is
>>>> fixed. This could be used for a recommendation message, like "upgrade
>>>> to
>>>> version xxx" or "no upgrade is available, please deinstall the port 
>>>> or
>>>> proceed with caution".
>>>> This could also realized as an alternate <lt> tag.
>>>
>>> I guess I don't understand this request.  That is the purpose of the
>>> <affected> element and children.
>>
>> There is no information whether there is an update available (and since
>> which date the vulnerability is fixed), or if I simply have to 
>> deinstall
>> the software and use a different product.
>
> Maybe you can describe this a bit more fully?  (BTW, Can you make a
> separate thread for each of your proposals?)  I felt that the affected
> ranges were sufficient, but if there is a better way let's hear it.

There is no way to deduce from the VuXML document alone whether a fixed 
version exists (like in port<1.5: is there a port-1.5 in the tree or 
not?). So basically the user has to do the research whether s/he could 
just upgrade or has to deinstall the port completely (or live with the 
vulnerability).

> [... snip: adding `description's to each reference ...]
>> xfdb does it that way, and I like it. It's especially useful for 
>> mailing
>> list posts, to see whether they contain an advisory or an exploit, for
>> entries that cover multiple vulnerabilities (to distinguish the CVE
>> references) and to filter out those `Updated packages fix multiple
>> vulnerabilities' references for other operating systems.
>>
>>> The whole point of VuXML is to give enough information but not too
>>> much :-)  ``Too much'' is anything that is not likely to be supplied 
>>> in
>>> the vast majority of cases.
>>
>> This could be automated by the tool that makes entries, or even by a
>> tool that adds missing descriptions, so it is likely to be supplied.
>
> Well, I'm not sure I'm convinced.  But let's try an example.
>
>     <references>
>       <cvename description="BMP heap overflow">CAN-2004-0691</cvename>
>       <cvename description="XPM crash">CAN-2004-0692</cvename>
>       <cvename description="GIF crash">CAN-2004-0693</cvename>
>       
> <url>http://www.trolltech.com/developer/changes/changes-3.3.3.html</url>
>       <url>http://scary.beasts.org/security/CESA-2004-004.txt</url>
>    </references>
>
> Is this the kind of thing you mean?

More or less. Since these CVE references have no title yet, they would 
be shown as simple URLs. But you could have <url 
description="Trolltech - Changes 3.3.3"> 
http://www.trolltech.com/developer/changes/changes-3.3.3.html</url>.

> I wonder if it will really be
> used much *shrug* (by authors or users).  I'd like to know what you
> mean by `a tool that adds missing descriptions'.  Are you thinking
> of following the references and snarfing the <title> or similar?

Jup, with a possibility to edit or deny it when it is too meaningless.

> That might be neat.  But, it would create two different meanings
> for this attribute: (a) editorial description, i.e. to disambiguate
> multiple similar references to CVE names, source code, whatever; (b)
> the referenced item's description, e.g. <title> for HTML documents,
> Subject: for email.  I'm not sure I like that.  As you seem to point
> out, (b) could be largely automated, so I'm not sure it is worth
> storing it in the document.  Then when rendering a document, how will
> a reader know that what she sees is (a) or (b)?

Probably (b) or the url is fine here. (a) is too much work in the 
general case.

> One more thing: I suggested `description' as the attribute name, but I
> have second thoughts.  I believe it might be better to pick something
> that is not also an element name, and perhaps `description' is a
> little too much typing.

Whatever you like. Since entries should be done by tools, I don't care 
how much typing is necessary.

-Oliver