Re: What to do, if chatgpt gives a wrong answer

On Sat, Oct 11, 2025 at 2:08 PM Julian H. Stacey <jhs@berklix.com> wrote:

> Sat, 11 Oct 2025 14:55:11 +0200 "Christos Margiolis" <christos@freebsd.org>
> wrote:
>
> "Christos Margiolis" wrote:
> > On Sat Oct 11, 2025 at 1:14 PM CEST, ft wrote:
> > > Hello,Â
> > >
> > > When I ask ChatGPT a question about FreeBSD, I very often get the wrong
> > > answer. Does FreeBSD have a plan for improving the quality of answers
> > > so that it can be taken into account in the next version?
> >
> > I'll bite, even though it's not the appropriate mailing list.
> >
> > Why should we take any LLM into account for the next version? We have
> > documentation (man pages, handbook, other docs), and real people you can
> > speak to. There is no reason to waste already scarce resources to
> > improve ChatGPT.
> >
> > Christos
>
> Agreed.  A very old music hall joke :
>   "Doctor! Doctor! It hurts when I do this ... !  "Well, don't do that
> then!"
> ie Don't expect quality AI about small moving targets (eg FreeBSD).
>
> AI will do better against static large targets, where a vast general
> public can be tempted to contribute responses free for AI to hone
> improvements.
> That's not us.
>
> AI is proving useful to smaller groups eg doctors, ("Did you also
> consider the rare XYZ disease?") I assume theres lots more doctors
> in the world than FreeBSD users, & some doctors will see it as part
> of their paid job to correct false responses from AI.
> That's not us.
>
> Unpaid volunteer time seems better spent improving FreeBSD & docs, not
> inefficiently correcting old AI mis-understanding of a moving target.
>
> If/ when AI learns source code & SVN etc, all bets off though - Shudder !
>
> Cheers,
> --
> Julian Stacey  http://berklix.org/jhs/mail/  @gmail blocks replies.
> Arm Ukraine.  Contraception V. global warming.  Israel starves Gaza.
>
>
All kidding aside, I run into this at work a lot. I work for a nameless
computer company , where we are not encouraged to ask an AI, but no one
really cares either way.
So a co-worker used one of them, whatever is approved don't remember, to
draw up some details on X vs Y for some upcoming project. The short
version, they made a report
based on what this thing said and it was garbage. If they had just asked
the SMEs in the group what do you favor , X or Y. They would have had a
great data, if they didn't
trust the results they could have gone off and investigated for their
selves.

So back to this mailing list. The ChatGPT's , Claude whatever scraped this
mailing list to build some sort of data about FreeBSD, who know what it
really did with any of that data.

But if you need a primary source or a subject matter expert, The Forums and
the mailing lists are the place to go.  If you do not trust what people are
saying, go test it, report back what
you see .

The only people who benefit form the LLM AI boom are some super rich people
who are trying to buy the world .



-- 
mark saad | nonesuch@longcount.org