Re: What to do about tgammal?

> On 18 Dec 2021, at 17:51, Steve Kargl <sgk@troutmask.apl.washington.edu> wrote:
> 
> On Sat, Dec 18, 2021 at 10:41:14AM +0000, Mark Murray wrote:
>> 
>> Hmm. I think my understanding of ULP is missing something?
>> 
>> I thought that ULP could not be greater than the mantissa size
>> in bits?
>> 
>> I.e., I thought it represents average rounding error (compared with
>> "perfect rounding"), not truncation error, as the above very large
>> ULPs suggest.
>> 
> 
> The definition of ULP differs according which expert you
> choose to follow. :-)  For me (a non-expert), ULP is measured
> in the system of the "accurate answer", which is assumed to
> have many more bits of precision than the "approximate answer".
> From a very old das@ email and for long double I have

<snip>

Thank you!

I checked the definition that I was used to, and it is roughly
"how many bits of the mantissa are inaccurate (because of
rounding error)".

I can see how both work. For utterly massive numbers like
from Gamma(), I can see how accounting for a much larger
range works.

It still feels slightly tricky, as e.g. how many digits after the
floating point do you account for?

> I don't print out the hex representation in ld128, but you see
> the number of correct decimal digits is 33 digits compared to
> 36.

Looking good!

M
--
Mark R V Murray