Proposed addition of malloc_size_np()

[John Baldwin]

On Monday 27 March 2006 11:34, Jason Evans wrote:
> John Baldwin wrote:
> > On Sunday 26 March 2006 13:04, Jason Evans wrote:
> > 
> >>Robert Watson wrote:
> >>
> >>>I wonder if the intuitive objection people are raising is actually with 
> >>>the name.  Since malloc_size() is defined on at least one platform to 
> >>>return the requested size, maybe a name like malloc_allocated_size() (or 
> >>>something a bit more compact) would help avoid that confusion, and make 
> >>>it clear that the consumer is getting back a commitment and not a hint 
> >>>for future realloc(), etc.
> >>
> >>Maybe you're right.  We could just call it malloc_usable_size() and be 
> >>compatible with Linux.
> > 
> > It would help to know why such a function would be useful. :)  Do you have
> > a specific use-case?  If the purpose is for a program to see that it really
> > as Y >= X bytes when it did malloc(X) so that the program can use Y bytes,
> > that would seem to only be a source of bugs and complexity.  If the program
> > wants Y bytes, it should malloc(Y).  Many folks in the thread seem to think
> > that this function would be used for a poor-man's realloc() wrapper or
> > something, and I think such uses would be short-sighted at best.  If there
> > are other uses such as for having a debug malloc wrap the real one, then
> > that might justify the API, but it is unclear what a useful use of this API
> > would be.
> 
> I can think of a few straightforward uses:
> 
> 1) Suppose that we are about to write to a string that we know is 
> malloc()ed, and we want to be sure that we aren't overflowing the 
> allocated buffer.  We can add an assertion that the buffer is indeed 
> large enough to contain what is about to be written to it.

In that case though you really want to know the size you allocated,
not necessarily the backing-store size, esp. if you are in a library
since your caller may (correctly) assume that if it allocates X
bytes for a string the library will only write up to X bytes.  So, I
think in that case you want the requested size, not the backing store
size.

> 2) Suppose that we are using some sort of dynamically scalable data 
> structure, such as a hash table that we double in size every time 
> fullness crosses some threshold.  In order to do that, we need to know 
> how large the table is.  We can store that information, or we could just 
> query the size.  (In this example, performance constraints might dictate 
> that we actually store the size of the table, but there are certainly 
> similar examples that wouldn't be so concerned with performance.)

I think this devolves into the poor-mans realloc. :)  Instead, the code
should just realloc() when necessary and rely on the system malloc to
know when it is safe for the realloc() to be done in place.

> 3) This is what I want malloc_usable_size() for: garbage collection.  In 
> order for a garbage collector to know when it should run, there needs to 
> be some way of tracking how much memory is in use.  By using 
> dlsym(RTLD_NEXT, ...), I can intercept all malloc/calloc/realloc/free 
> calls and track current memory usage.  However, unless there is a way of 
>   getting the size of each allocation, I don't know how much to subtract 
> from the count for realloc/free calls.

This is a good reason, and is what I hoped you had in mind when you
proposed it. :)  I think that other than this type of code though (malloc
reimplementations or wrappers), nothing else should use this function. :)

> 4) Porting code from Linux.  Case in point: Xara Xtreme, currently being 
> ported by Vasil Dimov.  At the moment, he has to use dlmalloc.

Does it contain a G/C of its own or some such?

> Jason
> 
> Following is what I've written for the malloc(3) man page:
> ----
> The malloc_usable_size() function returns the usable size of the 
> allocation pointed to by ptr.  The return value may be larger than the 
> size that was requested during allocation.  malloc_usable_size() is not 
> intended as a mechanism for in-place realloc(), though it can be abused 
> that way; rather it is primarily provided as a tool for introspection 
> purposes.  Any discrepancy between the requested allocation size and the 
> size reported by malloc_usable_size() should not be depended on, since 
> such behavior is entirely implementation-dependent.
> ----

I would word it stronger: "malloc_usable_size() should not be used as a
mechanism for in-place realloc().  It is provided solely as a tool for
introspection purposes."

:)

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org