Proposed addition of malloc_size_np()
jhb at freebsd.org
Mon Mar 27 20:19:32 UTC 2006
On Monday 27 March 2006 11:34, Jason Evans wrote:
> John Baldwin wrote:
> > On Sunday 26 March 2006 13:04, Jason Evans wrote:
> >>Robert Watson wrote:
> >>>I wonder if the intuitive objection people are raising is actually with
> >>>the name. Since malloc_size() is defined on at least one platform to
> >>>return the requested size, maybe a name like malloc_allocated_size() (or
> >>>something a bit more compact) would help avoid that confusion, and make
> >>>it clear that the consumer is getting back a commitment and not a hint
> >>>for future realloc(), etc.
> >>Maybe you're right. We could just call it malloc_usable_size() and be
> >>compatible with Linux.
> > It would help to know why such a function would be useful. :) Do you have
> > a specific use-case? If the purpose is for a program to see that it really
> > as Y >= X bytes when it did malloc(X) so that the program can use Y bytes,
> > that would seem to only be a source of bugs and complexity. If the program
> > wants Y bytes, it should malloc(Y). Many folks in the thread seem to think
> > that this function would be used for a poor-man's realloc() wrapper or
> > something, and I think such uses would be short-sighted at best. If there
> > are other uses such as for having a debug malloc wrap the real one, then
> > that might justify the API, but it is unclear what a useful use of this API
> > would be.
> I can think of a few straightforward uses:
> 1) Suppose that we are about to write to a string that we know is
> malloc()ed, and we want to be sure that we aren't overflowing the
> allocated buffer. We can add an assertion that the buffer is indeed
> large enough to contain what is about to be written to it.
In that case though you really want to know the size you allocated,
not necessarily the backing-store size, esp. if you are in a library
since your caller may (correctly) assume that if it allocates X
bytes for a string the library will only write up to X bytes. So, I
think in that case you want the requested size, not the backing store
> 2) Suppose that we are using some sort of dynamically scalable data
> structure, such as a hash table that we double in size every time
> fullness crosses some threshold. In order to do that, we need to know
> how large the table is. We can store that information, or we could just
> query the size. (In this example, performance constraints might dictate
> that we actually store the size of the table, but there are certainly
> similar examples that wouldn't be so concerned with performance.)
I think this devolves into the poor-mans realloc. :) Instead, the code
should just realloc() when necessary and rely on the system malloc to
know when it is safe for the realloc() to be done in place.
> 3) This is what I want malloc_usable_size() for: garbage collection. In
> order for a garbage collector to know when it should run, there needs to
> be some way of tracking how much memory is in use. By using
> dlsym(RTLD_NEXT, ...), I can intercept all malloc/calloc/realloc/free
> calls and track current memory usage. However, unless there is a way of
> getting the size of each allocation, I don't know how much to subtract
> from the count for realloc/free calls.
This is a good reason, and is what I hoped you had in mind when you
proposed it. :) I think that other than this type of code though (malloc
reimplementations or wrappers), nothing else should use this function. :)
> 4) Porting code from Linux. Case in point: Xara Xtreme, currently being
> ported by Vasil Dimov. At the moment, he has to use dlmalloc.
Does it contain a G/C of its own or some such?
> Following is what I've written for the malloc(3) man page:
> The malloc_usable_size() function returns the usable size of the
> allocation pointed to by ptr. The return value may be larger than the
> size that was requested during allocation. malloc_usable_size() is not
> intended as a mechanism for in-place realloc(), though it can be abused
> that way; rather it is primarily provided as a tool for introspection
> purposes. Any discrepancy between the requested allocation size and the
> size reported by malloc_usable_size() should not be depended on, since
> such behavior is entirely implementation-dependent.
I would word it stronger: "malloc_usable_size() should not be used as a
mechanism for in-place realloc(). It is provided solely as a tool for
John Baldwin <jhb at FreeBSD.org> <>< http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve" = http://www.FreeBSD.org
More information about the freebsd-arch