New libc malloc patch

[Jason Evans]

On Nov 30, 2005, at 4:30 AM, Ulrich Spoerlein wrote:
> Daniel O'Connor wrote:
>> On Wed, 30 Nov 2005 21:48, Poul-Henning Kamp wrote:
>>> In message <20051130111017.GA67032 at galgenberg.net>, Ulrich  
>>> Spoerlein writes:
>>>> I just read that mmap() part and have to wonder: Is it possible to
>>>> introduce something like the guard pages that OpenBSD has  
>>>> implemented?
>>>> I'd love to try this out and see the dozens of applications that  
>>>> fail
>>>> due to off-by-one bugs.
>>>
>>> Guard-pages are very expensive and that is why I have not adopted
>>> OpenBSD's patch.
>
> Yes, of course it should be disabled as default, but if it could be
> implemented so you can switch at runtime or compile time (think
> INVARIANTS/WITNESS) *and* there is no penalty for the disabled case,
> that be nice.

In a previous version of the patch, I included compile-time support  
for redzones around allocations.  Kris Kennaway did a full ports tree  
build with redzones enabled, and several ports caused redzone  
corruption, but in every case it was due to writing one byte past the  
end of an allocation.  None of these were serious, since word  
alignment required that the "corrupted" byte be unused.  I suspect  
that we would catch very few serious errors, even if redzones were  
enabled by default.

Due to some unrelated performance issues, I later did a significant  
rework of the internal data structures, and decided to drop redzone  
support since the new data structures weren't as conducive to  
redzones.  Ultimately, I don't think we would have wanted to leave  
this feature enabled, even for CURRENT, because it required that all  
allocations be larger, thus bloating memory usage for all applications.

As a runtime-switchable feature, I think we still wouldn't want to  
leave it compiled in for production systems.  I spent a lot of time  
looking at valgrind (cachegrind tool) profiles, and found that even  
innocuous additional features such as the tracking of total allocated  
memory had significant negative impacts on performance.  The feature  
that I really didn't want to remove, that is also important to  
redzone support, was byte-exact tracking of allocation size.  The  
extra branches that would be required for runtime support of redzones  
probably wouldn't be worth the cost.

Jason