Jim Nance did some testing on this a while back; the URL for info about that is above. See also <http://www.hoard.org/>. Patrick, forgive me for assigning this to you; if you won't have time to investigate further, please let me know, and I'll try and find another owner.

I'm happy to own this issue.

Jim, did you have some profiling information posted anywhere?

Yes, take a look at: www.inttek.net/~jlnance/mozilla/hoard

I just noticed that a couple of the Netscape server guys (David Boreham <dboreham@netscape.com> and Chuck Lever <chuckl@netscape.com>) are giving a paper at this summer's Usenix entitled "Malloc() performance in a Multithreaded Linux Environment." I've sent them mail and asked if they'd care to comment...

The work we did concentrated on SMP server scenarios. You may find little or no improvement on uniprocessor machines and with applicatons which have little concurrency.

The comments that follow are from Chuck Lever, and he asked me to post them here: I've spoken with Emery Berger, one of the authors of Hoard. The Hoard allocator is designed for large, long-running, multithreaded applications where memory objects are often freed by threads that didn't allocate them. If you believe Mozilla is this kind of application, then Hoard may be appropriate for controlling unbounded memory consumption caused by objects orphaned by the heap allocator. It is also a good choice if you believe Mozilla performance is being slowed by contention for the heap. I don't have much experience with OO languages, but I can imagine that they hit the heap very hard, and probably do some things that might be considered pathological to certain heap allocators. I'm also not familiar enough with Hoard to say whether it has decent debugging facilities (like poisoning and bounds checking). I know Linux glibc malloc() has some nice features that may be useful for tracking down certain allocation problems. Finally, I know that the NSPR team has been aware of Hoard for a while (that's how I found out about it).

Disclaimer: I don't know anything about memory allocators. I heard about hoard in comp.programming.threads and have exchanged email with the author, Emery Berger. I am very impressed by the scalability of Hoard on multiprocessors. Regarding whether mozilla should use hoard: 1. This decision must be made on a platform by platform basis. It can be a hassle to override malloc and free on some operating systems. 2. Hoard is targeted at multithreaded applications running on *multiprocessors*. Mozilla, being a client program, doesn't seem to be the kind of applications hoard is designed for. However, jlnance's experiments show that hoard does improve mozilla's performance on Linux. This issue could also be addressed by minimizing the number of malloc/free or new/delete calls made by mozilla. 3. I think hoard has a lot of potentials (the qualifications of the author and the multiprocessor scalability). If mozilla becomes one of hoard's earliest and biggest adopters, it will be a big help to its wider use and incorporation into libc. 4. It is risky to take a brand new implementation of a fundamental library at such a late stage of mozilla's schedule.

Using hoard should require no changes to the mozilla source code itself. You just either link in hoard or you dont. It would be nice add a --enable-hoard option to configure so that people could test it if they wanted to. It would be easier to do this if hoard were checked into the mozilla CVS tree, but Its not clear to me that that is a good idea. Any comments?

Adding a --enable-hoard configure option is a good idea. I think hoard should not be checked into the mozilla CVS repository (unless the author wants to use mozilla as his primary source repository), because hoard is not a required component of mozilla.

Instead of checking hoard into the mozilla tree, isn't the right thing to do to add a --with-hoard=/path/to/hoard?

just fyi, I have a --enable-hoard param in my local tree... viewer works ok but mozilla crashes on startup with it.

Hi, Hoard author here. Please send me mail to let me know what version of Hoard is causing problems, what platform, yada yada. Thanks, -- Emery (emery@cs.utexas.edu)

-> M18

someone else should probably step into the QA Contact field on this one.

Wade Hennessey tells me that the Linux memory allocator's overhead due to thread synchronization is pretty low already. How much would this work save us? I don't believe we can justify this right now, but we should study it in the future.

A new version of hoard was just released, and the changelog says: "This version features a major algorithmic change (essentially, a variant of thread-local allocation buffers) that dramatically boosts performance by avoiding atomic operations."

pav and vlad are thinking of trying out tcmalloc, nedmalloc, mtmalloc, and hoard. They might start by testing heap fragmentation behavior by replaying Firefox's malloc/free patterns, since getting Firefox to run with a different malloc might not be trivial.

i would like to do this

Hoard was among the options pav evaluated at in 2007. I'm not sure if he published detailed results, but we settled on jemalloc for Firefox. http://blog.pavlov.net/2007/11/21/malloc-replacements/