Allocating Too Much Heap

December 4, 2007 3:30 PM

java.lang.OutOfMemoryError: unable to create new native thread

This error means you have allocated too much heap, and you need to reduce your -Xmx setting. A similar symptom is this:

Caused by: java.lang.OutOfMemoryError
      at java.util.zip.ZipFile.open(Native Method)
      at java.util.zip.ZipFile.<init>(ZipFile.java:204)

The amount of memory that can be addressed by a process is dependent on the operating system that process is running under. The absolute maximum for 32-bit applications is 4GB, as this is the most memory you can address using a 32-bit pointer. On Windows systems the theoretical limit is 2GB, but don't try allocating even that much. There are various hacky ways to address more memory, but that's another story.

Say someone starts up a JVM with an -Xmx setting that reserves 3.5GB for the application heap. That means there is at most 512MB of addressable memory left outside the heap for that process. Even before the application starts up, some of that memory will be used to load the JVM itself: link in shared libraries, mmap rt.jar and so on. This leaves precious little memory available for anything else.

Starting a new thread (the first error) requires the JVM allocate space outside the heap for that thread's stack. Opening a zip file (at least before Java 1.6) memory-maps the entire file, leading to the perverse circumstance that the maximum size zip file you can open before getting an OutOfMemoryError is the inverse of the amount of heap you've allocated.

And people wonder why I'm so picky about the garbage-collection question in tech interviews...

Previously: Absolutely Fabulous

Next: Practical Uses for lolcats