One thing I've been vaguely interested in over the last year or so (enough to pique my curiosity on occasion, but too busy to do anything serious about it) has been the nuts and bolts of memory management, and how it affects virtual machine-based OO apps of the kind I've been writing for the last five years.
Our operating systems are the result of decades of research into memory management, which is then totally ignored by the virtual machine. The virtual machine carves out a gigantic chunk of memory and manages it all itself, because the virtual machine's requirements for memory management are completely at odds with the assumptions of the OS. Which is why you have to fiddle around annoyingly with every Java application's memory allocation like it's some kind of 1990's-era Mac app; and why crippling the operating system's memory manager can sometimes make your Java appserver run faster.
Given the prevailing popularity of VM-based object-oriented systems these days, especially ones with huge memory requirements running on big servers, I sometimes wonder why there isn't more research like this going on: Sun research: An object-aware memory architecture.
I'm looking for more like the above. If you find any, post it in the comments and I'll be a happy fish.
Addendum: When I posted this article, something was bothering me. So I went into the office and re-read it, and thought "Ah. Does VM mean virtual machine or virtual memory?" Overlapping acronyms should be banned.