Closures and Java: a Tutorial

May 16, 2003 6:32 PM

What is a closure?

A closure is an anonymous function that ‘closes’ over its surrounding scope. Thus when the function defined by the closure is executed, it has access to all the local variables that were in scope when it was created.

Closures originated in Lisp, and have made appearences in a number of languages since, but for the purpose of this post, I shall use Ruby for my examples. Ruby was designed to use closures pervasively, and a number of its other design decisions reflect this. Note that the example I use here is a little artificial (Ruby's IO already has a grep method, it is just used a different way)

What are Closures Useful For?

The two most common examples of uses for closures lie in iterating over lists, and in encapsulating operations that must be set up, and then cleaned up after. Here's an example that does both:

IO.foreach("foo.txt") do |line| 
	if (line =~ /total: (\d+)/)
		puts $1;
	end
end

This code searches a file, and prints the matches. The IO.foreach takes care of opening the file, delivering each line to our closure, then closing the file when we're done.

We can do better than this, though. If we find that searching a file for regular expression matches, then operating on the result is something we do often, we can create a new method that encapsulates that more detailed operation:

class File
	def File.grep(fileName, pattern)
		IO.foreach(fileName) do |line|
			if md = pattern.match(line)
				yield md;
			end
		end
	end
end
	
File.grep("foo.txt", /total: (\d+)/) { |md| puts md[1]; }

Here, md contains a MatchData object that describes the regular expression match, and the yield statement passes that match data into the closure. Now our file search takes a single line.

The advantage of closures is that they allow you to add new control structures to your language. Java 1.5 is looking to introduce a foreach construct to iterate over lists. With closures, such language constructs become unnecessary (Ruby has only very primitive native looping constructs) because you can define your own. Similarly, closures allow you to dispense with boiler-plate wrapping code such as we see everywhere with file or database manipulation in Java.

Closures are also great for implementing the Command pattern. Command implementations that use objects must explicitly have their state set up for them, closures can just close over whatever state is around when they are created.

Blocks in Java

You can approximate the functionality of closures in Java using anonymous inner classes. There are two problems with this, though. Firstly, anonymous inner classes are unnecessarily verbose. Closures are supposed to be a short-cut, and anonymous inner-classes are anything but. This can be got around by adding some syntactic sugar to the language of course, but the use of classes is still quite heavy-weight. (Each anonymous inner class is an additional compilation unit, for example)

The bigger problem is that anonymous inner classes in Java don't really close over their surrounding scope—they cheat. The best way to demonstrate this by example is this:

i = 1;
1.upto(100) { |num| i *= num; }
puts i;

The above code prints out the factorial of 100. As you can see, the variable i is modified inside the closure, but because the closure is executed in the surrounding scope, its value is changed afterwards. This is unlike the behaviour of a regular function call, where variable values in surrounding scopes are not changed.

If you tried to write the equivalent code in Java, it wouldn't compile. Java doesn't really close over the surrounding scope, it copies it. To hide this implementation detail, any variable referenced inside the inner class must be declared final outside it: which is fine if your inner class is manipulating a mutable reference type like a list, but breaks if you want to work with immutable reference types, or value types. (i.e. Strings and numbers)

This is why you're unlikely to see closures in Java, sadly. To implement them properly would involve making changes to some pretty fundamental parts of the JVM. It's a pity, though, because they're damn useful.

Random Links:

1 TrackBacks

Listed below are links to blogs that reference this entry: Closures and Java: a Tutorial.

TrackBack URL for this entry: http://fishbowl.pastiche.org/mt-tb.cgi/236

Charles Miller wrote a nice article: The Fishbowl: Closures and Java: a Tutorial. He talks about using inner classes to somewhat approximate closures. He also says that "To implement them properly would involve making changes to some pretty fundamental... Read More

6 Comments

Nice article. Cleared up niggling questions for me.

I find it very hard to fit the full concepts of closures into my largely Java-based mind. The whole manipulating of the outside scope thing kinda wierds me out.

"This is why you're unlikely to see closures in Java, sadly. To implement them properly would involve making changes to some pretty fundamental parts of the JVM."

Not so. Giving inner classes the full power of closures could be implemented in the compiler without changing the JVM at all. Inner classes are already able to both read and write fields from their enclosing class. The only thing they aren't able to write is locals in an enclosing method. But this can be worked around by replacing locals to which the inner class needs write access with single-element arrays of the same type. Since locals are not visible in the object model, the compiler could, without breaking compatability in any way, make this transformation transparently:

1. In the declaration

2. At each point in the enclosing method where the local is used.

3. At each point in the inner class whether the local is used.

Once this is done, inner classes will be able to modify locals just like regular closures.

Of course, this adds some overhead, but only for code that can't be compiled at all today (at least without doing the transformation manually, which has the same cost) and much of it could perhaps be optmized away by a smart JIT.

Very short, very useful discussion.

Is anyone aware of plans to incorporate closures into Java (e.g., into any compilers)?

If you were to try and cobble closures into a Java development environment might that be accomplished by interfacing to an external (e.g., Ruby)? Or is this too ugly?

IDEA even automates the conversion to single-element arrays where necessary when using it's 'surround with Runnable' intention. Awesome.

Jim, its quite a while since you're post, but for the record we implemented true closures (like Ruby 2.0) in Groovy ontop of the JVM compiling straight to normal bytecode. Its very clean & lightweight. It would have been pretty easy to add true closure support to javac - providing you support autoboxing.

True closures are possible in Java, including modifying local variables. You just need to do a little more work. I posted an article to my weblog at http://www.kataba.com/weblog/Entry.asp?blog=Chris&entry=2004-08-31-1-closure_idiom which describes how to do this.

Previously: JBoss vs JDJ Celebrity Deathmatch

Next: Technical Accuracy in a Movie?