The last interview I did was over the phone, for a job halfway across the country. Now that was stressful. Doing a phone interview makes you realise just how important body-language is, especially that attitude in an interviewer that says ‘OK, you've talked enough now, finish what you were saying so I can ask another question.’ Over the phone, in the absence of such clues, the interview tends towards you talking until you've bored the interviewer to tears, or tailing off into embarrassing silences where both sides try to work out which is due to talk now.
Anyway, I was reading artima.com's article How to Interview a Programmer, and the following leaped out at me. Joshua Bloch and Bruce Eckel had both just suggested getting an interviewee to perform some kind of design or code snippet, which garnered the following responses:
Scott Meyers: I hate anything that asks me to design on the spot. That's asking to demonstrate a skill rarely required on the job in a high-stress environment, where it is difficult for a candidate to accurately prove their abilities. I think it's fundamentally an unfair thing to request of a candidate.
Matt Gerrans: I don't like when I'm asked to write a program that does X on a piece of paper. Don't ask the candidate to write a program on paper. That is a waste of time and sweat. People don't write software on paper, they do it with computers using auto-completion, macros, indexed API documentation, and context-sensitive help. They think about it, refactor it, and even rewrite it. If you want to see a person's work, ask them to write some small module or implement some interface before the interview and bring the code on a notebook PC or on hard copy. Then you can review it and discuss the design, coding style, and decisions that went into it. This will give you a much more realistic and useful assessment of a person's work and style.
import com.aol.*1
I'm not a bad actor. Or at least I have the potential to be not a bad actor. But the thing that kept me out of acting in university (well, aside from the fact my older brother was President of the dramatic society, and I didn't want to be seen as following him) was that I'm very bad at improv. In all the auditions I attended, one of the things that was asked of you involved improvisation, and when you put me on the spot like that, I dry up. I'm creative, but if you point at me and say “Create something! Now!”, chances are I'll not do it very well.
I'm a good programmer, and a pretty good designer. On all other counts, I'm told I interview quite well. That said, if you put me on the spot and say “write this code in the next five minutes”, I'll flounder around, and probably make a pig's breakfast of the whole thing. Burningbird recently had a similar problem: knowledge that would normally come effortlessly abandoned her during the interview.
When I'm presented with a design problem to solve, you won't find me sitting in a chair with a piece of paper. You'll find me wandering aimlessly around the office with a can of coke, or tossing my pen in the air and catching it. You'll see me doodling on a whiteboard. You'll see me wander out the front door with my iPod playing at maximum volume, and walking around the block. When I need to think creatively, I get my best ideas when my feet are moving, not when I'm sitting in front of a computer, or in a meeting being pressured to come up with an answer right now. I guess my approach quite closely resembles that of the Postmodern Programmers2
:To write this program, we first connected our computer to the Internet, downloaded some music from Napster, and then read our email. (You have to receive email to perform a workday [11]). We received 25 pieces of email of which 16 were advertisements for Internet pornography, administriva, or invitations to invest in Nigerian currency trades. After dealing with this email, we typed ‘calculate prime numbers’ into Google. This found several web sites regarding prime numbers, and some more pornography. After a while, we were interrupted, and so moved on to the prime number web sites.
This may sound like a wasteful way to do things, but I find that the way I work is well suited to my mind, and if I'm allowed the freedom to wander around, go for my walks around the block, and generally look like I'm not working, I get things done much faster in the long-run. I'm reminded of a Dilbert cartoon I can't find a link to, where Dilbert was asked to fill in a timesheet: “So, I should fill in all the time I spend in pointless meetings, but not the time I spend in the shower thinking about circuit design?”
Maybe this leaves me with a certain bias against the “ask the programmer to solve a problem in the interview” approach. Or maybe it's just that the approach is an interviewing technique, not an interviewee technique. The interviewer is looking for someone who fulfils the requirements of the job, not for everyone who fulfils those requirements. If a few people who have the job skills, but not the interview skills, slip through the cracks, so long as somebody else shows up who is qualified, and who can jump through the interviewer's hoops, that's only really a problem for the person doing the slipping.
1 “Me too!”
2 Well, aside from the whole pornography thing. The closest I've come to viewing pr0n at work is when some inconsiderate person on my livejournal friends list has replaced their userpic with something obscene.
[11] throw new DanglingFootnoteException(11);
But, speaking as an employer, we have a problem. We can't trust people's resumes, and unless you ask technical questions, it's difficult to judge people's skills. At a company I worked for, we had someone in who was a "C++ guru" (his own words). He made three mistakes on the first page of the programming test. Perhaps he just did't test well, but if you don't know that a function calling itself is "recursion", then we really don't want to hire you.
We don't use the tests to decide between people, in general, but merely to weed out the people who obviously don't understand the languages they claim to know, and pointers if one of the languages is C++. We're a little interested in how they solve the problems we put to them, but that's not the main point of the test. Oh, yeah, and commenting the code gets you bonus marks.
The problem with the take-home exam is that we don't know where they got the code. It's all too easy to copy someone else's solution. I do wish that we could know which people are good without testing them (which demonstrably doesn't work), but damned if I can think of a better way.
I understand. It's very difficult to prove your programming knowledge in an interview. Perhaps that's one place that a weblog can come in handy: there's now a lot of public evidence on http://fishbowl.pastiche.org/nerdstories.html describing how I think about programming and technology. Not that anyone with a huge pile of resumes to get through would read that page unless I also did well in an interview...
Like I said in the post, the interviewer's point of view is different. There are lots of programmers out there, and the task is to find a way of discovering one of them that'll do the job. Having always been the interviewee, I'm just biased the other way.
Of course, a good interviewer relies on the intersection of a lot of techniques, not just a single questionnaire or task. I know someone who works in HR (and if I hear the phrase "core competencies" again, I may scream), and she has a lot of really interesting material on good things to ask in interviews, and what to look for in the answers. (Her favourite seems to be: "What's a problem you encountered in a previous job, and how did you overcome it?")
Once again, I guess the best thing to do is use a range of techniques. Descriptions of programming interviews always seem to focus on the "practical test", with sites such as Joel's focusing perhaps a little too much on the trick programming question of the day.
My favourite set of questions for interviewees is starts by asking them to describe a large system that they've worked on before and getting them to draw the system on a white board.
It's always interesting to see what kind of spin the interviewee puts on the question - do they describe the system from a business point of view (data flows between user departments, reports, and letters to customers) as a technical architect (hardware, IP networks, J2EE), as a bunch of classes (domain or framework), or as something else altogether? Are they confident talking at their chosen level? What other ways can they talk about it?
After a little bit of probing, and getting about 10 boxes on the whiteboard, I then ask what they would do differently if it were up to them to redesign the system from scratch. The intent of the question is to find out how they evaluate design. Very quickly you can tell if someone is a thinker, a leader, a starter or a finisher, experienced or new or just along for the ride.
You should have put a link to the rest of your comments, Alan. They're all on-topic, I think, and worth mentioning.
http://www.cardboard.nu/archives/000024.html
In the entry, you said:
> Not quite the same as a coding test, but valuable.
I agree that it's valuable if you're trying to hire someone for their design skills, but we already had three people (out of seven) with good design skills, and we were looking for more junior people who knew the syntax, and whom we could teach our design aesthetic to.
As an interviewee, I was completely opposed to programming tests, but as an interviewer I can see where they're useful, and so when I take them these days I write them as if I was writing production code. Simple to understand and fully commented. It's probably also worth mentioning that we've hired one of the senior guys here even though he completely bombed the test, because he had my very strong recommendation. His skills are a perfect mismatch with the programming test, but I'ld worked with him before, and knew that he would be great for the company. And he is.
i hear what you are saying about test being helpful and all, but i had to take a test for a web dev job and of the 10 questions there was only one web dev question. the rest had to do with C#, VB, and other high level stuff with a few diffrent answers... i almost wanted to get up and leave. it was extreamly stressfull. but i guess in this market companies can do what ever and still get 100's of entries... i just think there is a very good chance they will miss out on someone who would be a perfect fit due to how they answered some test.