Interviewing Through Pair Programming

June 8, 2012

The best interview I ever had was a pair-programming session with Luke Bayes and Ali Mills. I was put at ease by their demeanour and attitude. They are coders, and I'm a coder - all that was left was to work out if I was as good as they hoped I was, and whether we could stomach working with one another. As it happened, all of the above, but I had to decide between enterprise work and gaming work, and I took a different route. But the interview was fantastic.

Recruiting good developers is a difficult discipline. Most of the time you get the impression that developers conducting interviews haven't given the interview any thought until about 5 minutes before it starts. This is a mistake; recruiting good colleagues helps to solidify a team culture. Hiring the right candidate is important. Not hiring a wrong candidate is doubly important.

I have now been interviewer more than I have interviewed. In my experience, the most reliable approach for interviewing interactive developers has been to conduct the entire interview as a sort-of game, played through pair-programming. I have yet to recommend a candidate that I later regretted recommending as a result of this game, which is as good a testimony as an interview technique is likely to receive.

The Game

The game itself is trivially simple. Let's start with a trivially simple structure - for example a static circle in the middle of the screen - and start adding functionality in turns as a sort of free-association game.

If we're interviewing an interactive developer most familiar with AS3, I might start with something like this:

          package
          {
            import flash.display.Sprite;

            [SWF(width="800", height="600", backgroundColor="#FFFFFF")]
            public class Main extends Sprite
            {
              public function Main()
              {
                graphics.beginFill(0x1E90FF);
                graphics.drawCircle(400, 300, 50);
                graphics.endFill();
              }
            }
          }
      

Imagine you're an interviewee: We're going to take it in turns to add features to this code. You choose the first feature, and I'll drive implementing it. Then, I'll choose a feature and you drive it... and so on. We are pair-programming however; we need to work together all of the time. If there's something I do you dislike, you disagree with or you don't understand, you should say so immediately.

This simple structure leads to all sorts of interesting opportunities. When using the circle as a starting point, most often the developer chooses to have the ball fall and 'bounce', but some want to make it a speaker and hook it into a sound file, or to make some art by putting other circles next to it, or have it wobble, or chase the cursor, or rotate...

What Is An Interview For?

Principally, an interview is a mechanism to parse candidates for suitability for a job. A candidate is suitable for a job if they can do the work that will be expected of them. A candidate is not suitable for a job if they are disruptive or poorly suited to the existing team culture.

Can They Code?

Evaluating whether a programmer is good enough to do the work expected of them requires evaluating their ability to code. This is commonly achieved by posing technical problems on a whiteboard. Whiteboard problems happen away from a computer, so that there is no safety net of a test suite or just running the code to make sure it doesn't break. For this reason there is an impetus to solve the problem mentally and then to write down the results. This feeling is familiar: most students' experience of school is very similar; the interview becomes a test. A maxim in education is that testing students is a fantastic way of measuring how good students are at doing tests. I don't want to hire the best interviewee, I want to hire the best-fitting programmer.

Pair programming is extremely good at exposing an interviewee's programming style. The game established above offers the green-field of a project with no built-in dependencies or external concerns. Every developer loves to write code at the start of a project! That's why developers start so many projects and finish so few. At this point all potentialities are open. Interviewees may write solid, conservative code; they may write wildly creative prototype ideas; they might write badly designed spaghetti code... However the interviewer codes instinctively becomes apparent because the format is familiar and achievable.

What Are They Like To Work With?

After an hour of programming with someone you get to know them a little. If someone is thoughtful, open, aggressive or competitive, it is very difficult to sit side-by-side with someone discussing code with them without their personality expressing itself clearly. Programming is a coder's natural environment, and they tend to act naturally doing it.

How Creative Are They?

Creativity is an important part of interactive developers' jobs. Most of the time, requirements that come to interactive developers are poorly defined, or not thought through. An interactive developer needs to fill in the gaps or make the best out of obscure requirements. The reason for the open-choice task is to attempt to see the interviewee's creative faculties.

The game borrows from improvised comedy the idea of accepting your partner's idea and extrapolating. I tend to offer conservative extrapolations to the interviewee's ideas, allowing them to drive the direction of the feature-set more. In almost every interview that I have conducted in this way, interviewees drive the agenda to more complex features, to find interesting features and interesting problems within the context we have made for ourselves. Sometimes the game can break down, or we have to back-track. All of this should be embraced as part of the game. Allowing and encouraging interviewees to run with an idea is informative, as well as being fun for both parties.

Can They Solve Complex Problems?

Another developer who has used pair-programming for interviews Ivan Moore argues that pair-programming it is not a good technique for evaluating a developer's problem-solving skills.

However, he also points out that:

In typical enterprise IT projects, more problems are caused by over-complex solutions to simple problems, or by solving the wrong problem, than having problems that are unique and difficult and require a brand-new algorithm. Ivan Moore

I would contend that this is true not only for typical enterprise IT projects, but for almost all software. There exist very few software problems that can't be solved by a team of software engineers iteratively using simple, elegant solutions written in clean, easy-to-read and easy-to-change code.

Of course, there are arcane areas in software development and there are a handful of extreme cases where programmers are being hired because of their niche abilities. In this case, of course I would not advocate using the pair-programming game to evaluate their niche abilities. This is a 0.1% problem however; if you're reading this and think that you're a niche programmer, you're probably delusional.

But What About Problem Solving?

OK. Let's suppose you really need developers to solve some fiendishly compelx problems. Some companies ask quirky, brain-teaser questions. Most notably, Google's interviews ask famously inpenetrable questions to challenge interviewees' inventiveness.

You are shrunk to the height of a nickel and thrown into a blender. Your mass is reduced so that your density is the same as usual. The blades start moving in 60 seconds. What do you do? Google Interview question, from
How to Ace a Google Interview

This question is measuring a candidate's ability to make some mental leap of logic. A good leap of logic in this case would be to propose a physical leap out of the blender; if you're the same density then the ratio of strength to mass will have increased in your favour, so you can jump out of the blender.

Demonstrating such a leap of logic is a good positive indicator that an interviewee can think laterally, invent and innovate. It tests exactly those aspects that Moore argues are not well tested by the game. For this reason, it is worth considering asking such questions during an interview.

Conversely however, not demonstrating such a leap of logic is not a good negative indicator; an inventive problem-solver may fail to answer such a question successfully. Perhaps the question just didn't chime with them; perhaps they were too nervous.

The utility of these questions for Google is more obvious when you consider that Google have around 130 candidates for each job. A question that reliably filters inappropriate candidates but also filters many good candidates is not a problem in a scenario where there are more candidates than can be readily handled.

Most silicon valley companies have difficulty keeping their engineers, let alone elsewhere in the world. There are too few developers for the work available (worth reflecting on, if you're dissatisfied with your current working conditions). In a scenario with very few potentially appropriate candidates for a job, using questions like this may not be an appropriate strategy; there is no point in asking a question that nobody can answer!

Conclusions

I am not advocating pair-programming as an ideal interviewing technique, but it has served as a functional approach for me. Interviewing people for coding positions is complex, and many good techniques or ideas are available. Different styles will fit different cultures better, but it is important and instructive to reflect on how you interview and what you are looking for while you interview, to ensure that you maximise your chances of hiring the best candidates for your team.