The Terrible Technical Interview

Traditional technical interviews are terrible for everyone. They’re a bad way for companies to evaluate candidates. They’re a bad way for candidates to evaluate companies. They waste time and generate stress on both sides. Almost everyone, if pressed, will admit this. And yet they persist.

I humbly suggest that it is time for engineers who have the luxury of choice to start to flatly refuse to participate in them.

Don’t panic. I have a better alternative. Read on.

In the last month Danny Crichton has written a couple of excellent posts about technical interviews: you should read them, but let me just cite some highlights:

Few professions seem so openly hostile to their current members as software engineering … we expect people to do live engineering on a white board under stressful interview conditions because, well, because that is what we have always done … In a time of engineer austerity, we simply can’t afford to throw away so much talent.

He in turn was inspired by Thomas Ptacek:

The software developer job interview doesn’t work. Companies should stop relying on them. The savviest teams will outcompete their peers by devising alternative hiring schemes.

Indeed. And, anecdotally, I do have the impression that things are finally changing. More companies are asking candidates to do test projects rather than whiteboard interviews. Others are becoming less fanatical about eliminating false positives at the interview stage (but more ruthless about firing them after a couple of months.) It helped that Google’s head of HR admitted, a few years ago: “Brainteasers are a complete waste of time” and “test scores are worthless.”

https://twitter.com/sabreuse/status/578719749976301568

And yet. I may have aspirationally written “The Technical Interview Is Dead” a couple of years ago, but it ain’t so. Not yet, at least. It’s dying, but far too slowly.

We haters need to concede a point: there are reasons, some of them semi-valid, that companies persist with whiteboard-style interviews, even though they know they’re far from perfect. These include:

• Interviews are a relatively measurable and repeatable process, from a company’s point of view. Everyone understands how they work. Just come up with a few questions, and a few criteria for measuring the answers, and in a pinch, (almost) any technical employee can conduct an interview.
• Companies want to filter out obviously inappropriate candidates early, and it’s hard to fight the feeling that while you’re at it, you might as well ask them just one or two slightly more technical questions … which feature-creeps into a full-on traditional interview in a hurry.
• There is such a thing as talent, and you do want to filter out people without very much of it. Traditional technical interviews are perceived as more prone to false negatives than false positives. Historically, a false positive has been perceived as the disaster scenario; hiring one bad engineer was viewed as worse failing to hire two good ones. But good engineers are so scarce these days, that no longer applies.
• Assigning a test project — the current alternative to technical interviews — is still, at best, imperfect. It’s actually quite difficult to come up with real bite-size projects that are both meaningful and will only occupy a day or few of a candidate’s time. Meanwhile, candidates want to be paid for that time, and/or protest that they already have a job and can’t expend that much effort on a speculative project, while companies are concerned that projects might be plagiarized or even outsourced.
• This is why personal references and recommendations remain everyone’s favorite hiring technique…
• …which in turn is a major reason why the tech industry’s diversity numbers are so disastrous.

So: if we were to find a reliable replacement for the traditional technical interview, it would be good for companies, it would be good for candidates, and it would help to increase the numbers of underrepresented groups who are currently implicitly overlooked by the recommendation process. That’s a pretty huge win-win-win for everyoned.

I would be remiss to not mention that there are many startups trying to do all this. But I have a different idea. One which puts more of an onus on candidates … but in a good way, I think. I propose that:

• It is time for engineers–especially excellent engineers for whom demand is high–to start to flatly refuse to do whiteboard interviews.
• Yes, really. Nothing will force companies to move on to better techniques faster than losing appealing candidates before they even get to interview them.
• But. This refusal must be coupled with a counteroffer: replacing the interview with a discussion / walkthrough of a side project the candidate has built themselves, in their own time.

Here’s the process I have in mind:

1. The interviewer takes 30-60 minutes to familiarize themself with the candidate’s project
2. They then spend an hour or two discussing the project, the architectural and implementation decisions the candidate made, alternatives they could have chosen, features they’d like to add, the structure and line-by-line quality of the code, environment and configuration issues, etc.
3. Those general subjects of discussion are formalized so that they can be repeated across interviews, candidates and interviewers can be compared, and results can be measured. Do they use global variables? Does the code follow an MVC or other architectural pattern? Are methods reasonably structured and encapsulated? Does the user interface indicate an understanding of the problems of usability? Is there test code? Etcetera. Half a general code review; half “how much does the candidate seem to really understand this code they (claim to) have written”?
4. Then the interviewer has the candidate add a minor new feature to their project, in real time.
5. At this point the interviewer should be fully confident (or fully skeptical) whether this project is well-constructed, and whether the candidate actually built it themselves.
6. If the former, then go ahead and, at an agreed time, have the candidate branch the company’s predefined test project — maybe a single perennial project, maybe a new one every few months. (Building a new one is a good project for recent hires.) Then have them submit a pull request for a new feature, one that should take about 4-8 hours of work. A test project, of sorts, but quite a small one, just to serve as a sanity check and ensure that the candidate can work with reasonable speed.
7. Have a different interviewer evaluate that pull request, so that you have multiple perspectives on the candidate.
8. Or, alternately — and arguably more efficiently — have the candidate pair-program a smaller feature with a different interviewer for an hour or two.
9. Then, decide whether to hire.

Voila: a replacement for the technical interview, one with no whiteboard coding, no gotcha questions, no demands for intimate knowledge of the implementation details of algorithms the candidate will never have to write again. I don’t pretend that it’s a perfect final solution for all and sundry — but I do believe that for most companies still stuck doing whiteboard interviews, it or something like it would be a viable and far better alternative to the status quo.

Now, this does require one huge prerequisite: every candidate must have a side project that they wrote, all by themselves, to serve as their calling card.

I don’t think that’s unreasonable. In fact, I think you can very happily filter out anyone who doesn’t have such a calling card. (And lest I be accused of talking the talk without walking the walk: I am very happily employed as a full-time software engineer; I travel a lot, and I write books, along with this here weekly TechCrunch column; and I still find the time to work on my own software side projects. Here’s my latest, open-sourced.)

Four years ago, when I first started ranting here about the ineffective counterproductivity of the traditional software interview, I wrote: “Don’t interview anyone who hasn’t accomplished anything. Ever. Certificates and degrees are not accomplishments; I mean real-world projects with real-world users. There is no excuse for software developers who don’t have a site, app, or service they can point to and say, “I did this, all by myself!” in a world where Google App Engine and Amazon Web Services have free service tiers, and it costs all of $25 to register as an Android developer and publish an app on the Android Market.”

I think that’s even more true today. But the flip side is, if you do have an accomplishment, a pet project to point to, then you shouldn’t have to jump through the meaningless hoop of a whiteboard coding interview. You’re better than that. We all are.