Tough tests flunk good programmer job candidates

Even in today's economy, tech companies regularly complain that they can't find enough qualified candidates to fill software development positions. You might expect them to loosen their requirements and provide more training as a result, but actually the opposite seems to be true: Screening procedures are getting tighter. The latest trend is to subject interviewees to elaborate quizzes designed to assess their coding and problem-solving abilities.

The company best known for this is Google. Past applicants tell tales of a head-spinning battery of coding problems, riddles, and brain teasers, many of which seem only tangential to the task of software development. Other large companies have similar practices -- Facebook and Microsoft being two examples.

[ Speaking of quizzes, see if you can pass InfoWorld's programming IQ test, round 1, and programming IQ test, round 2. | Get software development news and insights from InfoWorld's Developer World newsletter. ]

Are these kinds of tests the right way to hire developers for your business? Maybe. You'll need to assess an applicant's skill in one way or another, but it's also possible to take the whole interview-testing concept too far. Here are a few thoughts to keep in mind when crafting your test questions, to avoid slamming the door on candidates unnecessarily.

1. Recognize that tests are artificial scenarios

When you administer a coding quiz or some other kind of test as part of the hiring process, you're putting candidates in a very strange position. You're usually asking them to solve some hypothetical problem based on sketchily defined requirements and parameters, and you're asking them to do so in an extremely limited period of time. They can't collaborate with anybody else, and they're not allowed access to any reference materials. Although this scenario is familiar to everybody from school, it's not one that crops up very often in the working world, which means it's not necessarily the best benchmark for how a candidate will perform on the job.

What's more, not everyone performs equally well at tests, and candidates are often under a lot of stress during job interviews in general. Don't assume you can gauge candidates' personality and working style based solely on their test performance.

2. Don't set the bar too high

When asked why they use tests as part of the hiring process, companies often say it's a way to find "the best and the brightest." But is that really what your vacancies call for?

True, some amount of software development involves crafting elegant new functional units and designing novel algorithms. But a good portion of it is devoted to considerably more menial tasks, such as debugging, refactoring, and code maintenance. Before you craft an interview exam that would put Bell Labs engineers to shame, ask yourself whether a candidate with a doctorate in computer science would really feel fulfilled working on the tasks your position actually requires.

Also, don't overlook the value of hiring candidates with less experience and training them to work in your company's style. Not everyone who takes an interview exam should have to score an A to pass.

3. Stick to the job at hand

Coding quiz problems come in all flavors, but don't include questions on your exam just because they sound clever. Broad, conceptual questions are fine, but if you get specific, make sure that what you're asking actually applies to the position on offer.

For example, if you're mostly a Java shop, it makes some sense to ask candidates to provide code samples in Java. But don't give candidates a problem in Lisp just to throw them a curveball. Likewise, unless you're working on embedded systems, it doesn't make much sense to grill candidates on which compiler options will produce the smallest object files. If the questions focus on trivia rather than your actual work environment, your test will have as much real-world value as a pub quiz, and you'll end up excluding candidates essentially for no reason.

4. Don't neglect the human element

Problem-solving and coding ability is important, but they're only part of a software developer's job. A successful candidate must also have the interpersonal and communications skills necessary to conduct the day-to-day business of meeting with stakeholders, gathering requirements, providing progress reports, and collaborating with other developers.

That's why you should put equal focus on ascertaining candidates' working styles as on their coding skills. Be honest about your organization's workflows, deadlines, and pressures. Ask candidates how they prefer to work and be managed, and encourage them to ask questions about your company and how it operates. Even a candidate who performs exceptionally on programming quizzes might be a bad fit for the position, so always counterbalance test results with less formal interviews.

5. Don't let HR be a barrier

We've all seen those ads for software development jobs that ask for seemingly impossible qualifications: "The candidate must have 15 years' professional experience in Microsoft .Net." Software development is a highly technical field, and positions often have complex requirements. Because of this, HR professionals can sometimes do a poor job of screening applications for programming positions -- for example, vetting candidates based on an alphabet soup of acronyms rather than their actual level of experience.

Even worse, many HR departments are now using automated systems as the first phase of the hiring process. Before they can submit a résumé, candidates are asked to answer a series of questions on an online form; for example, "How many years of experience do you have with .Net?" Answers that don't fit the predefined profile for a position can effectively blackball a candidate.

These kind of pass-or-fail quizzes are the worst kind of interview exams, and they should be discouraged. Instead, managers responsible for hiring developers should accept that they will need to take a more hands-on role in screening applicants, as time-consuming as that can be.

6. Employment is a two-way street

Finally, always remember that the goal of interviewing is to staff a position, not to put applicants through the wringer for testing's own sake. Candidates arriving for their first on-site interviews will have as many questions about your company as you have about them. The interview process should feel like a dialog, not a trial. Don't make it so one-sided and adversarial that it turns candidates off to working for you. Remember, if they have enough skill and experience to make them attractive to you, chances are your company isn't their only option.

This article, "Tough tests flunk good programmer job candidates," originally appeared at InfoWorld.com. Read more of Neil McAllister's Fatal Exception blog and follow the latest news in programming at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.