2. Don't set the bar too high
When asked why they use tests as part of the hiring process, companies often say it's a way to find "the best and the brightest." But is that really what your vacancies call for?
True, some amount of software development involves crafting elegant new functional units and designing novel algorithms. But a good portion of it is devoted to considerably more menial tasks, such as debugging, refactoring, and code maintenance. Before you craft an interview exam that would put Bell Labs engineers to shame, ask yourself whether a candidate with a doctorate in computer science would really feel fulfilled working on the tasks your position actually requires.
Also, don't overlook the value of hiring candidates with less experience and training them to work in your company's style. Not everyone who takes an interview exam should have to score an A to pass.
3. Stick to the job at hand
Coding quiz problems come in all flavors, but don't include questions on your exam just because they sound clever. Broad, conceptual questions are fine, but if you get specific, make sure that what you're asking actually applies to the position on offer.
For example, if you're mostly a Java shop, it makes some sense to ask candidates to provide code samples in Java. But don't give candidates a problem in Lisp just to throw them a curveball. Likewise, unless you're working on embedded systems, it doesn't make much sense to grill candidates on which compiler options will produce the smallest object files. If the questions focus on trivia rather than your actual work environment, your test will have as much real-world value as a pub quiz, and you'll end up excluding candidates essentially for no reason.
4. Don't neglect the human element
Problem-solving and coding ability is important, but they're only part of a software developer's job. A successful candidate must also have the interpersonal and communications skills necessary to conduct the day-to-day business of meeting with stakeholders, gathering requirements, providing progress reports, and collaborating with other developers.
That's why you should put equal focus on ascertaining candidates' working styles as on their coding skills. Be honest about your organization's workflows, deadlines, and pressures. Ask candidates how they prefer to work and be managed, and encourage them to ask questions about your company and how it operates. Even a candidate who performs exceptionally on programming quizzes might be a bad fit for the position, so always counterbalance test results with less formal interviews.