Are you trying to hire developers? How do you know if any particular applicant actually knows how to program? Even at a basic level?

A new research paper entitled Do you really code? Designing and Evaluating Screening Questions for Online Surveys with Programmers looked at 16 sample questions to identify which ones are the strongest predictor of “real programming” skills, and the results are interesting. 6 of the 16 questions provided a strong signal that the subject was a “real” programmer. One that I found surprising was:

Which of these values would be the most fitting for a Boolean?

• Small
• I don’t know
• Solid
• Quadratic
• Red
• True

For a high-level overview of the study and results, take a look at this summary.

So what’s the practical implication?

I think these questions are informative any time we need non-technical people (or machines) to do the initial screening phase for development roles, and we want to filter out the truly unqualified.

Asking 3-6 multiple-choice questions is pretty straight forward, and although an experienced dev will find it a bit silly, it only takes a minute or two, so I would see this is as a good alternative to relying on a coding assingment for the simple question of “Is this person telling the truth that they know how to code?”