How to detect "real" programmers

April 15, 2022
New research suggests simple questions for basic technical screening.

Are you trying to hire developers? How do you know if any particular applicant actually knows how to program? Even at a basic level?

A new research paper entitled Do you really code? Designing and Evaluating Screening Questions for Online Surveys with Programmers looked at 16 sample questions to identify which ones are the strongest predictor of “real programming” skills, and the results are interesting. 6 of the 16 questions provided a strong signal that the subject was a “real” programmer. One that I found surprising was:

Which of these values would be the most fitting for a Boolean?

  • Small
  • I don’t know
  • Solid
  • Quadratic
  • Red
  • True

For a high-level overview of the study and results, take a look at this summary.

So what’s the practical implication?

I think these questions are informative any time we need non-technical people (or machines) to do the initial screening phase for development roles, and we want to filter out the truly unqualified.

Asking 3-6 multiple-choice questions is pretty straight forward, and although an experienced dev will find it a bit silly, it only takes a minute or two, so I would see this is as a good alternative to relying on a coding assingment for the simple question of “Is this person telling the truth that they know how to code?”

Related Content

Are your hiring filters working?

How often would an otherwise great candidate simply not continue the application process due to your filtering criteria?

Mission: Impossile — Finding a Good Analogy

Not every company wants to hire a super star; also niching down can make you a star in your niche.

Does Tom Cruise still audition?

Asking Cruise to audition is likely a way to _not_ get him into your film. The same can be true for take-home coding challenges.