Hi Everyone!
I'm sure white-boarding and/or online coding challenges during interviews is something that everyone here must have faced sometime. Honestly speaking, I've always been skeptical of them and below is a reason why:
As an applicant, the result of a coding challenge with a timeline (like HackerRank) OR a take-home over a weekend is never accurate as most questions in it are purely academic such as calculating space-time complexities and also the questions tend to be theoretical.
Scoring high or low in them will hardly give you an idea of what a prospective candidate is capable of because, given an enterprise environment problem statement - a solution to such questions can be found in conjunction with Google/StackOverflow/Official Docs anyway.
I believe a person's will to find answers when they don't know and solve a problem with a genuine understanding of impact is more vital and therefore I'd rather prefer interacting with someone (or multiple) in a personal interview (phone, Skype or on-site) to solve a problem that mocks a real world scenario. This will help both sides to evaluate each other as to whether they'd be a good match (technically/behavioral) or not.
So I've the following questions for the Dev.to community:
- What’s your experience with coding challenges, technical tests, tasks? (e.g. codility, hackerrank or various custom “homeworks”) as an applicant and as an evaluator as well?
- Do you think it properly evaluates if an applicant is a fit for the job?
- What is a viable alternative that works for you as a hiring manager?
Top comments (16)
Let me approach this from the interviewer/employer's point of view.
You advertise a web programming position and get dozens or hundreds of applicants. Everyone's claiming to be just awesome in their resumes. But when you start interviewing people, many of them aren't remotely qualified. We don't have time to interview everybody to such a degree that we can be absolutely sure we have an accurate picture of their capabilities--if we had that much free time we wouldn't need to hire anybody.
So we need to streamline the process somehow. But where do we set the bar? There are two kinds of errors we can make:
This is the situation potential employers find themselves in. And the rational thing to do is filter out the weak candidates as quickly and cheaply as we can, even if it means potentially culling an excellent candidate. Hence, little tests and coding exams. Are they perfect? Hell no. But they do thin the list of candidates substantially.
From there, we try to gain the most information about the remaining candidates for the least cost. So we start with short phone interviews. Then in-person interviews. Perhaps followed by another in-person interview. And finally a small paid project.
If it helps, we focus on asking questions that are applicable to the position--we don't ask candidates to do puzzles or big O notation.
I don't know of a better way to do it but I can totally appreciate how challenging and tedious this process can be for applicants who are having trouble getting hired.
If I can offer a tip to people trying to get a job: be well prepared for the interview. It's shocking how bad people do in interviews. Yes, people get nervous. But even taking that into account, most people are shockingly unprepared. Get a friend to ask you questions. Do little coding challenges. Do mock interviews with a friend if you can't get your nerves under control. Prepare and rehearse answers to likely questions beforehand. Give the interviewer a reason to advance you to the next level.
The whole concept of thinning down the list using far from perfect online challenges seems odd to me (unless you're doing university campus placements wherein there tend to be 200 folks applying for 10-20 roles. For regular hirings, the ratio is mostly lesser)
Your procedure sounds appropriate but I'd rather start off with a short phone interview (or Skype) so that both sides get clarity on what they're looking for.
Yes I know, a short call is not enough to determine that but it's analogous to speed dating. By the end of that call, both sides can conclude (unfortunately, this heavily leans on the interviewer side but however I've said no to a fair share of companies after such a call myself) whether there should be a next round OR not.
It's good to know that your focus is on applicable questions rather than theoretical ones. In the rare few that I've partaken, I usually asked both technical / behavioral questions relevant to the role and to extract the interviewee's idea of working.
Lastly, please note that the preparations that you've mentioned like solving mock challenges, interviews & rehearsal are helpful but the real interview is a different ball-game altogether since little things can derail it.
I think your last sentence should hold true for the interviewer as well. They need to equally convince a prospective candidate as to why working with them is a good thing.
Yes, I agree with most of your points.
I think the way we do hiring is terrible (and not just in programming). Lots of surveys suggest that if you regret only half of your hires after a year you're about average. That's depressing to think about.
I think candidates need to be prepared for the interview even if there's a chance your interviewer will be unfair for some reason. Being prepared won't guarantee you an offer but being unprepared will almost certainly guarantee that you won't get one.
We tried doing phone interviews first and it didn't work for us. I don't know if "web developer" positions attract particularly unqualified candidates or it is related to the shortage of developers in my city but it didn't work.
Our current practice is to scan the resumes and as long as they are remotely qualified we invite the candidates to do a quick test online. All they have to do is write a very simple function to assess whether they can program at all. They are given 15 minutes to do what should take 5 not more than minutes. Less than 30% of the candidates succeed. They are cut if they don't pass.
I know that sounds harsh but we see a lot of people who say all the right things completely fail when we give them an absolutely beginner programming assignment.
Preparation undeniably counts but with no offense intended to anyone, I'll quote a proverb that I felt is apt here
While your approach is practical given the shortage of time, I think it's time we (as an industry) collectively revamp the interview process so that people are judged based on what they're really capable of rather than textbook assignments that hold no real world value whatsoever.
Agreed. How do we do it? What would a better process look like?
From my limited know-how, I'd ideally ensure the candidate undergoes 2 or 3 rounds of interviews with people of the designated team and if they clear it, a panel round with all of them so that it's easy to get a fair overview of the candidate while eliminating individual bias.
The idea here would be to ask (technical concepts - general and particularly role based along with behavioral) questions.
Yes, shortage of time is a major constraint here and so is proper execution of this but even if done with 70% efficacy, I believe the new hires will be of much better quality.
Thanks for sharing this, Vinay. I think the devil is in the details here. Executed well, this would probably work. Executed poorly, and it could be a time waster.
I definitely agree with the team being involved in the interview (and perhaps even giving each member a veto).
Glad to help Blaine! Do share your experiences if ever you put it into practice or even add something more so that the rest of us will find it useful too.
My experience is that if you skip coding challenges, about half of the candidates that otherwise seem fit for the job will not be able to program effectively. Unfortunately most of these will also not be able to learn it within an acceptable time span, even if they feel motivated.
The point of a coding challenge is not to find out if the candidate solves the task in exactly the way the interviewer expects. There are often many different approaches and solutions.
The point is not even to find out if the canditate can solve the task at all.
The point is to give the candidate a chance to show how she approaches a programming task. The solution itself is not important.
While you mentioned some valid points but I can tell you that candidates who ace those programming challenges aren't necessarily a better fit than ones who fare averagely.
Because most people are under duress to perform in an interview so they might blank out there but if evaluated humanely, they're often great problem-solvers and can handle crisis situations equally well. For all one can prepare, even the slightest thing can derail an interview.
Also, I disagree with your statement that coding challenges are solely about approaches and not about solutions since I've had multiple experiences wherein I got feedback my scores were too low inspite of me writing working programs that were logically and syntactically correct.
We want them to be so but often they're taken as the only yardstick to judge people.
I think you misunderstood some of my points because I didn't make them clear enough:
Most coding challenges are done wrong, not the way I described.
Coding challenges should never be the only criterion for selecting an employee.
Stress and anxiety have to be taken into account.
That's very correct but I'm doubtful if companies will shed the approach of NOT counting those code challenges as the only criteria along with accounting for stress & anxiety. The ones that actually care, will have a ready pool of diverse candidates almost all the time.
As a interviee: I get nervous when doing timed tests with people watching me. Every mistake suddenly seems magnified hundred-fold. It's definitely not representative of my true capability - like you said, the vast majority of the time you are in a environment where you can reference docs and think at your own pace.
As a interviewer: I don't have much experience here - I've only interviewed one person. But it was a interesting experience being on the other side. You give out a relatively simple question to a person with decades of experience and they can't even being to solve it. But we have all been there - I'm not only looking for a solution (although that would be nice), I'm looking for how they approach the problem. In this instance they got angry, complained about it being a unfair question, and left without shaking my boss's hand. No go.
There's no golden bullet for properly evaluating a candidate - it depends on how you use it. Ideally you would ask questions directly relevant to the job at hand. If the questions are tricky theoretical ones then you would ask just to see the thought process, not necessarily for a solution.
I haven't tried it, but some companies invite you to their office and pay you to work a full/half day like a real employee. That would be nice as a employee and as a employer it would give you a much better idea of how the employee performs than theoretical CS questions.
Been there, done that with the getting nervous part myself too.
While the experience you had as an interviewer seems to be unfortunate but in retrospect, if a person with decades of expertise doesn't have basic etiquette right - I'd be doubtful of having someone like that onboard if I were an interviewer myself.
Coming to the whole working as a real employee for a half/full day concept, it's interesting but not exactly feasible for everyone since there are so many variables to be factored.
From my knowledge so far, I'd ideally go with a 2-3 rounds of interviews with people of the designated team and then a panel round with all of them so that it's easy to get a fair overview of the candidate.
Since my last post I've done this and it's awesome. Not practical for larger companies since there is a lot of setup involved, yes, but for small companies I would reccomend it. For me it was the difference between accepting / not accepting the offer.
Glad to know this played a key role in helping you decide but then again it's easier said than done regardless of the company's size.