I hear this complaint a lot, but I kind of don't get it. I just don't know any way to make job interviews fair for people who are having a bad day. Can we just stipulate that any interview technique will fail to recognise the potential of some candidates? False negatives of that sort are kind of inevitable. And from the point of view of a hiring company? a few false negatives aren't the end of the world. A false positive is way more costly.
Because a candidate with a mediocre background yet a strong showing on the whiteboard will get the job. The problem isn't not hiring somebody having a bad day. The problem is the over reliance on the whiteboard signal, despite a candidate's history of contradictory evidence (either positive or negative). If you are going to use typical whiteboard problems as a signal, it is really really hard to not heavily weigh the results, especially when sour.
Your argument is that using coding tests under interview conditions is bad because a poor interviewer will overweigh it in their assessment. But the same applies to everything that takes place in an interview. Some interviewers will make up their mind on the candidate's handshake, so it's probably not fair to have handshakes in an interview. Others might place too much weight on the vocabulary and language an interviewee uses in answering questions, so we should probably not let them hear the interviewee speak. And asking candidates to write code risks the interviewer placing too much emphasis on the interviewee's ability to demonstrate their coding skills, so we should prohibit that too.
"Because a candidate with a mediocre background yet a strong showing on the whiteboard will get the job."
Is there evidence that this happens a lot? Does this happen at Google, where whiteboard programming during interviews is common?
"The problem is the over reliance on the whiteboard signal, despite a history of contradictory evidence (either positive or negative). If you are going to use typical whiteboard problems as a signal, it is really really hard to not heavily weigh the results, especially when sour."
What history of contradictory evidence? Do you have any suggestions for other signals that have proven effective in your experience?
I have very, very little experience as an interviewer and no experience dealing with and evaluating the success of new hires in the long term, so I definitely don't claim to have the answers here.
Sorry, the 'history of contradictory evidence' should have read the 'candidate's history of contradictory evidence' (e.g. having a really strong portfolio and work history to contradict a poor showing on the whiteboard).
I try to interview a candidate like he is a friend I haven't caught up with in many years. I want to have as honest and interesting of a conversation as possible, hear all his opinions, and learn all about what he has done and can do. If I feel the conversation was honest, the personalities were compatible, and there is evidence of good work and reliability in the past, I will likely consider the candidate a hire.
The problem with this method can be
1) It relies on the candidate having some sort of tangible history (referrals, solid portfolio or resume, open source projects, etc...), so new grads are tougher for example.
2) Having an 'on the level' conversation with somebody who knows they are being interviewed is not always possible. Nor are all interviewers capable of extracting useful information this way.
We hire based on personality. Before we bring an interviewee in we have looked at their bitbucket, github, portfolio, etc.. By time they get to the interview we know they are good enough to work for us, so we just talk technology with them and gauge based on that.
If the person is inexperienced or does not have a very public profile (no opensource projects), then we will drop back and throw them some random problems to solve but never a puzzle, since most people just memorize the solutions to puzzles or learn the solutions after doing enough interviews.
We prefer giving them problems like writing a text based guess the number game or tic tac toe. Mindless easy to solve project. They could spit it out in the shortest number of lines, which is what the bad puzzle questions encourage, but we are looking for how well the design/architect code. So the person who creates a good class structure and writes tests for his/her game is the one who gets hired, not the person who prefers shortest lines of code.
Well that is unfair. I got did not get an interview at a financial company for this reason. Unless stated explicitly, I'm not going to gold-plate interview code, because my assumption is that you are weeding out people who cannot code.
If you tell me to write it as OO as possible, then I'll do that. But if I can write it in a shorter, more efficient way using only arrays, I'll do that.
We tell them "Write this how you would write a real world application". We don't care if it is OO or functional or whatever... We just care if it is well designed. We don't tell them to write tests but if they don't then that means they don't write tests in the real world either.
There's just too much context missing from what this "real world application" is supposed to be or where its supposed to serve to make the concept meaningful.
What size inputs am I expected to handle? What platforms must this run on? What environment can I suppose is available? What performance is available? What kind of tests does your company use?
I think you're making too much implicit assumptions that everyone is programming in an environment similar to yours if you state it that way.
"We prefer giving them problems like writing a text based guess the number game or tic tac toe. Mindless easy to solve project."
Language, Environment, etc doesn't matter. If you are interviewing people on that stuff then you aren't finding the best candidates.
They get to choose everything outside of the problem to solve and it is an extremely dumbed down project like I said in the original comment. The "real world" part is in the implementation... How well is the code architected? Did they write tests? Are there performance issues? Bugs?
We don't expect people to be perfect either, we learn more from the discussion afterwards, since its not a puzzling problem we get to focus on software design and not the problem itself, which we feel is more important.
Why would I write tests for Tic-Tac-Toe? It's insanely easy to exhaustively hand-test, or just look at the code to see its validity. If you expect tests on something that simple, then you're expecting people to waste 1/3 of their time testing code that is sure to work.
Our interviews are never in the office. They are at a restaurant or bar. We just have drinks and food with them and chat. Talk about technology, them, us, etc... If we can have lunch and drinks with someone and they are talented then they are a good fit for the team.
The lunch/bar setting also loosens the interviewee up and they are always less nervous than if we had brought them into the conference room.
We don't care how talented you are if we can't stand being around you. If we had to choose between a mediocre programmer who has a lot of passion and fun to be around or a brilliant programmer who is arrogant, we'll take the mediocre guy and just train him up into being a brilliant programmer.
You're selecting for extroverts at the expense of introverts. Which may be desirable for your company, but I would (somewhat selfishly) not want that to be the common case.
we'll take the mediocre guy and just train him up into being a brilliant programmer
If you actually have a successful process for doing that, you could be owning many islands.
Its not about introvert vs extrovert, its about passion. I don't think any of the people on my team would do well socially at a bar but you ask us something about Linux, Python, etc and we could talk for hours.
"If you actually have a successful process for doing that, you could be owning many islands."
Don't pull a fox news on me, you can't rip out the most important part of that sentence ;)
