Your argument is that using coding tests under interview conditions is bad because a poor interviewer will overweigh it in their assessment. But the same applies to everything that takes place in an interview. Some interviewers will make up their mind on the candidate's handshake, so it's probably not fair to have handshakes in an interview. Others might place too much weight on the vocabulary and language an interviewee uses in answering questions, so we should probably not let them hear the interviewee speak. And asking candidates to write code risks the interviewer placing too much emphasis on the interviewee's ability to demonstrate their coding skills, so we should prohibit that too.

"Because a candidate with a mediocre background yet a strong showing on the whiteboard will get the job."

Is there evidence that this happens a lot? Does this happen at Google, where whiteboard programming during interviews is common?

"The problem is the over reliance on the whiteboard signal, despite a history of contradictory evidence (either positive or negative). If you are going to use typical whiteboard problems as a signal, it is really really hard to not heavily weigh the results, especially when sour."

What history of contradictory evidence? Do you have any suggestions for other signals that have proven effective in your experience?

I have very, very little experience as an interviewer and no experience dealing with and evaluating the success of new hires in the long term, so I definitely don't claim to have the answers here.

Sorry, the 'history of contradictory evidence' should have read the 'candidate's history of contradictory evidence' (e.g. having a really strong portfolio and work history to contradict a poor showing on the whiteboard).

I try to interview a candidate like he is a friend I haven't caught up with in many years. I want to have as honest and interesting of a conversation as possible, hear all his opinions, and learn all about what he has done and can do. If I feel the conversation was honest, the personalities were compatible, and there is evidence of good work and reliability in the past, I will likely consider the candidate a hire.

The problem with this method can be

1) It relies on the candidate having some sort of tangible history (referrals, solid portfolio or resume, open source projects, etc...), so new grads are tougher for example.

2) Having an 'on the level' conversation with somebody who knows they are being interviewed is not always possible. Nor are all interviewers capable of extracting useful information this way.