Yeah, usually we would give a little bit of context, and then look at the sprint tickets in Jira, and take one. The candidate was not expected to be familiar with the codebase or product at all beforehand, and they weren't expected to become an expert on it during the course of the interview. But coming up to speed quickly is one of the skills we liked to see, since that was part of the job.
They should be able to 'hang', in the sense of being familiar with how we approach problem solving as programmers. They should ask questions when needed, and answer questions as they arose. The goal was to just figure out what level they were at in general, and what it would be like to work with them. They weren't expected to come up with the best possible solution, because usually there wasn't a single right solution.
We would ask them questions like "well, you saw the ticket, how do you think we should approach this?" or even just natural things like "this function we just wrote isn't working, what's going on here?". The point is that they are working on real problems — literally real problems — and not abstract CS exercises.
Since we mostly did pair programming every day, the other thing you'd get a sense of is how easy someone is to work with. Not everybody had experience pairing, which is to be expected. You found out quickly who took to it, and who wanted to be a lone wolf. We didn't not hire someone because they weren't good at pairing, but we factored it in. Some people just did not communicate very well, which is a problem.
(This also gave them a good sense of what it would be like to work with us, and sometimes people decided not to take an offer because they didn't like the way we worked: good outcome for everybody)
They would usually do two rounds of this, both on the same day. Morning session, then an afternoon session. We offered to take them to lunch in between, which exposed them to more of the team, and let us talk to them in a more casual setting. Part of the interview would also be a private talk with one of the founders, and an HR person (in a different session).
The people the candidate paired with would provide input like: "I would recommend hiring this person" or not, but didn't make the decision themselves.
Here, I should admit that we did have a set of artificial problems we'd give candidates if we were hiring for a skillset that did not have a real project to test them on. We did both web and mobile contracts at this (consulting) company, so if we were hiring, e.g., an Android dev but didn't have an Android project for them to work on, we'd have them check out a repository and work on a fake ticket instead. Usually this was with a developer beside them, but I can remember times when the candidate had to do it solo.
This is great. If I had my own company I would do basically the same:
1. have a look at the candidate's github / stack overflow profile. If you like what you see, invite him/her for an interview;
2. have a couple of pair programming sessions as you described, with different team members (and don't let team members influence each other during the hiring decision);
3. invite the candidate for lunch and see how he/she treats others (basically an asshole test)
They should be able to 'hang', in the sense of being familiar with how we approach problem solving as programmers. They should ask questions when needed, and answer questions as they arose. The goal was to just figure out what level they were at in general, and what it would be like to work with them. They weren't expected to come up with the best possible solution, because usually there wasn't a single right solution.
We would ask them questions like "well, you saw the ticket, how do you think we should approach this?" or even just natural things like "this function we just wrote isn't working, what's going on here?". The point is that they are working on real problems — literally real problems — and not abstract CS exercises.
Since we mostly did pair programming every day, the other thing you'd get a sense of is how easy someone is to work with. Not everybody had experience pairing, which is to be expected. You found out quickly who took to it, and who wanted to be a lone wolf. We didn't not hire someone because they weren't good at pairing, but we factored it in. Some people just did not communicate very well, which is a problem.
(This also gave them a good sense of what it would be like to work with us, and sometimes people decided not to take an offer because they didn't like the way we worked: good outcome for everybody)
They would usually do two rounds of this, both on the same day. Morning session, then an afternoon session. We offered to take them to lunch in between, which exposed them to more of the team, and let us talk to them in a more casual setting. Part of the interview would also be a private talk with one of the founders, and an HR person (in a different session).
The people the candidate paired with would provide input like: "I would recommend hiring this person" or not, but didn't make the decision themselves.
Here, I should admit that we did have a set of artificial problems we'd give candidates if we were hiring for a skillset that did not have a real project to test them on. We did both web and mobile contracts at this (consulting) company, so if we were hiring, e.g., an Android dev but didn't have an Android project for them to work on, we'd have them check out a repository and work on a fake ticket instead. Usually this was with a developer beside them, but I can remember times when the candidate had to do it solo.