I do worry about requiring a one-week project from someone.

However . . .

Sometimes I worry "what if I lose my job today?" It really could happen, even today, through no fault of mine.

People who live in Silicon Valley will say "well you have five other jobs waiting for you." But people not in a top-5 area don't have a job market 5 deep. It takes time to get hired again.

If I were to suddenly be unemployed, I would really love it that there are some companies out there willing to hire me for a one-week project at a much lower activation energy than taking on a full-time hire. It would 1) give me a chance to prove I can do the job, and 2) even if it failed it would give me money in my pocket right now which would be a very nice stress reliever. (In fact, it reduces my stress right now knowing that I could find a few short jobs like this. I'm happier even if I never need to use it.)

I certainly wouldn't want all companies to hire like this, because most of the time when I am searching for a job I am employed. But I like knowing that there are some employers out there that it would be easier to work with being unemployed instead of the reverse.

Sure, but the possibility of going off of unemployment for a week and then back on again might be somewhat painful (especially since the interruption while reopening an unemployment benefits claim might last longer than just a week).

That's not how unemployment works, at least not in Texas. You wouldn't go off the unemployment until you were actually hired. You would simply report the income during that period (2 week periods in TX) and they deduct the amount from your benefits for that period.

Yes, most states encourage you to take work, even temporary work. A contract job won't require you to stop it and start it, you just "skip" a week.

Last time I was on UI, it didn't actually last for X weeks, it lasted for Y dollars, and if you earned more than a certain threshold in a given week, they would reduce your payout that week by 50% of the amount you went over that threshold. So one week of high earnings would, in effect, mean you get $0 that week but give you an "extra" week at the end.

I completely agree.

I interviewed with Skype once, I ended up not taking the job because it would have meant moving to SF and I don't really want to do that, but they asked me to do something interesting. They asked me to complete an assignment that was similar to the work I would have been doing, but did not require me to install their codebase, nor was it such a huge task that it would have taken a week to do.

This, to me, feels more like a happy medium between whiteboarding and "auditioning". The task was simple enough to be completed in 8 or 10 hours that I could do after work, and I even had some time to add some flair to it for style points.

Ditto here - I did an interview with AirBnb once where I had 3 hours to build a mobile app from the ground up to a specification. They provided all the designs and assets necessary, as well as the API and specification. The exercise was broad, covered multiple areas, and was holistic (you get a shipping app out the other end, not an isolated non-contextual piece of code).

I rather enjoyed it, and at 3 hours wasn't a huge intrusion on my time. This sounds a hell of a lot more attractive than throwing a whole week of my spare time at a company I'm not sold on, and IMO is just as representative.

The extra perk of this approach is also that, assuming you give the same/similar assignment to every candidate, it makes their performance directly comparable.

I could see that 3 hours is fine for something you are familiar with. Right now I am at my desktop and everything is ready to go for building something Django based. But say someone decides I should do it in Flask. I have looked briefly at the docs, but the fact I don't know it means I am unlikely to get far in the first 3 hours.

Yes, exactly. That's what I do when I interview people. I make it clear that this is something I am actively trying to solve (or maybe I've just recently solved it), and not just some academic exercise I'm doing that isn't productive.

I think that makes a huge difference. And in the end, I don't care what language you do it in or how you do it, so long as you solve the problem. If I write it in Ruby and you write it in Java, then if it works, I can hire you and your first job will be to port it over ;)

I moved to SF and I love it, but I understand not wanting to move around. I'm resisting moving away from SF for other opportunities...

Oh I'm not against moving around (in the last 4 years I've lived in Toronto, San Diego, Sacramento, and Los Angeles)... My wife and I just found SF/Northern California is not really our taste ;)

If only somebody made software which allowed video calls with remote employees they'd have been able to hire you without having to move to SF.

Interesting. Could you please clarify if your interviewed when Skype was already acquired by Microsoft?

I wander if Skype does similar thing now, i.e. maybe they preserved some autonomy and did not adopt general MS style puzzle interviews.

This was in 2010 IIRC, so I guess it was before the Microsoft acquisition

MS employee here. AFAIK puzzles have not been used in a long time.

I meant CS puzzles (like in programming olympiads), which are still widely used I believe.

The article actually has a very, very nice suggestion that I hadn't ever thought of - the idea of having that audition be a todo-item on some open source project your company uses.

Many companies rely on a bunch of open source components/tools/whatever, and they don't fit perfectly; so if you think about it, it's quite likely that you already have some 'good-to-have' items that you'd like to be done on those projects; important enough to pay for it but not important enough to have it done right now by your current people.

And the good thing is that it'd allow you to solve the problem your way with your favourite tools, not require any NDAs or granting access to internal systems, and also be useful for the rest of community.

That's true if and only if the codebase is something that's easy to set up. I tend to get these assignments from companies that are like "Here's our Github repo. Clone it and fix issue #12345." When I get to the repo, there's an INSTALL.MD file a mile long, where I have to install a dozen dependencies and so on. And it never, ever goes as smoothly as it should. They inevitably forget to document some quirk or another.

That's not true for everyone, but it seems like it's the rule rather than the exception.

My point being that you're not just asking me to fix a to-do item, you're asking me to recreate your dev environment locally, research your codebase (to isolate the bug and figure out a fix), and then actually fix it. And if this takes more than a few hours, I may not have time to do it whether you pay me or not. Like I said, I'm looking for full-time work, not contract work...

That's all I'm really saying. I agree with you, though, that the to-do thing is a good idea. I just think there might be some middle ground, like if your task is "Build a feature to send verification codes through Twilio", then instead of using your codebase, I do it on my own in my own way. Then you can use my code as a model for your own. You don't have to research the whole problem -- just port my code into your codebase. And I don't have to fiddle with your codebase until you hire me.

Something like that. (Just thinking out loud here.)

You could also, as an employer, supply a pre-built development environment appliance for VirtualBox that already has your source and everything set up. Then a person just has to install VirtualBox, add the appliance, and boot it. I think there is even a way to get it where the person doesn't even have to install VirtualBox.

As an employer you could also allow access to a remote instance of a machine that is already set up and configured and the person can SSH/RDP into the box and do development there.

I'm thinking that there probably isn't going to be a clear cut, one size fits all answer to this, but I do think there are some good options in any case.

This solves the "setting up the codebase" question but doesn't quite solve the "learning your codebase" issue. Especially if we're talking about a codebase with any sort of custom frameworks or heavy modifications of existing frameworks.

If I can't get a "Here's a real problem we have, solve it however you want" thing, then this is the second best answer. You're absolutely right.

I like the way you think! This is the sort of thing one tries to get out of audition projects.

Open-source projects will generally be much easier to set up than a typical corporate codebase. A lot of effort to set up a corporate dev environment is annoying, but to an open source project it's death.

Getting familiar with the codebase is still an issue, but at least that knowledge has the potential to be useful even if you don't get the job.

This is true, but generally speaking, most companies aren't working on open-source projects like Jeff's Discourse is. Most companies that want us to do this "real" coding assignment are in corporate dev environments.

I have no doubt this process works well for Jeff's company which is open-source (as far as i'm aware), but it doesn't work so well for, say, Stripe or Kickstarter, whose codebases are NOT open source and likely have complicated dependency requirements.

The solution here is obvious to me.

The specific problem with this scenario is that they give the audition too early in the process: they want to have a set of fully fleshed out apps to compared candidates from and use that as an initial/mid level filter. This is disrespectful on the part of the company precisely because in the best case only one will be hired (more likely none will be hired). It utterly devalues the time of prospective employees for the benefit of the company. This is wrong.

The solution here is to give the "audition" after the candidate has passed all of your other filters, and commit in writing to hiring the candidate if they do a decent job on the audition. And no wishy-washy bullshit: have an objective criteria to measure against and judge it fairly. This is the kicker here, which of course will be the tough swallow for most companies. Paying for their time is nice, but its really missing the pain point here. If you want someone to commit a disproportionate amount of time in the process of auditioning for your company, you need to put yourself on the hook in a similar manner.

I agree 100%, beyond that my current contract strictly prohibits this kind of work. One of the premises of the article is that the restrictions on location undermines the goal of hiring the best. Given his hiring requirements I couldn't even look at his company--it might as well be on the moon let alone the valley.

Ditto.

We have just had an article on here about Susan Kare - icon designer extraordinaire.

I guess that when Facebook picked up the phone they didn't bother getting her to do a new 'like' icon or whatever it was they needed using their tools in their special dev environment with their special workflow.

They probably noticed that her other work had done quite well and thought they might as well chance it with her. They probably hired her without her drawing one pixel for them. Her competency was not in doubt.

This cumbersome audition process people want to put you through might work well for someone fresh out of college that has no portfolio of work, however, for those of us with work that is live and on the internets, isn't that, with a few code samples, good enough?

It's a little more complicated than that. I do agree that given a few samples and an in-person interview process, you should be able to determine my coding skills without me having to do a huge, week-long project. But it's not quite so simple as "Hey, look at my Github. See? I can code." Because you don't really know how much that was cut and pasted and how much of that was written by someone else and how much of that was written by the candidate herself.

But I agree that there should be some metric that we can gauge candidates on coding ability without giving them a week-long assignment.

If it was an easy fix, Jeff wouldn't have had to write this blog post, so... Gotta keep thinking about it, though. There's gotta be a better way.

I think portfolios for developers is a great idea, but it is difficult to ascertain if its really the developer's work, and what their contribution was.

Portfolios are all well and good, but a large proportion of IT work (I would guess the majority) is doing in house apps, where you aren't actually able to build a portfolio.

Auditions where you get a week to do it could easily be put on a freelance site.

I see where you're coming from, but I come at it from another angle to arrive at the opposite conclusion about "Hire by Audition" or "Overcome specific real problem" tests.

That is to say: I don't have the money, time, connections, nor geographical/familial luck to have been born in America or in a supportive environment and attend a "top college". Nor do I desire to throw away several years of my life and six figures to get a piece of paper from people dumber than me to say i'm as smart as they are.

In that respect, i'll take the one week of unpaid work over several years, thousands of dollars, and social/political/time barriers which are completely insurmountable to me because it puts me on equal footing with the stanford/mit/harvard graduate. If implemented well it means its more likely to have something to do merit, and i'll respect companies that have a process that at least TRIES to implement a merit based process far more even if i don't get the job.

I don't like it, but i prefer it over the alternatives...

For what it's worth, I went to a tiny college in rural Alabama, and I've got a six figure job at a pretty large startup that I'm almost positive you've heard of, if you live in the US. The "top college" thing is bunk, as far as I'm concerned.

I think you're misunderstanding me though. I'm not saying they shouldn't require that you do a work-from home assignment. I'm talking about the fact that most "auditions" I get are far more than a simple homework assignment, and those employers often expect far more time than I have available. Like I said in another comment, they aren't just asking me to solve a to-do item from their to-do list, they're also asking me to install their codebase (and all that entails), learn it enough to isolate the problem and formulate a fix, and THEN and only then do I actually get to implement my fix and submit it.

That's more like contract work and less like an audition.

To make another comparison and riff on the word "audition", as an actor (I am one), when I go audition for a play, I'm not rehearsing with other actors and performing on stage for 1 night (the equivalent of a 1 week contract in web dev). Instead, I'm showing up with a prepared monologue (60-90 seconds) and possibly doing some cold readings (the equivalent of a "homework assignment"). The entire audition lasts an hour, maybe two.

One of those requires far, far, far more work than the other -- and the second one is what an actual audition is.

So what I'm getting at is that the problem isn't a merit-based approach -- the problem is a merit-based approach that doesn't respect my time and energy.

Does that make sense?

Here's the thing: I've got a job and degree too.

I actually agree with you, it doesn't respect a person's time and energy. It sucks. But the status quo doesn't respect it either, and it in fact requires even more time and energy (several years, several thousands of dollars), but we just don't account it as such because its just the societal norm. I'm not proud of my degree. I view it as essentially a societal bribe i had to pay to the indentured institutions to be considered for 99% of decent paying jobs. So it goes.

I can't think of a way to accurately measure someone's abilities that doesn't require significant time and energy from both parties :( But i'll take such solutions over the current alternatives...

There are of course some ethical problems when companies literally try to get you to do their work during the hiring process which i can't pretend to support.

I don't really disagree with you. The entire situation sucks, and you're right. To quote a quasi-famous mad scientist, the status quo is... quo.

I just think you can give me a programming assignment that I can do in a few hours instead of an entire week, and still get the information you need to determine whether I can code well or not.

I personally love my degree. It's actually in theatre (not CS), and I really, truly believe that it taught me a lot about how to contribute to the workplace and to society in general. If I had to do it all over again, I'd probably do the same thing. But I guess we just got different things out of our degree programs. I do wish that there was a better alternative, though, for those like you who would (I assume) rather just go straight into the workforce.

What if the hiring company culled the list down to a set number of candidates and gave them all paid contract work, on something they needed done anyway, as an audition?

This gives both the applicant and the company a chance to see how working together might go. The company gets something on their to-do list done, and the candidate gets paid for their time.

Yep id want £350+ per day for contract dev work and as I woudl have to do this after hours or at weekends thats time and 3/4 so £600 per day x 5 thats 3 Grand for a week.

Oh and £350 is low for a day rate you can easily get £500 plus for high end devs in the city / big data.

I would absolutely charge for this kind of thing, but like I've said before, if I was looking for contract work, I'd be a freelancer. I'm not looking for a one-week contract with a possible extension, I'm looking for a full-time job. For Jeff to offer me a one-week contract as an "audition" completely misses the point of my goals in favor of his goals.

But yes, if you wanted me to work on something for a week, I'd absolutely want to be compensated for it. But in my current situation, I just personally wouldn't take that offer.

Auditions are wonderful for job seekers: they allow you to evaluate the company much more deeply than the company evaluates you. If the work is tedious, boring, and uninspired, you know that this job isn't for you.

I once applied to a company that insisted on interviewers doing a test first. Just looking at the questions on the test, I knew this wasn't a company I wanted to work for, so I didn't even bother doing it. They asked me "why didn't you do it?" and I told them I knew this job was a poor fit just by looking at your test questions. I'm not sure if that is what they intended, but the benefit was good for me.

You are not wrong. I have experienced the same thing myself. I'm not saying the audition system is a bad system -- I'm merely pointing out a flaw, that a week-long contract is a lot of time to ask me to work on a project for you if I already have a job.

There are certainly many benefits to the audition system, or Jeff wouldn't have even mentioned it!

And if you have a job doing this outside contract without your current employers permission your breaking your employment contract.

How about these companies set up a virtual desktop instance with the environment all set up and ready to go, and just give applicants a login?

You're on the right track. At my job, we use Vagrant and VMWare Fusion to load up our dev environments. I'm thinking it might be trivial (or at least, less complicated) to simply give the VM to my candidate... The problem is that sometimes they don't have VMWare or VirtualBox. Ideally, I'd just write a single script that installs everything for them, so all they have to do is write code...

Maybe for the upcoming hack week...

Even "Lets have a quick chat on Skype" can be a pain in the arse. I use Linux, and don't use Skype regularly, so its frequently the case that I haven't set up /used Skype on whatever laptop I happen to have at the time.

People might assume that I am some sort of idiot for not using Skype, but I have a proper VOIP number setup, so I don't have the need.

Perhaps use a preconfigured Amazon AWS Workspace. I believe it comes out to less than $100/month for a fairly powerful remote workspace instance.

something like nitrous.io would work for this.

At this point in Jeff's company he is looking for someone who wants to get paid to work on Discourse, not simply have a job. Someone who wants that would not think much of following the easy instructions to get Discourse, install it, and be able to make a small contribution to the project (not sure about contributing but the download and install process is extremely easy and is sort of a fizz-buzz test in itself).

I don't know if this good advice for everyone looking to hire but it seems like filtering for both ability and motivation is a good idea for young, growing companies.

The problem with any discussion here on HN is that, judging by replies, many commenters feel insecure whenever an idea is proposed that they feel could exclude them from a job, even though typically these ideas do not spread, and most people need only one full-time job at a time (writing about interviewing has the same problem. See the first few paragraphs of http://steve-yegge.blogspot.com/2008/03/get-that-job-at-goog...).

I agree that if every company started this practice in the same way, applying for five jobs at the same time while currently working at one would be difficult. But it seems very unlikely that would happen in practice.

True, but just because I want to get paid to work on Discourse doesn't mean I want it "simply to have a job." I already have one, remember? I would also love to get paid to work on any number of other projects, but that doesn't mean that I magically have a week worth of evenings available to work on your contract project.

I'm not upset at being excluded from a job at Discourse -- in fact, I'd probably exclude them from my search! Quite the opposite, my time is valuable to me.

Just because I can't take a week to work for you on a contract basis doesn't mean I lack motivation or ability. It just means I have priorities that don't include sitting in front of a computer screen 24/7. I may have a wife and kids, or I may be an Olympic runner, or I may be struggling to make ends meet with other contract work, or I may be taking care of a sick relative. Who knows?

All I'm really saying is that there's a happy medium here. Some way for you to give me a programming assignment, and for me to complete it in a reasonable amount of time. Offering to pay me just means that you want a LOT of my time -- time that I likely don't have to spare.

If I were a junior dev and desperate for work, then yeah, I'd make this a priority. But Jeff isn't talking about junior devs -- he's talking about the best. And the best already have jobs and don't need to prove anything to anyone.*

* (I'm not saying I'm one of the best*)

Are you concerned that every employer would begin do follow these practices and require auditions? How likely do you think that is?

Nah, not concerned at all. I'm just pointing out that auditions like this necessarily rule out a large subset of the population right off the bat. Maybe that's okay for your business objectives, maybe it's not. Maybe you're looking for "the best" and not qualifying "the best what." Maybe you're ruling out the best senior developers (who may not have time for this or have little to prove) and picking up the best junior developers (who may have spare time and everything to prove). Maybe that's acceptable to you. Or maybe that's exactly what you want.

I'm not saying "don't do it." I'm saying that from my perspective, I think it's a flawed interviewing technique and it's not something I would be interested in when applying for a job.

If it works for you, then by all means go for it. I'm just saying it doesn't work for me.

He's partners with Joel Spolsky, whose old article on hiring [1] set the stage for a lot of the modern hiring era. Even though almost all of the questions are considered unfashionable today, I think the gist of "find any excuse not to hire" still holds in a lot of people's minds. The more people you reject, the better your candidates are, right?

[1] http://www.joelonsoftware.com/articles/fog0000000073.html grep for "no hire" just to get a feel

The weirdest about Joel's method becoming the gold standard was that FogBugz around that time (I last used it around 2004) just wasn't that great. It was clunky, weird, and required a Windows server. It was served through a browser at a time when presenting GUIs through the browser was a slow and tedious string of refreshes.

Trello and StackOverflow are so much nicer now than FogBugz was then so I guess he hired better people. But I think a lot of his advice became vogue more because he was an entertaining writer than a great software entrepreneur.

Joel and Jeff are not partners in this venture; Discourse is Jeff alone.

The process of applying for a new job while you still have a current job will take time, no matter what the process looks like. The hiring company will want a series of interviews, and either a whiteboard, pair program, or code audition. A small project would perhaps take a little more time than these other methods, but should not take take that much longer, provided the hiring company has put some thought into what small projects they are giving to you to audition. If they pay you for your time, that is also respectful.

Jeff probably has enough "internet fame" to get away with asking folks to audition with a longer project and get good results.

Personally I'd be happy with anything that should take less than a day to do, but giving folks access to a pre setup client that has RDP or SSH setup would make that a lot less painful. A week is a bit too long.

I'm currently looking around for a new job and I'm being fairly picky about what it is I want to do, my biggest pain point is getting time to actually interview. My own personal preference would be a couple of phone interviews, then an audition project followed by something face to face. By then it seems likely we're going to get on so taking a day off is worth the effort.

Finally as someone currently looking around salary is something I'd like to at least potentially discuss relatively early on (or at least check we're both in the same ball park) I'm going to ask slightly high but caveat that with a statement saying that I'm potentially flexible for the right job (I like your problem domain, it will look good on my CV, you're offering equity etc).

I agree with you on the general workflow of an interview process. Phone screen or two, programming assignment (not 1 week though), face-to-face, etc.

I need to get better about the salary thing. I've gotten several offers from companies (even YC companies) in the last year, and each time we're just not in the same ballpark. I think a lot of it just has to do with equity. Startup CEOs treat their equity as worth a lot of money, but at the end of the day, I can't use my equity to pay my rent. So unless I really, really, really believe in your company's ability to dominate in that problem domain and/or exit successfully, your equity just isn't worth the pay cut for me...

But that's another discussion entirely.

I, too, am picky about what I want to do. I want to solve real problems. I don't want to reinvent the wheel with yet-another-message-board (sorry Jeff). I want to save the world. I want to do something meaningful. And a lot of these startups I see around here are solving what I consider to be superficial problems. Yet another ephemeral chat service. Yet another Instagram. Yet another subscription clothing service. Zzz.

If Elon Musk called me up and asked me to come "audition" for SpaceX, I'd probably do it. That's an exciting enough field and opportunity that I'd spend a week of my free time to get it. If a company that was building out tools for combating malaria called me up and asked me to audition, I'd probably do it. Or updating farming with 21st century technology. Or engineering a way to purify water for pennies. Or... well, any number of what I consider "epic" problems.

But that's another discussion, too. :)

I totally just went off on two different tangents. Whoops. :)

If you are happy with your current job, then there is no need to apply to a different job, so this scenario would never come up for you.

A more serious problem: it's not arithmetically possible for everyone to hire the "best".