I keep meaning to write some kind of blog about this, but I call it "time opportunism" as opposed to "time management". That's the best name I could come up with for this thing I do that keeps me creative and flexible yet still getting things done.
In a nutshell, I quit trying to explicitly plan and organize my life (not that I was any good at that) and instead I have a giant bag of things I need to do. There's usually a bunch of projects, and each has a todo list. All I do is when I have the time to work on one AND the motivation, then I go bang out as much as I can until I can't. By having lots of different little projects I'm pretty sure I'll always have something to work on that motivates me at that time.
The only caveat on this is I then do one thing at a time. I see too many people who can't get their todo list down are usually trying to do 5 things at once. A good sign you're this kind of person is if you love patch queues. Just do one thing, and make sure before you do it that you can do it. Don't juggle 10 things at once.
Thank you for posting this comment. I don't know why but this completely makes me think of context switching in the kernel and everything then becomes clear. I am guilty of "multitasking" and at the end of it accomplishing almost nothing, only recently have I realised the error in my ways. Your recent work and comment are confirmation that focusing on one thing at a time is the way forward when working on substantial tasks.
I don't know how you got downvoted, but I don't suppose it changes the intent of my reply either way:
That MIGHT just be the secret to Zed's success -- that so much in the world seems to piss him off means that there is so much stuff he feels he needs to fix that he's constantly drowning in the need to fix something.
Some of us are more complacent -- I'm happy to fix the things that irritate me, but my body of work may be less prolific because I'm generally happy with the products that I use.
For years, I worked for a boss who was always right. Not necessarily by being smarter, but by sheer force of will. She's a brilliant lady, and figured out the way to get some of the more competent technicians to perform was to subtly infer that she didn't think we could make deadline, knowing we'd bust our asses and put in 30 hour shifts to do it, just so she wouldn't be allowed to 'win'.
We called this process 'Hate Driven Development', and yes, in hindsight (and even in the moment) we all knew what she was doing, but she was clever enough to put it out there in ways where it worked even still.
I am, but to be honest, I got more done when I had a job. See all those sites? I did all of those, wrote for them, worked on LPTHW, wrote mulletdb.com, hacked on lamson, a few other projects, and still played guitar about 4 hours a day.
Now that I'm just working on Mongrel2 it feels like I do less. Weird.
Oh, and just out of curiosity, what guitar music are you spending the most time on? Is it mostly your own stuff, or are you learning a lot of things written by others?
He didn't cover defining first-class functions that I could see in a quick check, didn't cover generators, and I think most disappointingly, didn't cover __metaclass__. That's some critical stuff for Python programming right there, everybody needs __metaclass__.
Ahem.
You have to pick what to cover. For a beginning Python book, I'm underwhelmed by the need to drag an entirely new and foreign(-to-Python) paradigm into the beginning Python. I say this as someone who does a lot of functional programming, someone who is using a lot of lambda, filter, map, and reduce is someone who is stubbornly refusing to program with the native idioms, which are named functions (defined inline is fine), list comprehension or loop, list comprehension or loop, and loop respectively. It is not that enormous a gain, and will lose from Python fighting you all the way. (Plus you'll probably miss some opportunities to use generators, overload the special methods, and the other things you can do with native idioms, if you're trying to program Haskell in Python.)
I have programmed literally hundreds of thousands of lines of python without using classes or __metaclass__, as a counter example.
When dealing in industry, we can't always hire the best talent, so there is no need to make things overly complex. Simple solutions work best for a lot of scenarios, and I haven't been limited by it yet.
Not to bash Perl, but the whole reason we write in Python is because there were too many people on our team writing in Perl using too many obscure constructs that took too long to review / revise (especially years later when team members changed and code was forgotten in parts)
You’re implying that language syntax is primarily technical rather than social. I’m not sure I can agree. The reason that Python code uses standard idioms most places you go is that the larger-scale Python community is committed to quite a different set of social norms than the Perl community.
Pulling in that language-community momentum can definitely make a difference to the social/technical dynamics inside a smaller group.
... if you have multiple developers who cannot or will not find some way to write maintainable code, you have a problem independent of language.
I've maintained some hideous code written by people who had no business being professional programmers. In every case, adhering to a handful of very simple rules would have improved their code dramatically. Only one case needed a language change, and that one was away from a proprietary 4GL.
Keep in mind too that the new class decorators can do a lot of what metaclasses were good for, in a way much easier for the uninitiated to figure out. (Partly because once you understand function decorators class decorators seem obvious and straightforward.)
Metaclasses can do some pretty powerful stuff though. One of those features that you might never need, but if you do, you really do.
As far as I know, this piece of code (containing a lambda, map and filter) is standard idiomatic python:
[ (student.name, 'A') for student in students if student.grade > 90]
I'd assert that first class functions are idiomatic python as well and are also critical for python programming. They play an integral part of django, for example.
Yes, that's the idiomatic way of doing it, when people complain about lack of map, filter, and lambdas, they're usually missing an opportunity to do it more easily with list comprehensions. Note that doing this with list comprehensions is pretty competitive with Haskell on a keystroke-by-keystroke basis; insisting that your lambdas MUST have the word lambda and the filter MUST have the word filter and that maps MUST have the word map is getting caught up in the surface appearance and missing the deeper flows, which actually, IMHO, isn't a very "functional" attitude to have anyhow.
(Functional programming, IMHO, isn't about maps and folds and filters anyhow; what those are are the tools it uses to gain composability. Python uses other tools for that goal; when in Python, you should use those tools. Ultimately, Python is an imperative language and will have all the associated flaws, functional probably does a better job of composability overall. But if you're in Python, you're not going to build a cute monadic combinator library that you can use in Python no matter how hard you try to box it about the ears with map and reduce, so you might as well use what Python actually has, which is classes and generators and list/generator comprehensions and lots of very overloadable syntax and so on and so forth. maps and folds and filters are the surface of functional programming, not the essense.)
(And yes, you can build a combinator library, you just can't make it slick.)
Reduce is the only one that is not clean in Python ("fold"), and the official word is to use loops. Again, it's usually not that great a loss, but the loss vs Haskell is greatest here. On the other hand, compared against some actually-functional languages you're still not typing much more; Erlang's folds are hardly any better than a Python loop, character-per-character.
accum = 0
for i in l:
accum += i
vs
lists:foldl(fun (I, Accum) -> I + Accum end, 0, L)
First class-functions are absolutely idiomatic; what's not idiomatic is smashing things together with crazy "and" or "or" chains (or a variety of other hacky approachs) so you can have the magic-goodness-conferring word "lambda" show up in your source, instead of (horrors!) having to give your function a name.
Speak the native idioms of the language you are in. A lot of people are missing a lot of the good things about Python because they are too busy complaining that it isn't something that isn't Python. It isn't functional at its core, but it can be a very powerful, concise language, if you work with it instead of fighting it.
I thought the book was to be an introduction to programming through Python, that's a good opportunity to show "by the way, you can also see things the functional way...".
I'm no Python expert, I couldn't say what's idiomatic or not. That being said, I never had the feeling of struggling with the language when using map and filter, quite the contrary.
What's the idiomatic way of doing this?
suffix_str_lst = map(lambda x: x + "_suffix", str_lst)
If you are okay with a generator instead of a list, then you can flip the square brackets to parentheses and then you do not need to hold the whole thing in memory at once.
EDIT: You can add a filter in a similar way:
suffix_str_list = [x for x in str_list if x.startswith("meep")]
Read that out loud and tell me that doesn't grate on your ears? That's my basic test if code for the book would be understood by the reader. What you have here, this "symbol soup" is what makes programming hard for people in the beginning.
Some of the simpler list comprehensions (like the student one two parents up) are fairly "symbol soup" free, though - and a very clean way of expressing what you want.
OTOH, I can see how they could still be a hard concept for a beginner who hasn't done any functional programming before, particularly because they don't read front-to-back like most imperative statements.
EDIT: You can always split complex ones up as well, for most common usages:
meeps = [ s for s in str_list if s[:4] == 'meep' ]
suffix_str_lst = ['{0}_suffix'.format(m) for m in meeps ]
This one was probably a bit simple to really need that, but it's a bit easier to read. :).
I actually find it really elegant. I fell in love with comprehensions as soon as I discovered them. They were the terse, expressive syntax I'd long been searching for.
Is there a tutorial on learning simple one liners like this? I find myself frustrated after seeing simple and obvious solutions like the above, but can never bring myself to code them as such.
I'm only half kidding. The best way to get used to using functional concepts is to actually use them in their native setting, and then apply (giggle) them elsewhere. When you _have_ to solve problems this way, at some point, it clicks.
You are not mistaken, there is no chapter on functional programming. Why? Because "The Little Schemer" is a way better book for teaching better functional programming than I could possibly do in a Python book. Python's functional concepts are just not really that great, so I couldn't explain them very well.
If you're not teaching pythons functional concepts, then you're not teaching python.
However I think you have accidentally... consider this short example which shows off first class functions, map, reduce, no side effects, and filtering.
[(lambda a,b,p:(a+p,b+p))(x,y,2) for x in (1,4) for y in (10,15,22) if x*y > 25]
Python does functional programming without people even knowing that it is functional programming :)
While I do agree those principles are useful, I think introducing those topics would diverge the focus of the book. I think the goal was of introductory nature not comprehensiveness.
There are some great beginner's books out there that barely cover functional constructs (e.g. PragProg's 'Learn to Program'). But I have always felt that such an omission is really a bad mistake. map/fold/filter introduce two very important ideas:
- Higher-order functions
- Many tasks are just list transformations
What makes things more painful is that Java is so omnipresent in CS education. Since Java only contains such concepts implicitly (and require a lot of verbosity), I haven't encountered many (if any) Java books that go into functional concepts.
It's like learning UNIX without covering pipes. Sure, you can work with UNIX, inadvertently emulate it using redirections from and to files. But it will never be as effective.
Sure, that's why Graham Hutton can do it in one chapter. Most books discuss lists and functions anyway, and from there is not such a big step to give an introduction to lambdas/closures, maps, folds, and filters.
It's not as if we are discussing monads (which are also easy to understand, given a good teacher).
Yeah. The argument is based on reduce not being clear and easy to understand, which is frustrating to those of us who actually find map/reduce/filter to be more clear, not less.
It's in a module in the standard library, a module which contains a bunch of tools for working with higher-order functions. Or, in other words, if you've got a Python 3.x installed, you've got a reduce function available. The only difference is that it's no longer in the default global namespace (map and filter still are, of course).
This matches up pretty cleanly with my actual experience of writing Python; I've written a hell of a lot of Python code these past 6-7 years and so far as I can recall I've never used reduce.
(and if you want to get picky about this, keep in mind how much of Haskell requires import statements to get at -- moving less-commonly-used stuff into standard modules is not a bad thing at all)
Meh. I just sent this to a friend of mine that knows nothing about coding, but is taking a class (biology) that uses python. She really has no need of functional programming, she just needs to know how to survive her problem sets. I think that's the audience.
Exactly. I taught programming, not dogma. Frankly, if I can teach two paradigms, and one is easy to explain while the other is hard, well I'm going to teach the easy one to explain.
Functional programming is just hard to explain to people who know next to nothing. Especially in a language like Python where the functional concepts are kind of half-assed.
Ditto for generators. In the process you could introduce pipelining and thus give a window into the Unix way which is probably pretty relevant for a newcomer. Or I've just been around too long to know what's relevant for a newcomer.
I am learning python through this book at the moment. It is my first attempt at learning to code, so I have nothing to compare it with, but I must say that this book is very good.
My only gripe is that he poses certain questions/exercises that he fully expects us to solve, one way or another. But if we cannot; we're screwed. He does not give the answer. It has only happened once so far, but that was enough for me to miss out on what seems like a pretty crucial part of the book.
Welcome to graduate school! (And, for that matter, real life.)
As in grad school, the secret is: If you can't figure it out, ask around. Form teams that stay up until 3am waving their hands and sketching on chalkboards. If nothing else, it's a bonding experience that you'll treasure for life.
I get your message, but the problem is this is not grad school. This is a book; one way communication. I have no one to ask, as I am the only one I know who has an interest in coding.
Of course I can always ask on the web, but having to break off from coding, pose a question, and then wait for it to get answered before I can continue really breaks up my learning curve.
Asking clear questions will help you learn. You can ask on Stack Overflow (or #python) - make sure to credit the source of your question[0] and at what point you are stuck. Often by trying to explain the precise thing that you don't understand, you will have a moment of zen.
[0] People will be a lot more helpful if they know you are trying to learn, rather than posting a "send me teh codes plz" request. Also they will hopefully give you just enough of a hint and not spoil the whole thing for you.
Often by trying to explain the precise thing that you don't understand, you will have a moment of zen.
This.
I often find that I run into a problem, try to solve it on my own, then bring someone unfamiliar in to get a fresh set of eyes on it. It usually goes like this
Me: "Hey, I honestly can't figure out why this is doing this particular thing, can you take a look? <start explaining problem here in detail> ...FUCK! Nevermind, I know exactly whats going on. Thanks for your help"
Person I asked: "Huh?Uh sure? I can solve problems with my presence alone"
One of my professors in college would bring a small teddy-bear to the lab. Whenever one student wanted to ask a question, he would just put the teddy bear on top of the monitor, and you'd have to talk to the teddy bear first.
It seemed ridiculous, but it worked wonders. Most of the time said professor didn't have to open his mouth. It was just a matter of organizing your thoughts in a coherent way.
In the book "The Pragmatic Programmer" they refer to this as "rubber ducking". The origin of the term was from a particular programmer who kept a rubber duck sitting near his computer (I believe on his monitor, I don't have the book handy right now). When questioned about why the duck was there, he said that when he got stuck on a problem, he would explain the problem to the rubber duck (full well knowing that the duck was not going to respond :-D ). The purpose of having the duck was just to have a target to explain the problem to, knowing that the act of explaining the problem would often lead him to the solution.
One of my favorite troubleshooting techniques at the office is to have someone SHOW me the problem. Often, in just trying to show me how they're gettingn whatever error they're complaining about, they realize they're doing it wrong in the first place.
Then I get to stamp something 'solved' on my task list, and everybody's happy.
> Of course I can always ask on the web, but having to break off from coding, pose a question, and then wait for it
I've been programming professionally for almost 20 years. What you described right there is the fundamental skill/activity I have performed. Whether waiting for a spec, or decision if this feature is going in, or most commonly like your self looking for answer to some problem I don't get.
Good programmers either know everything (ha) or know how to learn/find the answer quickly. It's a great skill to start honing now.
>Of course I can always ask on the web, but having to break off from coding, pose a question, and then wait for it to get answered before I can continue really breaks up my learning curve.
You're skipping a vital step. Searching for the answer on the web. Most of the time the question you want to ask has already been answered.
Very true! Most things can be solved that way. But sometimes your issues are not googleable. I remember back when I was just learning javascript, it would never in my wildest dreams occur to me that there were host objects and that you couldn't write over them, since I'd only used compiled languages before... it took 5 seconds for my friend to start laughing when he looked at my code because I had named a variable "top."
Hey, can you tell me which ones you got stuck on? I can try give a bit of help either here or in the book.
One thing about those though is that if you can't complete them, then just move on and maybe mark it for later. You should try to figure them out for a bit, but sometimes you just can't and have to come back to it later.
Finally, make sure you have this latest version of the book. I added a small section in the beginning which talks about this, so you might have an older version.
EDIT: Oh I'm assuming you mean the Extra Credits. The lessons you should try really hard to do and figure out before moving on. But skipping Extra Credit is quite alright.
One thing that really took me a long time to figure out was that using python on Windows XP is something different than using it on a mac or on linux. Some things just don#t work or at least work slighlty different. Maybe you could ad a more explicit warning about that.
One thing you might add to the book is a “what to do if I get stuck” section. I think especially pointing people (even newbies) at IRC is a good idea, as long as there's a bit of elaboration about how to effectively ask IRC questions.
Freenode’s web client http://webchat.freenode.net/ isn’t perfect, but it should be figure-outable when accompanied by some simple directions.
Learning how to Google problems is an important programming skill.
Every programmer in the world is pissed that "I'm not sure - I'd Google It" is not an acceptable answer to a question about "how do I do X in programming language Y". Happens every day.
That being said - a book forum or Q/A section on the site would likely be very helpful. I'm learning Objective-C right now and the book I'm using has a corresponding forum associated which usually has the answers I'm looking for.
Same here. On the one hand it might be a good thing not to give the answers away so you also have to learn to search the web, use stackoverflow... but on the other hand it can be a huge time sink if you just want to know if your solution is correct.
Maybe some kind of forum in addition to the bok would be a good idea.
It depends on the question, and if you know the right keywords to use. People starting off in programming and working through a book alone generally won't.
Totally agree. Just trying out stackoverflow was probably worth it. Nevertheless when I thought I had the right answer I would have liked to compare it to other/the "right" solution. Also if I already spent a fair amount of time searching and had problems to phrase the questions the right way for Google (not knowing the terminology) some kind of help might have been a good thing.
On the other hand I might have used thatcher help too early...
The #python IRC channel on FreeNode is a great resource for programmers learning python. (But don't expect to be spoonfed, or have everything served on a silver platter, it'll be greatly appreciated if you can show your efforts (using a pastebin site, don't paste into the channel ;-))).
It's a lovely chapter! Thanks for pointing it out.
Programming as an intellectual activity is the only art form that allows you to create interactive art. ... . The world needs more weird people who know how things
work and who love to figure it all out.
It's inspiring to see him talking about the beauty of (software) engineering as a means of making which has the quality of an art.
Programming as a profession is only moderately interesting. It can be a good job, but if you want to make about the
same money and be happier you could actually just go run a fast food joint. You are much better off using code as
your secret weapon in another profession.
People who can code in the world of technology companies are a dime a dozen and get no respect. People who can
code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing
things to advance those disciplines.
I have always had the suspicion that this is the case but now I have some validation
May I suggest making it available in other formats, assuming that it is possible, once it is finished? Perhaps as text or HTML files in a tarball? Or even as an epub (for Nooks and iPad)?
The source files are written in rst(ReStructured Text) format, the standard documentation format for Python. Specifically using [Sphinx](http://sphinx.pocoo.org).
One of the default output formats IS HTML.
Build instructions on *nix:
$ fossil clone http://learnpythonthehardway.org lpthw.fossil
$ mkdir lpthw
$ cd lpthw
$ fossil open ../lpthw.fossil
$ make html
$ open _build/html/index.html
Please don't put it online in HTML just yet. I need to get it edited, and I'm afraid people will put versions of the book up in HTML format with errors and problems. Once the book is 1.0 then I'll have it online and anyone else can put it online. Can you take your link down please?
In the absence of an epub, GoodReader on the ipad is very handy for reading PDFs as books. The type is a little small on this document, but I am able to read it without zooming and panning.
What does GoodReader have over using iBooks? I've mostly just used it for storing manuals for tech products so far, but it seems like a reasonable way to read PDFs.
Mainly performance and options. Page turns, navigation, zooming are all faster. iBooks is my favorite eBook reader but GoodReader is noticeably better for reading pdfs. A mandatory app for iPad, along with Dropbox, SimpleNote and Instapaper.
Also you can crop the margins of pages so that they fit better without zooming in. It can convert pdf to text on the fly too, though of course there are formatting issues.
Nice, but that's an annoying license. I'm tempted into porting his assignments to Ruby, but do to licensing I'll need to do them completely separately - I can't make a cohesive document out of it.
Ah well. His prerogative I suppose. Just makes me feel like "teaching people to program" isn't his primary goal.
Don't copy, imitate. The book was very easy to write. Just lay out 52 exercises, 26 for super basic stuff, 26 for progressively more advanced stuff. Take about an hour or two every night and write a couple. Pretty simple really. You may not have my style in your writing, but you'll definitely get a book that is more Ruby than what I've written.
One warning though, if you think it'll be better to have the book crowdsourced, think again. Programmers seem to be pretty bad at teaching programming, so it's best to just write your book and have people give you feedback as you go.
Very nice. I just randomly choose to view the arguments unpacking section and found a defect (no example given on pg 36 for passing in only two args instead of all three, but later cited as "see how I passed in two args"). Created a ticket.
I am training myself in Python with this book from a week or so.
I know how to code in other languages and programming basic so I am not a first time coder...
yet this little book is absolutely useful and I am enjoying it exercise after exercise.
At the end of the day, it's not about functions or closures or monads. It's about building something interesting or useful. Whether you used php or arc is not relevant.
I recommended this to a new hire (frontend HTML/JS guy, but wants to learn Python). He really found it easy to get into, and appreciated the exercises. Thanks for publishing it.
