I'm seriously surprised at how many people here are fans of Yudkowsky, given his lack of any real productive output beyond thought experiments and fanfiction, the distinctly weird stuff he judges as right by taking his philosophy to the extremes (e.g. "it's better for one person to be tortured for fifty years than for an extremely large number of people to each get a speck of dust in their eye"), and the way he treats certain fringe positions as absolutely true despite the lack of any solid evidence to support them (for example, many-worlds theory).
> I'm seriously surprised at how many people here are fans of Yudkowsky
In my (admittedly extremely limited) exposure to him as a person through videos of his talks and some of his transhumanist writing, I didn't really get the impression that he's anything but a very nice and gifted person. So I think this practice of judging or dismissing entire lives by applying a simplistic theme is an antipattern, and this is true here, too. There's a reason why ad hominem attacks are generally frowned-upon.
Yes, I personally found the certainty with which several conclusions are asserted as universal truths on LessWrong in general offputting at times, especially when used in conjunction with the Rationality label, insidiously implying any other analyses of the subject matter would be inherently irrational.
However, some subjectively bogus tenets notwithstanding, I still think it's a valiant and intellectually stimulating attempt at building a new philosophical framework which could potentially keep up with science and future human development. At the very least LessWrong Rationality is a good basis for an ongoing discourse on the subject, and at its best it demonstrates a unique exploration of ethics and, indeed, rationality.
You may argue most or even all of this framework is lifted from earlier philosophical and scientific achievements, but in some areas standing on the shoulders of giants is actually a good sign you're in the right place.
>You may argue most or even all of this framework is lifted from earlier philosophical and scientific achievements, but in some areas standing on the shoulders of giants is actually a good sign you're in the right place.
The good bits are not original and the original bits are not good.
The problem is that LessWrong has a habit of neologism - so EY will use his own term for something ("fallacy of gray" is one example - known for 2000 years as the "continuum fallacy"), then his young readers, who have met whatever it is for the first time ever, will think his work is much more original and significant than it is 'cos they can't find his term for it. This cuts them off from 2000 years of thinking on the topic and increases LW's halo effect.
What about people like me that would never have learned about the "continuum fallacy" if it weren't for Eliezer's willingness to stoop to my level and explain things like I'm 5 (or, more accurately, like I'm a fan of Harry Potter)?
I personally don't care one bit if the good bits aren't original. They are approachable, and nobody else has done that for me. So I applaud Eliezer and his efforts, regardless of whether or not he has broken ground philosophically.
Would you have known that everyone else had been calling it the continuum fallacy all that time? No, you wouldn't - you'd think Yudkowsky was uniquely insightful.
Furthermore, you wouldn't learn anything beyond the limits of Yudkowsky's knowledge, or - more importantly - that there was anything beyond those limits.
The habit of neologism makes stuff impossible to look up, and creates the illusion that this is new ground, not old, and that there isn't already a world out there.
su3su2u1, debating this matter with Scott Alexander (Yvain), sums up a lot of their problems with the world view (which I am as familiar with as anyone who doesn't actually drink the Kool-Aid can be, having been on LW around four years and read not only the Sequences through twice but read literally all of LessWrong through from the beginning twice), which I largely agree with as a summary: https://storify.com/lacusaestatis/sssp-su3su2u1-debate
I'll quote one telling bit, which points out the level after Bayes:
> Heck, there are well defined problems where using subjective probability isn’t the best way to handle the idea of “belief”- when faced with sensor data problems that have unquantified (or unquantifiable) uncertainty the CS community overwhelmingly chooses Dempster-Shafter theory, not Bayes/subjective probabilities.
Do you remember the Sequences post mentioning the words "Dempster-Shafter"? Me neither.
(And then there's the use of "Bayesian" to mean things that nobody else uses the term for. As su3su2u1 puts it: "I suspect I’d be hard-pressed to write about probability theory in a way that wouldn’t fit some idea you cover by the word 'Bayesian.'")
Yudkowsky definitely gets credit as a good pop science writer. The habit of neologism, not so much. And definitely if he wasn't into the encapsulated, self-referential world that LW builds. In philosophy, Yudkowsky is the quintessential Expert Beginner: http://www.daedtech.com/tag/expert-beginner
untiltheseashallfreethem notes in http://untiltheseashallfreethem.tumblr.com/post/107159098431... : "I think Eliezer did a great service in writing these ideas up. But they are not his ideas, and I’m really worried that a lot of people read LessWrong, see that Eliezer is right about this stuff, assume he came up with it all, and then go on to believe everything else he says." And that's a serious problem when the good stuff is not original, and the original stuff is not good.
I haven't read everything on LessWrong, nor do I have time to keep up on the meta-discussion of Eliezer's neologism habits, but I can say that I've never thought that he invented any of the concepts that I learned through HPMOR or the sequences that I have read.
On the contrary, he seems very intent on citing the very books and people from which he learned these things. At least in my more limited experience. You definitely seem to have studied up on the issue much more than I.
It took way too much reading to realise there wasn't actually a "there" there, that none of the pointers-to-pointers-to-explanations actually resolved in the end. The evidence is pretty clear that I have way too much time on my hands.
>.g. "it's better for one person to be tortured for fifty years than for an extremely large number of people to each get a speck of dust in their eye"),
The number in question was 3^^^3 using Knuth's Up Arrow Notation. As he explains, that's a lot of people: (3^(3^(3^(... 7625597484987 times ...)))).
If we decide that one person being tortured for 50 years is worth a quick blink by all those other people, then a still tiny amount of people (say, 7625597484987) being tortured would be worth everyone else blinking furiously for their entire lives.
So it's not about deciding one person's fate. It's about being consistent, because if you make an exception for one person, that turns into several hundred billion, then that small sacrifice everyone else was to carry now has destroyed everyone else, too.
(I think the issue is that people tend to round down "speck of dust" to 0, then multiply, then find of course any amount of torture isn't worth 0.)
Personally, I take a more negative view and see the immense possibility for suffering as a reason that we should seek to destroy the entire multiverse. It's just not very nice otherwise.
I've thought about this and concluded that this thought experiment ignores the subjective way people experience reality. If a dust speck fell in your own eye, you would round that experience down to zero. You wouldn't even remember the event an hour later. I think this subjective experience of reality should be taken into account.
As for the part of the argument that dust specks in so many eyes would cause the death of a small fraction of the people, that's far from a given but for the sake of "steelmanning" let's assume that's true. At some point my own ethics place death as a higher utility outcome than prolonged suffering. How many deaths vs how much suffering? That's hard to quantify.
> I think the issue is that people tend to round down "speck of dust" to 0, then multiply, then find of course any amount of torture isn't worth 0.
Why not round it down to 0?
There's some level of discomfort in everyday life to start with that's effectively negligible for any normal person, simply because it's a prerequisite for interacting with the world (getting a raindrop in your eye, dealing with a minor wedgie, having an itchy nose, whatever).
If you round down before multiplying, then you get an invalid answer. First multiply it out. So in my example, torturing 7,625,597,484,987 people for 50 years, versus that many specks of dust for everyone else. (7 trillion is essentially 1, compared to 3^^^3, right?) 7 trillion specks of dust is enough to turn all those other lives into torture, as 7 trillion specks of dust is a few thousand a second, for a very long life.
That's why you can't round down first. Eliezer's entire intent there is to illustrate that by default, we suck at scale, we're scope insensitive.
Another way: If everyone in the US gave just a penny directly to a poor person, once in a lifetime, that person would be wealthy for life and be OK, right? A penny rounds down to zero, thus we can determine that the right action is to always give money directly to poor individuals, as this will cure poverty at a cost of nothing.
You're right that if the only choice, ever, in the entire multiverse was this one 50 years or 3^^^3 specks of dust, then sure, you don't need to follow the rules/math. But we're never going to be faced with just a single decision. Instead, we should be consistent and follow the numbers to determine utility.
> If you round down before multiplying, then you get an invalid answer.
Your argument here is based on the premise that the subjectively-inflicted-pain of "a whole lot of specks of dust in a person's eye" can be treated as a multiple of "one speck of dust in a person's eye", which to me is the point of absurdity that makes the whole thought exercise fall apart.
Yes, it's the sort of thing that makes people go to second-order utilitarianism.
(It's a bit like when someone says something absurd about Uber or Bitcoin and, when called out, says "But that's Econ 101!" Yes, but in second and third year you find out it's all a bit more complicated than that.)
That some arbitrarily minimal amount of subjective pain can be presumed to be actually distinguishable from the general background unpleasantness of being human in the first place.
Edit: Rephrased to remove gibberish resulting from temporary brain/keyboard disconnect.
For some reason HN won't let me reply to Houshalter's post, but...
> objectively
How can pain be objective? The universe doesn't give a shit that some lumpy sacks of meat dislike certain experiences, and exactly the same action can be unpleasant for some people but pleasant for others. (BDSM enthusiasts, for example, often actively seek out experiences that other people would consider painful.)
That depends on the context. If it's acupuncture, for example, it can be quite good, even if the precise sensation is exactly the same as the otherwise unpleasant case of poking myself with a needle while sewing.
It doesn't matter if it's "distinguishable". It's objectively worse. A pin prick might not be noticeable but a million is the worst torture imaginable.
You're assuming that dust specks across different people can be added, and then asserting that as evidence that dust specks across different people can be added. The addition is ridiculous.
In real-world AI, they tend to go for minmaxing, which avoids such absurdities. Minmaxing says "don't torture people, your dust speck has literally been defined to be insignificant."
That in real-life AI, they don't use simplistic utilitarianism.
You're dodging the core of the problem, as stated above: "However, when pushed too far, those tools tend to break down—but the rationalist answer to that breakdown is all too often to embrace the model and discount the reality." https://news.ycombinator.com/item?id=9204442
Just because you love your model doesn't make its conclusions at extremes true.
It's not so hard to explain: he writes well and generally has something interesting to say. This is more than enough to keep people reading.
That doesn't mean everything he writes stands up, but he's smart enough that you have to think a while about what might be wrong with his arguments, and that's entertainment enough.
I'm seriously unsurprised to find the token why-does-anyone-like-Yudkowsky-he-doesn't-have-a-PhD-and-hasn't-even-saved-the-world-yet criticism in a thread about his Harry Potter fan fiction.
He's a great writer, and that should be more than enough for the purposes of recommending a thing he wrote.
I left this out from the parent post because it's a subjective judgment, but I absolutely disagree that he's a "great" writer, or even anything really more than "serviceable" at best.
"Scientific parable fiction" is a pretty narrow genre (though I'd personally argue that Yudkowsky does a pretty poor job of it - for example, Harry constantly assuming things without testing them and just happening to be right because the author says so).
alexanderwales writes relatively short, punchy stories that explore specific academic and narrative themes, and, importantly, generally work extremely very well as stories even if you discount the thought experiment aspects.
> I'm seriously unsurprised to find the token why-does-anyone-like-Yudkowsky-he-doesn't-have-a-PhD-and-hasn't-even-saved-the-world-yet criticism in a thread about his Harry Potter fan fiction.
I don't think that this was the kind of criticism that was expressed, but on the other hand it's always easier to fight a straw man.
I haven't read all of the "Sequences," nor am I familiar with most details of the Less-Wrong-mindset's idiosyncrasies. However, I just re-read the whole book and I'm not sure either of those concepts have any bearing on this work.
There is a bit of assertion that "timeless physics" is representative of true reality, but my reaction was to look it up and find that it's an interesting fringe theory. For all I know, EY doesn't even assert that same theory's truth. Nothing harmful in sparking curiosity, especially in things such as statistics and logical biases, which is where HPMOR puts its emphasis.
Interesting. I still have no problem with it, of course, buttressed by my quick glance at that page to see, at the very top:
Warning: The central idea in today's post is taken seriously by serious physicists; but it is not experimentally proven and is not taught as standard physics.
Today's post draws heavily on the work of the physicist Julian Barbour, and contains diagrams stolen and/or modified from his book "The End of Time". However, some of the arguments here are of my own devising, and Barbour might(?) not agree with them.
Yeah. Despite the disclaimer, he then goes on to assume it's literally true. And, for relevance to this post, he does so in the story as well, in Chapter 28.
> There is no “true math of quantum mechanics.” [...] These are different mathematical formulations, over different spaces, that are completely equivalent.
> What Hariezer is doing here isn’t separating the map and the territory, its reifying one particular map (configuration space)!
>I also find it amusing, in a physics elitist sort of way (sorry for the condescension) that Yudkowsky picks non-relativistic quantum mechanics as the final, ultimate reality. Instead of describing or even mentioning quantum field theory, which is the most low-level theory we (we being science) know of, Yudkowsky picks non-relativistic quantum mechanics, the most low-level theory HE knows.
> So this is more bad pedagogy: timeless physics isn’t even a map, it's the idea of a map. [...] It seems very odd to just toss in a somewhat obscure idea as the pinnacle of physics.
