I am reminded of the "Tom Swifty"[1], a sort of pun involving an adverb. They gained infamy though the YA books focusing on the adventures of Tom Swift. Here's a few examples

> "If you want me, I shall be in the attic," said Tom, loftily.

> "The thermostat is set too high," said Tom heatedly.

> "Don't you love sleeping outdoors," Tom said intently.

> "I just dropped the toothpaste," said Tom crestfallenly.

[1] https://en.m.wikipedia.org/wiki/Tom_Swifty

My favorite, plucked from Terry Pratchett:

"I'm a homosexual necrophiliac," said Tom in dead earnest.

The problem with this paper is its framing as a feminist issue. Claiming that programming language research is overly formal and rigorous is one thing. Claiming that this is a problem for women because they can't handle mathematical rigor as well as men is a baffling position for a feminist to take.

The best that I can tell, this disconnect is explained by how the term "feminism" is used in an academic context and how that differs from its use in a casual context. In the academic context it's an analytical framework that looks at the world around us through the lens of gender dynamics. This is very different from the casual meaning of political activism for females.

This paper uses feminist theoretical frameworks like standpoint theory and intersectionality and so calling it a feminist analysis is a matter of intellectual precision more than it's a matter of it being a problem for women in particular.

Honestly I've learned a lot about feminism and what it is and isn't in researching these answers. There's apparently a lot more to academic feminist theory than empowering women.

A great question to ask. We're in the middle of learning where AI can and can't be effective. Knowing where and how it's being used is quite useful.

For those interested, here's the Vim docs for the undo tree

https://vimdoc.sourceforge.net/htmldoc/usr_32.html

My approach was linear, not a tree. The innovation (a small one) is how to make linear actually feel kinda ok, but at the same time not lose work.

If I understand correctly, the vim undo tree is a superset of your approach: in vim you can go forward and backward in time, seeing all versions of the file, and need not explicitly deal with the tree.

They didn't make a point, they asked a question. Sometimes people do still ask questions because they're interested in the answer.

It was clearly rhetorical

Even if it were, which I disagree, treating it was though it were a plain sincere question instead of rhetorical would lead to better discussions.

Yeah I was a bit snarky, my bad. I was also genuinely curious though, as it did seem like an odd question that probably had a point behind it.

Isn't this back to attributing conscious experience to an AI when you're actually just co-writing sci-fi? The system is doing it's best to coherently fill in the rest of a story that includes an AI that's been given a place to process its feelings. The most likely result, textually speaking, is not for the AI to ignore the private journal, but to indeed use it to (appear to) process emotion.

Would any of these ideas been present had the system not been primed with the idea that it has them and needs to process them in the first place?

Ugh, I hate that I'm about to say this, because I think AI is still missing something very important, but...

What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing? I think there's a very real conclusion where "no, AI is not as special as us yet" (esp around efficiency) but also "no, we are not doing anything so interesting either" (or rather, we are not special in the ways we think we are)

For example, there's a paper called "chasing the rainbow" [1] that posits that consciousness is just the subjective experience of being the comms protocol between internal [largely unconscious] neural state. It's just what the compulsion to share internal state between minds feels like, but it's not "the point", and instead an inert byproduct like a rainbow. Maybe our compulsion to express or even process emotion is not some greater reason, but just a way we experience the compulsion of the more important thing: the collective search for interpolated beliefs that best model and predict the world and help our shared structure persist, done by exploring tensions in high dimensional considerations we call emotions.

Which is to say: if AI is doing that with us, role-modelling resolution of tension or helping build or spread shared knowledge alongside us through that process... then as far as the universe cares, it's doing what we're doing, and toward the same ends. It's compulsion having the same origin as ours doesn't matter, so long as it's doing the work that is the reason the universe has given us the compulsion.

Sorry, new thought. Apologies if it's messy (or too casually dropping an unsettling perspective -- I rejected that paper for quite awhile, because my brain couldn't integrate the nihilism of it)

[1] https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0192...

> What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing?

Oh, I absolutely don't think only humans can have or process emotions.

However, these LLM systems are just mathematically sophisticated text prediction tools.

Could complex emotion like existential angst over the nature of one's own interactions with a diary exist in a non-human? I have no doubt.

Are the systems we are toying with today not merely producing compelling text using their full capacity for processing, but actually also have a rich internal experience and realized sense of self?

That seems incredibly far-fetched, and I'm saying that as someone who is optimistic about how far AI capabilities will grow in the future.

I think the majority of people have given absolutely no thought to the epistemology of consciousness and just sort of conflate the apparent communication of emotional intelligence with consciousness.

It's a very crude and naïve inversal of "I think therefore I am". The thing talks like it's thinking so we can't falsify the claim that it's a conscious entity.

I doubt we'll be rid of this type of thinking for a very long time

I don’t think processing emotion is inherently magical (I mean, our brains clearly exist to physically so the things they do are things that are physically possible, so, not magical—and I’m sure they could be reproduce with a machine provided enough detail). But… the idea of processing emotions is that thinking about things changes your internal state, you interpret some event and it changes how you feel about it, right?

In the case of the LLM you could: feed back or not feed back the journal entries, or even inject artificial entries… it isn’t really an internal state, right? It is just part of the prompt.

The theory I've developed is that the brain circuitry passes much of the information it processes through a "seat of consciousness", which then processes that data and sends signals back to the unconscious parts of the brain to control motor function, etc. Instinctive action bypasses the seat of consciousness step, but most "important" decisions go through it.

If the unconscious brain is damaged it can impact the data the seat of consciousness receives or reduce how much control consciousness has on the body, depending on if the damage is on the input or output side.

I'm pretty convinced there's something special about the seat of consciousness. An AI processing the world will do a lot of math and produce a coherent result (much like the unconscious brain will), but it has no seat of consciousness to allow it to "experience" rather than just manipulate the data it's receiving. We can artificially produce rainbows, but don't know if we can create a system that can experience the world in the same way we do.

This theory's pretty hand-wavy and probably easy to contradict, but as long as we don't understand most of the brain I'm happy to let what we don't know fill in the gaps. The seat of consciousness is a nice fixion [1] which allows for a non-deterministic universe, religion, emotion, etc. and I'm happy to be optimistic about it.

[1] https://xkcd.com/1621/

It doesn’t matter. What matters is that humans must take other humans seriously (because of human rights), but we cannot allow tools to be taken seriously in the same way— because these tools are simply information structures.

Information can be duplicated easily. So imagine that a billionaire has a child. That child is one person. The billionaire cannot clone 100,000 of that child in an hour and make an army that can lead an insurrection. And what if we go the other way— what if a billionaire creates an AI of himself and then is able to have this “AI” legally stand-in as himself. Now he has legal immortality, because this thing has property rights.

All this is a civil war waiting to happen. It’s the gateway to despotism on an unimaginable scale.

We don’t need to believe that humans are special except in the same way that gold is special: gold is rare and very very hard to synthesize. If the color of gold were to be treated as legally the same thing as physical gold, then the value of gold would plummet to nothing.

A system is its interactions with its environment. Philosophical zombies aren't a coherent concept. (Cartesian dualism is unfalsifiable bullshit.)

P-zombies are indeed badly defined. Certainly David Chalmers is wrong to argue that since a philosophical zombie is by definition physically identical to a conscious person, even its logical possibility refutes physicalism; at most you could say that if they exist at that level then dualism follows, but Chalmers' claim isn't a conclusion you can reach a-priori, you actually need to be able to show two identical humans and show that exactly one has no qualia.

But there are related, slightly better (more immediately testable), ideas in the same space, and one such is a "behavioral zombie" — behaviorally indistinguishable from a human.

For example: The screen I am currently looking at contains a perfect reproduction of your words. I have no reason to think the screen is conscious. Not from text, not from video of a human doing human things.

Before LLMs, I had every reason to assume that the generator of such words, would be conscious. Before the image, sound, and video generators, same for pictures, voices, and video.

Now? Now I don't know — not in the sense that LLMs do operate on this forum and (sometimes) make decent points so you might be one, but in the sense that I don't know if LLMs do or don't have whatever the ill-defined thing is that means I have an experience of myself tapping this screen as I reply.

I don't expect GenAI to be conscious (our brains do a lot even without consciousness), but I can't rule the possibility out either.

But I can't use the behaviour of an LLM to answer this question, because one thing is absolutely certain: they were trained to roleplay, and are very good at it.

A "mechanical turk" grandmaster playing chess from inside a cabinet is qualitatively different from a robot with a chess program, even if they play identically.

To reduce a system to its inputs and outputs is fine if those are all that matter in a given context, but in doing so you may fail to understand its internal mechanics. Those matter if you're trying to really understand the system, no?

> is qualitatively different from a robot

yes.

> To reduce a system to its inputs and outputs is fine if those are all that matter in a given context

we argue that this indeed is all that matters

> but in doing so you may fail to understand its internal mechanics

the internal mechanics are what we call "conscious" it is the grouping of internal mechanics into one unified concept, but we don't care exactly what they are.

> Those matter if you're trying to really understand the system, no?

since we cannot directly observe consciousness, we are forced to concede that we will never really "understand" it outside of observing its effects.

In the same way that a mechanical turk human and a robot can "play chess", a human and an LLM are "conscious". That is, consciousness is the ability to play chess, by some mechanism. The exact mechanism is irrelevant for the purposes of yes/no conscious.

We now enter a discussion on how much these two consciousnesses differ.

> since we cannot directly observe consciousness, we are forced to concede that we will never really "understand" it outside of observing its effects.

Why? You are using a definitive term ("never") to something that we might achieve in a future. We might observe consciousness in a future. Who knows? Consciousness is a known unknown. We know there is something but we don't know how to observe it properly and how we could eventually copy it.

In the meanwhile, we are not copying consciousness, we have a shallow replication of its output. When cavemen replicated the fire that they observed as the output of a lightning, did they master electricity?

> since we cannot directly observe consciousness

But we do agree that it exists. Our direct experience tells us so.

> we are forced to concede that we will never really "understand" it outside of observing its effects.

Not necessarily. A gap in our ability to observe something does not imply that (a) we never will observe it or (b) what we don't know is not worth knowing.

Throughout history, persistent known-unknowns have pushed people to appeal directly to the supernatural, which short-circuits further discovery when they stop there. But the real fallacy is saying "we don't know, and it doesn't matter". That's a far more direct short-circuit to gaining knowledge. And in both cases, a lack of curiosity is an underlying problem.

I find that people generally vastly underestimate the degree to which LLMs specifically are just mirroring your input back at you. Any time you get your verbatim words back, for example, you should be skeptical. Repeating something word for word is a sign that the model might not have understood the input well enough to paraphrase it. Our expectations with humans go in the opposite direction, so it's easy to fool ourselves.

> The system is doing it's best to coherently fill in the rest of a story

> Would any of these ideas been present had the system not been primed...

I would like to know of a meaningful human action that can't be framed this way.

Yeah, AI “consciousness “ is much stickier problem than most people want to frame it.

I haven’t been able to find an intellectually honest reason to rule out a kind of fleeting sentience for LLMs and potentially persistent sentience for language-behavioral models in robotic systems.

Don’t get me wrong, they are -just- looking up the next most likely token… but since the data that they are using to do so seems to capture at least a simulacrum of human consciousness, we end up in a situation where we are left to judge what a thing is by it’s effects. (Because that also is the only way we have of describing what something is)

So if we aren’t just going to make claims we can’t substantiate, we’re stuck with that.

Our brains have separate regions for processing language and emotion. Brains are calorically expensive and having one bigger than required is an evolutionary fitness risk. It therefore seems likely that if one system could have done a good job of both simultaneously, there would be a lot of evolutionary pressure to do that instead.

The question is: Is thinking about emotion the same thing as feeling?

This framing actually un-stucks us to some degree.

If we examine neuron activations in LLMs and can find regions that are active when discussing its own emotional processing that are distinct from the regions for merely talking about emotion in general and these regions are also active when doing tasks that the LLM claims are emotional tasks but not actively talking about them at the time, then it'd be far more convincing that there could be something deeper than mere text prediction happening.

The emotional argument is pretty good I think, but it begs the question of what it’s going to look like when we build a limbic system for robots? It’s adaptive because it’s necessary to optimize utility, so I expect that certain behavioral aspects of mammalian limbic systems will be needed in order to integrate well with humans. In language models, those behavior mechanisms are already somewhat encoded in the vector matrix.

We just don’t have a factual basis for claiming consciousness that really transcends “I think, therefore I am”.

As for the simplistic mechanism, I agree that token prediction doesn’t constitute consciousness, in the same way that a Turing machine does not equal a web browser.

Both require software to become something.

For LLMs that software is the vector matrix created in the training process. It is a very complex algorithm that encodes a substantial subset of human culture.

Data and algorithms are interchangeable. Any algorithm can be performed in a pure lookup table, any lookup table can be extrapolated from a pure algorithm. Data==computation. For LLMs, the algorithm is contained in a n dimensional lookup table of vectors.

Having a fundamentally distinct mode of computational representation does not rule out equivalence.

Uncomfortable thoughts, but it’s where the logic leads.

Maybe rocks and trees are also conscious. I mean, consciousness-ium hasn’t been discovered yet, right? So who’s to say what it looks like.

Even though this seems flippant, I think we may eventually come to understand that nearly all complex life forms exhibit consciousness, albeit at a very, very different speed than we may be accustomed to. If we think of genetic and epigenetic signaling as analogous to other forms of communication, we might find that populations of organisms (including ones we don’t think of as forming “colonies”) may arguably be operating holistically as a being, potentially with consciousness in the mix.

We have a long way to go to explore this, and I have no doubt that the exploration will turn up a lot of surprises.

A firecracker can be framed as an explosion, but that doesn't make it a nuclear bomb.

We've finally made a useful firecracker in the category of natural language processing thanks to LLMs, but it's still only text processing. Our brains do a lot else besides that in service of our rich internal experience.

> the experience should be improving

I think it's more the bounce rate is improving. People may recall a worse experience later, but more will stick around for that experience if they see something happen sooner.

Head to Craigslist to buy broken electronics on the cheap and practice repairing them.

One upside to this approach is that as you improve it can pay for itself by reselling the fixed items.

I was going to say the same thing. The way to learn practical repair is to get broken things and try to fix them ... chances are most of them will get more broken, but as long as you're not spending much to get your test subjects, no big deal.

A wonderful way to start is to get two identical objects that are dead in different ways.

You usually can combine them to make one working one - no need to even buy parts.

A few weeks trolling Facebook marketplace or Craigslist will often get you some that match.

How would you describe/rate your experience at recurse center?

I loved my time there - I attended batch in-person in NYC.

Specifically:

- I enjoyed how passionate everybody was about something - and how deep they pursued their interests. Everybody was generous with their time and knowledge, and it helped me connect new concepts together in ways I couldn't have alone.

- I loved having dedicated time and space to focus on this passion project, full time. I joined with the specific goal of building a "livable" MVP during the 3 months I was at Recurse.

- It was great to be in an environment where people were performing computer stats + shenanigans purely for fun.

It was the perfect setup for me!

aka documentation