Just a tiny rant: In my view complex numbers are really about the concept of "orthogonality". The complex 'dimension' is orthogonal to the 'real' dimension, but anything in reality that's a continuum of values can be seen as a dimension, and therefore each one must have an orthogonal. That is, whenever you have a direction in a higher dimensional space (regardless of dimensionality) any vector will have a normal direction (perpendicular direction).
What basic complex numbers represent is a way of doing rotations where something moves from one direction towards it's orthogonal. That's what Euler's Formula is about also, which shows the relationship of 'e' and 'i' in this of course.
Now what Quaternions represents is the realization that if complex numbers have two components (real, imaginary) then we can treat each of those as a base vector and find a sort of 'next level up' orthogonality to each one individually.
I'm not good enough at math/geometry to know if this kind of 'next level up' bifurcation of dimensionality extends up past Quaternions or not (like something called Octernions, 16ions, 32ions, 64ions, etc), but it seems like is would?
Octonions and so on up are indeed a thing, but I don't think they do what you want. Even aside from the fact that they're restricted to power-of-2 dimensions, their algebraic properties get worse as you iterate the Cayley-Dickson process. The octonions aren't even an associative algebra, although they do have some weaker associativity properties which I'll skip detailing here. The quaternions are as far as most mathematicians are willing to go -- non-commutativity is commonplace, but who wants to deal with non-associativity?
But while the octonions at least have some mathematical relevance (they're actually connected to various exceptional objects, such as the exception Lie group G_2!), the sedenions and beyond basically don't. They have a tiny bit of associativity but not enough that they connect to any things or that hardly anyone wants to study them -- and worse yet, there are zero divisors so cancellation (ab=ac => b=c for nonzero a) doesn't even hold. (Inverses exist, yes, but without associativity, inverses don't imply cancellation! And therefore aren't much use.)
As another commenter mentioned, what you might be looking for instead if it's orthogonality you're focused on is Clifford algebras (aka geometric algebra). However, if you want to get the complex numbers or quaternions out of it, you'd need to use a negative-definite quadratic form -- if you use a positive-definite one, you'd instead get the split-complex numbers, which are much less interesting (and you'd get something similar instead of the quaternions).
From your post I can tell you're way better at math/geometry than me, but I understood 80% of that. :)
Cayley-Dickson is interesting especially for Physics of course, because it brings in the concept of 'variable dimensions'. I think the flattening of objects, and the stopping of clocks (in Relativity), due to Lorentz effects in Minkowski space both on Black Hole Event Horizons and for objects approaching light speed (anywhere Lorentz holds) is, at the limits, ultimately the loss of a dimension, which would be my overall interpretation of what Cayley-Dickson is about too, in very broad terms.
So if Minkowski space is 4 dimensional, there would be some geometry for a 5-Dim Minkowski and it would use Octonians maybe, and that would be the geometry of the universe our universe is "embedded in"...I mean assuming of course you believe our universe is a Black Hole and we are all on an Event Horizon embedded in a 5D universe. Ya know, as one does. lol.
> I'm not good enough at math/geometry to know if this kind of 'next level up' bifurcation of dimensionality extends up past Quaternions or not (like something called Octernions, 16ions, 32ions, 64ions, etc), but it seems like is would?
Octonions and up (more generally known as hypercomplex numbers) exist, but every time you pull the "double dimensions by adding more imaginary components" trick[0], you lose another useful property.
Real to complex loses total ordering. Complex to quaternion loses commutativity. Quaternion to octonion loses associativity (but they are at least alternative). The sedenions aren't even alternative, and they have zero divisors to boot.
You can also generalize hypercomplex numbers to the study of Clifford algebras.
Algebraically, and also in group theory, it's exactly that. A complex plane is re-interpreted as a half-way mirror operation on an orthogonal axis or quadrature plane (eg the imaginary axis in the case of the 2D Argand plane). Which is what you you get when you multiply by i in complex numbers or i/j/k in quaternions - a 90 degree rotation. The correct way to do arbitrary rotations in this case is to use an exponential process, and then you get Euler's equation, the quaternion symmetry operation or, in general, the exponential map of an infinitesimal transformation in a Lie group.
In higher dimensions you get other type of (sometimes weird) operations, related to the Cartan–Dieudonné theorem.
I don't agree. Complex numbers are the algebraic closure of the reals. Or the quotient of the real polynomial ring by (x^2+1=0). Or whatever other construction. The multiplication rule is the essence of C.
Orthogonality is captured linear algebra over R^2, but R^2 isn't a field or an algebra.
I think there is still a geometric viewpoint you can bring to the multiplicative structure of C. For example there is the extremely natural homeomorphism between unit C and SO(2). And C minus origin to (R+, SO(2)). It’s completely intuitive for mathematicians to say that 1 and i are separated by 90 degrees.
Indeed, there is a polar coordinates representation of the complex numbers with the nice property that when you multiply you multiply lengths and add angles.
I really like the welch labs series on imaginary numbers which covers the first part of what you talk about -- leveling up the notion of what a complex number is. Though his focus was more on solving simple equations with no real roots, but really detailing how/what is really going on.
It is a great precursor to then thinking about quaternions
That exact video is the one that always comes to my mind when thinking of YT videos on this! I've seen it years ago. Definitely worth a watch for anyone who hasn't see it!
Also the (provocatively titled) "Let's Remove Quaternions from every 3d Engine" [1]
Spoiler alert: rotors are mechanically identical to quaternions, while being easier to understand. If you understand rotors, you understand quaternions. You can fit the laws you need to understand rotors on a business card.
Plus, rotors abstract to higher and lower (well, there's only one plane and its two respective orientations in 2d, but still) dimensions.
Complex numbers as planes (bivectors in GA parlance) has been the most mind-opening mathematical concept I've been exposed to in the last decade. The associated geometric product has helped me better understand concepts (like "handedness") that troubled me during undergrad engineering.
I had never even heard of rotors! Thanks for this. I watched that video. The video doesn't really explain how it extends to higher dimensions tho, that I could discern.
I wonder how/if any of this can be applied to LLMs 'Semantic Space'. As you might know, Vector Databases are used a lot (especially with RAG - Retrieval Augmented Generation) mainly for Cosine Similarity, but there is a 'directionality' in Semantic Space, and so in some sense we can treat this space as if it's real geometry. I know a TON of research is done in this space, especially around what they call 'Mechanistic Interpretability' of LLMs.
> The video doesn't really explain how it extends to higher dimensions tho, that I could discern.
The neat thing is that it "extends" automatically. The math is exactly the same. You literally just apply the same fundamental rules with an additional basis vector and it all just works.
MacDonald's book [1] proves this more formally. Another neat thing is there are two ways to prove it. The first is the geometric two-reflections-is-a-rotation trick given in the linked article. The second is straightforward algebraic manipulation of terms via properties of the geometric product. It's in the book and I can try to regurgitate it here if there's interest; I personally found this formulation easier to follow.
If you really want your mind blown, look into the GA formulation of Maxwell's laws and the associated extension to the spacetime (4d) algebra, which actually makes them simpler. That's derived in MacDonald's book on "Geometric Calculus" [2]. There's all kinds of other cool ideas in that book like a GA formulation of the fundamental law of calculus from which you can derive a lot of the "lesser" theorems like Green's law.
Take all of this with a grain of salt. I'm merely an enthusiast and fan, not an expert. And GA unfortunately has (from what I can tell) some standardization and nomenclature issues (e.g. disagreement over the true "dot product" among various similar but technically distinct formulations)
> I wonder how/if any of this can be applied to LLMs 'Semantic Space'.
Yeah, an interesting point. Geometric and linear algebra are two sides of the same coin; there's a reason why MacDonald's first book is called _Linear and_ Geometric Algebra. In that sense, Geometric Algebra is another way of looking at common Linear Algebra concepts where algebraic operations often have a sensible geometric meaning.
Interesting ideas there thanks. I do know about that Maxwell derivation that involves Minkowski space, Lorentz transform consistency, etc, although I haven't fully memorized how it works, so that I can conjure up how it works from memory. I don't really think in equations, I think in visualizations, so I know a lot more than I can prove with math. You're right it's mind-blowing stuff for people like us that are interested in it.
I noticed several people mentioned Karpathy already, but I wanted to include that his tiny "Micrograd" project (see Youtube Video and GitHub) is a great introduction to Neural Nets (Multilayer Peceptron), which is at the core of [most] machine learning of course.
All closed-source models censor to the liking of their investors. Open Source models are generally less censored, but yeah DeepSeek is censored for sure.
It's going to be funny watching the AI bro's turn anti-communism while they also argue why private ownership (such as copyright) is bad and they should be able to digest every book, every magazine, every piece of art in history with zero compensation so that they can create their tools.
Everything is built on previous knowledge. And at some point, things need to transition to public domain and the compensation has to end. Do artists that draw a car, compensate the first guy that drew a wheel? Do kids with crayons need to compensate the inventors of specific pigments for example. It would get absurd.
> they should be able to digest every book, every magazine, every piece of art in history with [as if] zero compensation so that
? That is the state of facts. «So that» is "so that you build up". It does not limit machines: it applies to humans as well ("there is the knowledge, when you have time, feed yourself"). We have built libraries for that. It is not "zero compensation": there is payment for personal ownership of the copy - access is free (and encouraged).
Laws have to change when technology changes. AI will benefit all of humanity, so I'm someone who believes AI should be allowed to train on copyrighted materials, because it's better for society.
However, like you're getting at, there are people who would say personal rights always outweigh society's rights. I think we can get rid of copyright law and still remain a free market capitalist economy, with limited government and maximal personal freedoms.
'Some people's property has to become everyone's property because AI'. Should Microsoft's software be free to everyone because humanity would benefit? Nintendo's? Oracles? Or only movie studios, musicians, and authors property rights should lose protection?
If an AI can look at a bunch of Picasso paintings and "learn" how to then replicate that same style, I don't think that's stealing. And I think the same concept applies to the written word.
However even if you were correct, would you be willing to trade copyright law for having a cure for most diseases? I would. Maybe by allowing 1000s of people to sell books, you've condemned millions of people to death by disease right? Can you not see that side of the argument? Sometimes things are nuanced with shades of gray rather than black and white.
Maybe by having copyright law we have allowed the authorship of books to flourish and critical mass to drive down the costs of books and allowed people to dedicate themselves to writing books as a profession, or made giving up weekends on a passion project worth completing. Maybe the world you want is less literate/less thought provoking because people can't feed themselves on 'your work is free' resulting in less being written because people who would have been authors are no longer rewarded.
All I know is society decided that copyright was worth the tradeoff of having people release their works and now huge corporations want to change the rules so that they can use those works to creative a derivative that the corporation can profit from.
I think both copyright law and AI consumption of copyrighted material can coexist peacefully. I can learn from what I read and then process that information to create my own novel works, and I think that's what LLMs are doing too.
If LLMs were just doing data compression and then spitting out what they memorized then that would violate copyright, but that's now how it works.
Nope, companies always do what's in their "self interest", whereas the Open Source community is concerned with improving the human condition for all. This applies especially to AI/LLM censorship vs freedom.
The fact that some kind of tags or Key/Value storage as attributes on files, has been missing until 2025 (and still is) seems so bizarre to me. Our file systems have hardly changed since the 1960s. We get filename, timestamp, filesize, and that's about it. Pathetic.
Imagine the opportunities if a folder structure could represent a "document" where each file represents a paragraph, or image, chunk of that document. We would be able to do 'block-based editors' (like content management systems, or Jupyter Notebooks) without having to have some large XML file holding everything.
Even if we had simple "ordinal" (ordered position) for files that would open up endless opportunities for innovation in the 'block-editor' space, but sadly File Systems development has been frozen in place for decades.
I'd sort of invert that and say it's better to use LLMs to just generate tons more test cases for the SQL DBs. Theoretically we could use LLMs to create 100s of Thousands (unlimited really) of test cases for any SQL system, where you could pretty much certify the entire SQL capability. Maybe such a standardized test suite already exists, but it was probably written by humans.
At that point, you'd get a ton more value from doing Property Testing (+ get up and running faster, with less costs).
If I'd had to have either code or tests generated by a LLM, I'd manually write the test cases with a well-thought out API for whatever I test, then have the LLM write tests that implements what I thought up, rather than the opposite which sounds like a slow and painful death.
I hadn't heard of "Property Testing" if that's a sort of term of art. I'll look into it. Anyway, yeah in TDD it's hard to say which part deserves more human scrutiny the tests or the implementations.
Are you sure that LLMs, because of their probabilistic nature, would not bias against certain edge cases. Sure, LLMs can be used to great effect to write many tests for normal usage patterns, which is valuable for sure. But I'd still prefer my edge cases handled by humans where possible.
I'm not sure if LLMs would do better or worse at edge cases, but I agree humans WOULD need to study the edge case tests, like you said. Very good point. Interestingly though LLMs might help identify more edge cases us humans didn't see.
Or due to their power, they've already secretly been taken over by the US Gov't. That's not really a "big conspiracy theory" at this point. I was mocked by the left for years for saying that the Gov't was involved in Facebook censorship. Turns out I was right. The biggest battle our Gov't has to wage is the battle for hearts and minds, and the control of information, and so they're trying to get in as deeply rooted as possible with every big AI company.
To be clear I wasn't blaming just the Gov't for all the censorship, because 99% of Facebook employees (including Zuck himself) were strongly in favor of censoring all conservative viewpoints, as well, and were in lock-step with Big Gov't controlling speech.
Zuck recently tried to blame it all on the FEDs (on JRE podcast) but he was obviously lying because Facebook even built a special portal for the FEDs to log into, for moderating/controlling the public, so he was the ring-leader of all the censorship, for about a decade.
my God what social media does to people to write and believe stuff like this… hopefully one day everything will be banned… amazing to read this - just amazing what seemingly normal human being can be made into believing!
99% of Facebook employees (including Zuck himself) were strongly in favor of censoring all conservative viewpoints, as well, and were in lock-step with Big Gov't controlling speech.
how can you write this with the straight face when top like 100 accounts spreading shit on Facebook are right-wing nutcases. Give me on popular left-wing nutcase on Facebook?! believing that conservative views on any social media platform were ever being supressed is downright craaaaazy :)
Sorry dude. Nobody believes any of that in 2025. The claim that conservatives weren't being singled out on Social Media is laughable nowadays. All the companies have ADMITTED doing it. The Gov't has ADMITTED doing it.
I mean, conservatives were getting censored/attacked even for true things like saying COVID might have leaked from a lab, that there are only two sexes, that Hunter Laptop was real, that FBI lied on FISA warrant, that people meds are have horse-versions of them, Biden is senile, and on and on and on for countless other things.
And yet Twitter is censoring progressive viewpoints and even skewed the algo toward promoting a certain political candidate and yet not a single word from those free speech warriors.
I'm generally in favor of Free Speech, but after Silicon Valley censored conservatives for a decade, during the entire Cancel Culture era, I think for them (liberals) to get to experience what it feels like to be censored themselves is probably a good thing. A good learning experience for them to begin to understand first hand how it feels, and what they did to others, without remorse, and with ill intent, for a decade.
So I say to all liberals complaining about Twitter censorship: "Turnabout is fair play" and "You deserve it, because you invented it."
Every Democrat in the DC Swamp is quaking in their boots right now out of fear that the Republicans might do to them what they did to the Republicans. They expect it to happen, because being so revenge-oriented themselves, they know it's what they'd do.
The difference is that, unlike liberals, Conservatives (especially MAGA) are God-fearing law-and-order abiding people, not motivated by hate, and not the evildoers that they were falsely accused of being, by the insane left and their Senile POTUS, for the past 8 years.
So don't worry, Democrats have nothing to fear but the exposing of their hypocrisy, and more loss of reputation as more truths come out.
To me this just shows that Perplexity is definitely being controlled by some other force other than AI developers, and different from what they publicly would admit to. Perhaps the US Gov't is involved? ...because I don't think any other investors would be able to hide their true funding source and motives quite like the Gov't.
The problem with atproto is that it's rediculously complicated. Every detail in the spec done in the most difficult and stilted way possible.
They could've made something much more like Nostr be at the core of it all, so that the barrier to entry is small for people wanting to write their own implementations, but the developers/designers of atproto put very little value in simplicity. They wanted everything to be as powerful as possible at every single layer, which means far too many levels of abstraction, super heavy-weight implementations, and stacks upon stacks of specs that are hard to unravel, etc.
Anyone can learn Nostr in minutes. To learn atproto you need weeks.
Can nostr be used for the same form of social network as bluesky (ie, a twitter clone)? It seems that it would only show messages from people the user explicitly follows, for example, and not replies by a third user.
Also it's distasteful to do any sort of content addressing on json data. One would think they'd learn and use CBOR after seeing secure scuttlebutt, but no? Now you have to worry about only sending text payloads, escaping some characters, avoiding whitespace when printing the json... Guaranteed to be a source of bugs...
1) It's up to the relay and the software how much of the firehose of posts to present to users. Your client is in control if whether you see posts from your follows (who you follow), or the entire world.
2) One of the things Nostr did get [slightly] wrong was the hashing of the JSON. It's pretty straight-forward to sort properties, and remove spaces, to create canonical JSON that can easily be hashed, but (and I forgot the specific reason) Nostr made it where you can't directly store Nostr in IPFS (for example) and have the post hash/ID be identical to the IPFS CID of the canonical JSON. They missed that opportunity because fiatjaf was not well enough versed in IPFS, so he got that a bit wrong.
All that being said I am still a fan of Nostr. It is far better than other Social Media protocols imo.
I think they should have not reinvented the wheel ;-) and use dCBOR42, which has an actual canonical form. But somehow people like to use json in places where it's shown again and again it's a terrible choice.
The problem with CBOR is that once people relate it to IPFS their will be massive push-back, because IPFS is seen as too complex. I know this is true because I saw it happen. To these kids even XML is deemed "too complex", thus their love affair with JSON instead.
The reason Social Media protocols need to be kept simple is precisely for this reason. Getting developers to all agree on things is next to impossible. So the way to combat that is by removing all those "things", and go with the simplest design that's workable. It's almost like politics in that it's "The Art of the Possible". And to be "possible" in this context means universal acceptance.
Maybe you're right, that's quite sad. I found dasl.ing to be quite nice and simple as a foundation for dCBOR42, without needing the full ipfs craziness. Oh well.
ok. And you're right too that CBOR is the "correct" thing to use (despite what I said about acceptance), if we wanted to do it right. CBOR could be the only "complex" piece. Everything else could be Nostr-like (i.e. simple, and using relays).
It's not so bad. I managed to create a minimal implementation of a PDS for a Bluesky bot that can make simple text-based posts, and it only took a couple of days. Kind of lost interest in it after that but it was reasonably straightforward to iterate on. The trickiest bit was getting the subscribeRepos websocket to work, mostly because the documentation was unclear.
Maybe so but I bet you 90% of the code you were running was already written because it was in a library right? So you didn't really have to understand it. You were just running someone elses code.
With Nostr, for example (or even RSS), you can fully understand it from end to end, in minutes. As a former IPFS deloper myself I can assure you in just 2 days you didn't even understand the CAR format of a repo, unless you had prior experience.
I don’t know, as an IPFS developer you might be discounting what prerequisite knowledge you possessed while hammering out a nostr implementation.
I looked at nostr but lost interest when I noticed there was no provision for key rollover. I guess that’s fine but ephemeral identities. Is there a concept of using a domain name as a handle like bluesky? It’s been a few years so maybe it’s worth a second look.
I always considered the permanent keys/identities of Nostr to be a desirable thing. I had basically invented Nostr before fiatjaf did. It's a trivially simple "concept".
Key rotation is an unnecessary complexity, imo. I never heard of anyone wanting to use a domain name as identity in Nostr, but there's probably a NIP for that where a domain can prove it owns the private key, and be used as an identity.
I mainly quit Nostr development because it was all essentially controlled by 'fiatjaf', and he was making bad decisions, and a childish intolerable arrogant person in general whenever people asked him to justify those decisions.
The one thing that completely turned me off nostr is the idea that my identity is tied to my private key and that it can not be recovered. I'd guess that the reason you don't hear from people like me is that we simply don't bother to work with such a boneheaded design.
I always felt like in the age of cryptocurrencies, people can be expected to protect their private key, and yeah pretty much if they lose it, they lose their data. I mean that's how encryption works also. No key, no data. I don't call that boneheaded. I think it's a feature not a bug.
Nostr is so far ahead it's not funny, in part because of this, in part because it's actually decentralised, in part because of already existing features built in to almost every client, like zaps.
I don't understand why anyone would invest their time in ATProto over Nostr, and don't know anyone who has studied both and has.
When BlueSky was being designed I was involved in the developer discussions and continually urged them to make simple RSS be the least common denominator of their entire spec, and build on top of RSS rather than trying to replace RSS. They listened to a lot of what I said, including my "Repository" concept where every user basically has a big repository of IPFS content addressable elements, and incorporated that into their design, which I was glad to see, but BlueSky failed miserably on the "KISS" principle, because they made every detail super complicated. It could've been RSS-compatible, and that would've changed the world, and revived RSS, which is badly needed, but sadly they were unable to see the wisdom in that.
EDIT: And most of them (BlueSky devs) indeed were far left-leaning progressives who were much more concerned with censorship than freedom of information (this being around 2020 to 2022 Silicon Valley mindset), so they continually wanted to impose lock-downs and controls on the flow of information, rather than fostering principles of openness and freedom like what RSS is all about.
Yeah, I've written an ActivityPub implementation of my own. Very familiar with Mastodon too of course. RSS is such low hanging fruit and so obvious a thing to use as the basis for social media posts.
What basic complex numbers represent is a way of doing rotations where something moves from one direction towards it's orthogonal. That's what Euler's Formula is about also, which shows the relationship of 'e' and 'i' in this of course.
Now what Quaternions represents is the realization that if complex numbers have two components (real, imaginary) then we can treat each of those as a base vector and find a sort of 'next level up' orthogonality to each one individually.
I'm not good enough at math/geometry to know if this kind of 'next level up' bifurcation of dimensionality extends up past Quaternions or not (like something called Octernions, 16ions, 32ions, 64ions, etc), but it seems like is would?
reply