Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
New sphere-packing record stems from an unexpected source (quantamagazine.org)
245 points by pseudolus 8 hours ago | hide | past | favorite | 110 comments





I have trouble explaining to my parents how my job is a real thing. I can only imagine trying to explain ‘I study shapes, but only ones that don’t jut inwards’.

I've found it's best to explain my job using unintelligible jargon.

There are three choices, really:

You can give a quick explanation in terms they understand, which makes your job sound easy and makes them wonder how anybody gets paid to do it.

You can explain what you do and why it's important in terms they understand, but it'll take so long they'll get bored and wish they hadn't asked.

Or you can give a quick explanation using jargon that they don't understand, which will leave them bored but impressed, which is the best of the bad options.


When I meet people who immediately use hyper-specific jargon with strangers, I either distrust them, or assume they’re not emotionally intelligent (because it’s a choice demonstrates little respect for the person they’re addressing). It also projects that they may be compensating for some emotional insecurity on their own end, trying to assert intellectual “superiority” in some way.

The first option (explaining things simply) might make your job sound easy to a very small minority of extremely uneducated, under-stimulated people, who also have unaddressed insecurities around their own intelligence. But that’s not most humans.

Moderately-to-very intelligent people appreciate how difficult (and useful) it is to explain complex things simply. Hell, most “dumb” people understand, recognize, and appreciate this ability. Honestly, I think not appreciating simple explanations indicates both low mathematical/logical and social/emotional intelligence. Which makes explaining things simply a useful filter for, well… people that I wouldn’t get along with anyway.

With all that said, I prefer to first explain my job in an “explain like I’m 5” style and, if the other party indicates interest, add detail and jargon, taking into account related concepts that may already be familiar to them. If you take them into account, they won’t get bored when you go into detail.


> With all that said, I prefer to first explain my job in an “explain like I’m 5” style and, if the other party indicates interest, add detail and jargon

It is just your choice. I'd prefer a short answer full of jargon. It gives people the opportunity to clarify what they want to ask. Do they really want to know details? Or they want a rough idea of an answer? Or they just filling silence with small talk?

Though other times, when I really want to talk about it, I'd go with some ELI5 explanation, while watching people, are they interested or not?

> Honestly, I think not appreciating simple explanations indicates both low mathematical/logical and social/emotional intelligence.

It can be. But mostly it is not. People are sending signals by choosing one form of the answer or another, you just need to decode their signals. And it will be better, if you don't jump to conclusions about their persistent psychological traits, based on the first impression.


The correct option is to treat such conversations as a protocol with a negotiation at the onset.

SYN ACK

>extremely uneducated, under-stimulated people, who also have unaddressed insecurities around their own intelligence. But that’s not most humans.

This isn't going to be most humans you encounter if you're in the HN demographic, but that's a bubble. It does describe most people in the world.


This is, in a word, nonsense.

> or assume they’re not emotionally intelligent (because it’s a choice demonstrates little respect for the person they’re addressing)

Intelligence, in the traditional sense, also involves understanding when to give up. Part of "emotional intelligence" is judging whether the other party actually cares about what you're about to say.


Your default position is distrust and anxiety though. Most people aren't wired that way.

> When I meet people who immediately use hyper-specific jargon with strangers, I either distrust them, or assume they’re not emotionally intelligent (because it’s a choice demonstrates little respect for the person they’re addressing).

For me, it's quite the opposite: such a choice demonstrates that they their prior is that I'm sufficiently smart and knowledgable to be likely able to understand this explanation - which I rather consider to be a praise. :-)


I think "with strangers" is the important bit. If a nuclear engineer is talking to some lay person and uses hyper specific jargon, then grandparent is correct. If you've established a shared competency with the person, and are therefor no longer total strangers, that's totally different.

The way I think about it is this. There are roughly two groups of people:

- Some people will not care / be dismissive of what I have to say. I probably don't want to talk to these people much.

- Some people will be interested! I probably will like these people.

If I use technical jargon, I am optimizing to impress people I don't really care about impressing - and I will be pushing away the people that I would actually be interested to spend time with.

If I speak respectfully, i.e. the simple explanation, it will resonate more with the people I like. I will push away the people who don't care, but I didn't really want to talk with them anyways.


>You can give a quick explanation in terms they understand, which makes your job sound easy and makes them wonder how anybody gets paid to do it.

What is the problem with this?

Most jobs, when simplified, sound like "anybody can do it". I think it's generally understood among adults who have been in the workforce that, no, in fact anybody cannot do it.


There is no problem with it, but I assume there are many people who will look upon you favourably if they think you do a highly skilled job. While many of us may not care to impress those people, there are certainly those who do (possibly people with similar attitudes who care more about validation from people who think like them)

A somewhat ungenerous characterization of the attitude may be something like the Rocket Scientist vs Brain Surgeon sketch - https://www.youtube.com/watch?v=THNPmhBl-8I

But we should also acknowledge that there's an entire culture built around valuing people and their time relative to one's perception of their "importance", that this culture can influence one's earning potential and acquisition of material possessions, and that many people do care about things like "seeming important" or moving upwards in this hierarchy as a result.


I think which direction you choose is about knowing your audience. As you mentioned, different people value different things and humans often want to present a different view of ourselves to different people at different times.

I don't see what's hard about threading the needle, or maybe I'm completely lacking in EQ

"I'm a mathematician, I study how shapes fit together, which surprisingly, is being used for new methods of secure communication by so and so university, but I just love the math"


Or “I’m a mathematician. I try really hard to find things I can prove that have no practical application. But frustratingly people keep finding important practical applications for my work.”

I choose the worst of all options and go into excruciating detail.

Thereby minimizing how often anyone asks you - which makes that the best long-term option?

That would only work if you were getting repeat inquiries from the same person. Otherwise it's just the longest possible option for each new inquiry.

I always opt for excruciating detail because it's what I enjoy the most.


> That would only work if ...

Sounds like none of the people you answered, in excruciating detail, cared to warn other people about what would happen if they asked you.


It really can go both ways. Was told that "Ask baobun about hacking when you're high" came recommended.

Ahh! I didn't think about the word-of-mouth. Good call.

My wife's eyes just gloss over. Maybe I should try with some other test subjects.

One of the classical assessments in strategic behavior is "be worse than your roommates at chores so they do them, but not so bad they kick you to the curb."

I kinda love doing the quick+easy explanation... And especially in professional contexts.

"I teach computers what sounds different aminals make."


As a person who uses the Merlin app regularly, I appreciate this field of study.

Great pickup line.

> You can give a quick explanation in terms they understand, which makes your job sound easy and makes them wonder how anybody gets paid to do it.

What's wrong with this? Making it look easy is why you get paid for it.


The latter option always comes across as rude. It's a very clear 'piss off you insect'

> The latter option always comes across as rude. It's a very clear 'piss off you insect'

To me, it rather tells: "I consider you to be likely to be sufficiently smart and knowledgable to understand this topic if you put in some effort: do you want to learn some cool stuff which otherwise would demand a lot of literature research to learn? And since I already hinted that I consider you to be smart and knowledgable: would you like to teach me some cool, complicated stuff, too?"


> You can give a quick explanation in terms they understand, which makes your job sound easy

This is always the right answer. It is the only answer that respects the listener and contains a seed to further conversation.


I once told my dad that if the subject of my thesis was something I could easily explain then it wouldn't be interesting enough to do a PhD in. I said it half-jokingly and he laughed about it, but he stopped asking me what I'm studying after that so maybe he did take it more seriously.

There’s something bittersweet about that moment when someone you love stops asking about your research. It’s a quiet kind of respect, but also a reminder of the communication gap academia often creates.

Hard to explain doesn't make it interesting either.

There's more choices than three.

Eg you can focus on what you actually do, or you can focus on the benefits you bring to other people.


Or tell them about the bit of the job they understand. "I teach maths to adults".

> There are three choices

There is another:

Give away as little information as you can about it.

Don’t say or agree that it’s secret or that you can’t talk about it- just be tight-lipped, and don’t divulge.

If you do it right, you will seem mysterious.

If you do it wrong, they probably won’t talk to you much again.

Win-win.


If you can't explain something in simple terms, you don't understand it well enough

Some ideas are too complex to explain accurately in simple terms.

You can give someone a simple explanation of quantum chromodynamics and have them walk away feeling like they learned something, but only by glossing over or misrepresenting critical details. You’d basically just be lying to them.



Reminds me of the old videos on the Mill CPU architecture. There is multi hour long video about “the belt”, a primary concept in understanding the Mill architecture and instruction scheduling. It’s portrayed in the slides as an actual belt with a queue of items about to be processed, etc.

Only in the end to reveal the belt is truely conceptualized and does not formally exist. The belt is an accurate visual representation and teaching tool, but the actual mechanics emerge from data latches and the timing of releasing the data, etc.

I thought it was helpful.


https://youtu.be/QGw-cy0ylCc

Is this an asynchronous architecture CPU?


It's not. I'm curious what gave you that idea, though?

The belt moves once per cycle, if that wasn't clear? He says the word "cycle" (and measures latency in cycles) a lot.


That's how you get a whole population imagining mitochondria as puffy gelatinous beans, instead of network around other organelles.

https://www.nature.com/immersive/d41586-025-00269-y/index.ht...


Quantum Mechanics is the example of a subject where supposed experts don’t really understand it either and hence can’t explain it adequately.

Also, it’s hilarious to get comments like this voted down by non-experts who assume this must be an outsider’s uninformed point of view.

I have a physics degree and I studied the origins and history of quantum mechanics. Its “founding fathers” all admitted that it’s a bunch of guesswork and that the models we have are arbitrary and lack something essential needed for proper understanding.


Take for example entanglement.

The math that describes it is known precisely. Specific implications of this are known. There's no information transfer, there's no time delay, etc.

And yet lay people keep incorrectly thinking it can be used for communication. Because lay-audience descriptions by experts keep using words that imply causality and information transfer.

This is not a failure of the experts to understand what's going on. It's a failure to translate that understanding to ordinary language. Because ordinary language is not suited for it.

> Its “founding fathers” all admitted that it’s a bunch of guesswork and that the models we have are arbitrary and lack something essential needed for proper understanding.

We don't have a model of why it works / if there's a more comprehensible layer of reality below it. But it's characterized well enough that we can make practical useful things with it.


> This is not a failure of the experts to understand what's going on.

> We don't have a model of why it works / if there's a more comprehensible layer of reality below it.

Counterpoint:

You’ve just admitted they don’t understand what’s going on — they merely have descriptive statistics. No different than a DNN that spits out incomprehensible but accurate answers.

So this is an example affirming that QM isn’t understood.


The only descriptive / empirical parts is the particle masses.

But it sounds like your objection is that reality isn't allowed to be described by something as weird as complex values that you multiply to get probabilities, so there necessarily must be another layer down that would be more amenable to lay descriptions?


That’s not my point, nor close to what I said.

My point is that their models are fitted tensors/probability distributions, often retuned to fit new data (eg, the epicyclic nature of collider correction terms) — the same as fitting a DNN would be.

Their inability to describe what is happening is precisely the same as in the DNN case.


'It's the study how the particles that make atoms interact... it's fiendishly complicated'

And that’s why Feynman was always happy to explain how magnets work!

Feynman was happy to explain why he couldn't explain how magnets work!

https://m.youtube.com/watch?v=MO0r930Sn_8


Simple terms need not be short terms.

A horse is just a bunch of chemicals in a skin sack. Gee, I understand it!

Hmmmm, what might Feynman say about a horse?

So, what's a horse? Well, you look at it: it’s this big animal, standing on four legs, with muscles rippling under its skin, breathing steam into the cold air. And already — that’s amazing. Because somehow, inside that animal, grass gets turned into motion. Just grass! It eats plants, and then it runs like the wind.

Now, let’s dig deeper. You see those legs? Bones and tendons and muscles working like pulleys and levers — a beautiful system of mechanical engineering, except it evolved all by itself, over millions of years. The hoof? That’s a toe — it’s walking on its fingernail, basically — modified for speed and power.

And what about the brain? That horse is aware. It makes decisions. It gets scared, or curious. It remembers. It can learn. Inside that head is a network of neurons, just like yours, firing electricity and sending chemical messages. But it doesn’t talk. So we don’t know exactly what it thinks — but we know it does think, in its own horselike way.

The skin and hair? Cells growing in patterns, each one following instructions written in a long molecule called DNA. And where’d that come from? From the horse’s parents — and theirs, all the way back to a small, many-toed creature millions of years ago.

So the horse — it’s not just a horse. It’s a machine, a chemical plant, a thinking animal, a product of evolution, and a living example of how life organizes matter into something astonishing. And what’s really amazing is, we’re just scratching the surface. There’s still so much we don’t know. And that is the fun of it!


Not every subject has simple explanations.

How simple? Simple to who?

The quip you're referring to was meant to be inspirational. It doesn't pass even the slightest logical scrutiny when taken at its literal meaning. Please. (Apologies if this was just a reference without any further rhetorical intent though.)

It's like claiming that hashes are unique fingerprints. No, they aren't, they mathematically cannot be. Or like claiming how movie or video game trailers should be "perfectly representative" - once again, by definition, they cannot be. It's trivial to see this.


I have my own micro business where I make equipment for high energy physics machines.

I have yet to figure out a way to tell people what my business is in a way that is even slightly accessible. Everything about it is so esoteric and multiple steps removed from regular life. It's not necessarily complex, it just contains a ton of details that the average person has no familiar contact with, and don't really have everyday analogues.


Isn't "I have my own micro business where I make equipment for high energy physics machines" a good description already?

> I make equipment for high energy physics machines

> I have yet to figure out a way to tell people what my business is in a way that is even slightly accessible.

You ... just did? In a remarkable short, concise, and very accessible way. I can ask as many follow up questions as I want and we might even have an engaging conversation. Sounds interesting!


It doesn't really tell you much, and frankly my audience is mostly non-tech people. And no doubt some people really are curious and keep asking questions, but most people you can kinda see their head uncomfortably spin.

I also obfuscated it a bit by giving the most general name just for privacy reasons since not many people do it. But rest assured it is a "Retro Encabulator" type machine, and as you add details it just becomes more and more alien.

This is not at all what I do, but its similar esoteric-ness to "I make differential gear sets for calibrating ion trap interferometry systems". A collection of words where every one of them the average person struggles to place.


Help me that you're not a doctor a lawyer accountants software engineer working for a large company. It tells me you're a small business owner and you work on advanced things. You're not manufacturing knick knacks or toys.

Really if we're at a party that's more than enough unless I want to ask you more and you want to talk more about it. If you were a lawyer I'd probably ask what area of law that I probably stop and talk about something else. So I agree with others that you said was a very good distillation of what you do to the level that most people probably care about


My cousin has his own metrology business, and it took me a long time before I understood how he was doing so well financially. Kinda get it now.

At least in the case of sphere packing it's closely related to some core problems in information theory that helped make the Bell phone system so reliable.

(not sure about convex shapes)


Yeah I'd definitely explain that one as "I study ways to make wifi faster", doesn't cover all the nuance, but it's definitely better than the alternative.

Convex shapes, well, annoyingly it's too broad. It has way more applications than sphere packings but it's hard to pick a good example. It's like trying to explain you design screwdrivers to someone who doesn't know what a screw is.


I describe myself as a plumber but with systems for moving around masses of data instead of water.

betjeman's delightful poem "executive" had a great humorous take on this:

You ask me what it is I do. Well, actually, you know,

I'm partly a liaison man, and partly P.R.O.

Essentially, I integrate the current export drive.

And basically I'm viable from ten o'clock till five.


shapes that exist on higher dimensions we can't mentally comprehend.

“I’m an electron wizard. I write spells and magical constructs appear on the mirror slate”

Neat. I spent a month trying to use sphere packing approaches for a better compression algorithm (I had a large amount of vectors, they were grouped through clustering). Turned out that theoretical approaches only really work for uniform data and not any sort of real-world data.

EDIT: groped -> grouped


The usual trick is to use domain-specific knowledge to translate that asymmetry to uniformity.

E.g.: Suppose the data has high-order structure but is locally uniform (very common, comes about because of noise-inducing processes). Compute and store centroids. Those are more uniform than your underlying data, and since you don't have many it doesn't really matter anyway. Each vector is stored as a centroid index and a vector offset (SoA, not AoS). The indices are compressible with your favorite entropic integer scheme (if you don't need to preserve order you can do better), and the offsets are now approximately uniform by assumption, so you can use your favorite sphere strategy from the literature.


I'm sure you've already explored this, but is there some precompression operation that you could do to the vectors such that they're no longer sparse, and therefore relatively uniform?

They weren't sparse, they were dense but the "density" was quite non-uniform (think typical learned ML vectors). Not too far from an N-dimensional gaussian (I ended up reading research on quantizing Gaussian distributions, but that didn't help either as we didn't have a perfectly gaussian thing).

If you start from N-dimensional Gaussian-distributed vectors and normalize them, you wind up with uniformly distributed vectors on an (N-1)-dimensional hypersphere. Of course, sphere-packing on the surface of a hyperspehere is its own fun problem, and if your data is not actually Gaussian may still not be exactly what you want, but coding the vector magnitude separately from the vector direction is probably a good start.

I see, thank you.

Another thing that I'm sure you explored, and I'd love to hear how it went, would be to rearrange the elements in the vectors such that perhaps the denser parts could be more contiguous, and the sparser parts could be more contiguous, on average. That sounds like something that would be easier to compress. Were the distributions such that a rearrangement like this might have been possible? Or were they very evenly distributed?

I.e. could you have rearranged a Gaussian-like distribution into a Poisson-like distribution?


VAE objectives are useful for pushing embeddings into a Gaussian distribution.

Here's some work on low-latency neural compression that you might find interesting: https://arxiv.org/abs/2107.03312


You really shouldn't grope your vectors.

Roger, Rodger. Over, Oveur.

you're Kareem Abdul-Jabar!

_May_ be a case for extending out what has been explored by theory to cover more useful ground (or not, depending on whether real-world usecases like yours are too heterogenous for effective general techniques).

It's usually the case that the low hanging fruit in a decades old commercially valuable field have already been picked.

I feel like mathematicians should be able to do a second doctorate level degree a few years after their first PhD, that must be in a adjacent field of their own, but not the same.

> For a given dimension d, Klartag can pack d times the number of spheres that most previous results could manage. That is, in 100-dimensional space, his method packs roughly 100 times as many spheres; in a million-dimensional space, it packs roughly 1 million times as many.

Those numbers sound wild. For various comms systems does this mean several orders of magnitude bandwidth improvement or power reduction?


I think not, because moving to a higher dimension is exponentially worse (density ~ n^2/2^n) than this linear improvement.

So it's only helpful for naturally high dimensional objects. Digital objects do not have a natural dimension (byte length), so you can choose a small dimension.

https://en.m.wikipedia.org/wiki/Sphere_packing


This should have practical applications for cow packing in physics.

> The answer matters for potential applications to cryptography and communications

Someone can take this challenge to provide a more secure and reliable communication systems hopefully with more energy efficiency, very much an exciting research direction.


does anyone know at what lowest dimension does this construction beats known best packing?

Noob question: Is the optimal sphere packing correlated with a regular lattice? I.e. that's the case for 2D,3D right? If so does this extend to ND?

Besides 2 and 3 dimensions, it's also the case in 8 and 24 dimensions (The E₈ lattice and Leech lattice, respectively). These were proven in 2017 by Maryna Viazovska, with some collaborators for the second paper. https://doi.org/10.4007/annals.2017.185.3.7 https://doi.org/10.4007/annals.2017.185.3.8

See also https://www.ams.org/journals/notices/201702/rnoti-p102.pdf

For other dimensions, this is an open question; it seems unlikely to be true in general. For some dimensions the densest known irregular packing is denser than the densest known regular packing.


> For some dimensions the densest known irregular packing is denser than the densest known regular packing.

I thought that was one of the important results from the paper, the most efficient packing for all dimensions is symmetrical again and this increase was significant enough it seems unlikely that existing non-symmetrical methods will be able to beat it.


Not necessarily—in 3d there are uncountably many non-lattice packings. They all have the same density as the FCC lattice though. To construct these packings, shift horizontal layers of FCC horizontally with respect to each other.

It is conjectured that in higher dimensions, the densest packing is always non-lattice. The rationale being that there is just not enough symmetry in such spaces.


Well these new results (denser packings than before) are regular lattices which might suggest that the optimal packing could be a lattice. (Until the record is broken again by a irregular packing ;-)

I hated maths as a kid, now I love this stuff; pure maths for its own sake. Super impressive! It's a dream of mine to discover anything useful in the field.

The delight of pure maths is in its uselessness :-)

Very cool. Sphere packing comes up in a lot of contexts in applied problems. Looking forward to reviewing the paper.

Can’t you just montecarlo the hell out of this thing?

According to the article, that's precisely what he did.

But he did it in a better way, randomly expanding ellipses. And when you're dealing with hundreds of dimensions, it's incredibly easy to be montecarlo-ing in the wrong directions...


I don't think so. You can't just "Montecarlo the hell" out of a post-quantum cryptographic scheme [1] which are also based on lattices. Even if that analogy isn't quite apt, I don't think looking for a good random lattice is a good way to find the densest possible lattice [2].

[1] https://en.wikipedia.org/wiki/Post-quantum_cryptography

[2] https://en.wikipedia.org/wiki/Random_close_pack#For_spheres


Earlier today there was an article about neanderthal's rendering fat.

The comments pointed out that anthropologist did not know that boiling was possible before the invention of pottery. Another comment pointed out that science teachers knew that it was possible because that was something they would do in class.

Final comment was about how people ReDiscover things in different fields - - like the trapezoidal rule for integration being discovered by someone studying glucose.

This is just yet another example of how bringing expertise from a different area can help.


The aforementioned trapezoidal rule (Tai's method): https://diabetesjournals.org/care/article/17/2/152/17985/A-M...

I haven't read that thread, but I don't believe that anthropologists thought boiling was impossible before the invention of pottery. Here's one youtube video that demos a method for survival scenarios, I'm sure there are many others: https://www.youtube.com/shorts/0zun_UxO2vU. I know I don't have the context, but unless there are sources for the remarkable claim, it just doesn't make sense. It doesn't pass "the laugh test"

https://www.science.org/doi/10.1126/sciadv.adv1257 "Underlining earlier work by Speth (61), experiments recently demonstrated that organic perishable containers, e.g., made out of deer skin or birch bark, placed directly on a fire, are capable of heating water sufficiently to process food, with the advantages of wet-cooking beginning at lower, sub-boiling temperatures than thus far acknowledged (62)." Reference 61 goes to this 2015 paper: https://scholar.google.com/scholar_lookup?title=When+did+hum... "to alert archaeologists and others to the fact that one can easily and effectively boil in perishable containers made of bark, hide, leaves, even paper and plastic, placed directly on the fire and without using heated stones."

Thanks for the link! Not to be argumentative, but I do want to highlight a couple points that I think may be the root misunderstanding:

1. The 2015 paper doesn’t indicate that it was unknown that you can boil water without pottery. It is meant to alert people to an already known fact. Maybe silly, but I do think there’s a distinction here — it’s not that the pinnacle of human knowledge was missing this, it’s that common sense was missing it. I’d compare this to the article “things SDEs assume about names”

2. The 2015 paper is not about boiling water in general, it’s about boiling it in a specific way. (Whereas the initial discussion was about boiling water by any method)


I know right? If I, as a yokel kid knew this, it astonishes me people like that wouldn’t know it

If only there were some sort of expert in everything that we could ask, it could pull expertise from all various sciences into one response. I think everyone just needs to start using LLMs.

Joey Chestnut?

This was a very confusing article, full of filler. I couldn't stand to read the "detective story" style.

Sounds like the technique is for high-dimensional ellipsoids. It relies on putting them on a grid, shrinking, then expanding according to some rules. Evidently this can produce efficient packing arrangements.

I don't think there's any shocking result ("record") for literal sphere packing. I actually encountered this in research when dynamically constructing a codebook for an error-correcting code. The problem reduces to sphere packing in N-dim space. With less efficient, naive approaches, I was able to get results that were good enough and it didn't seem to matter for what I was doing. But it's cool that someone is working on it.

A better title would have been something like: "Shrink-and-grow technique for efficiently packing n-dimensional spheres"


"Shrink-and-grow technique for efficiently packing n-dimensional spheres" isn't obtuse enough.

I think something like "Hypertopological Constriction-Expansion Dynamics in Quasistatic R^n-Ball Conglomeration" would be even more apt.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: