Oh wow, that's just amazing, thanks for the link! From the article:
'''
Interestingly, there was a technical bump or glitch in the Unity Engine. A feature that no user would normally have any control over, that caused a bump that made it all the way from the D23 footage to the final film. Every once in awhile the Unity Engine clears our any unneeded data or assets from the Engine.
Unfortunately, when this random function, deep in the code called ‘garbage collection’ ran it could cause a tiny pause in the smooth movement of the master Unity Camera. One such ‘bump’ happens in a shot in the D23 trailer. After the trailer was complete, the team discovered what the issue was and fixed it. But even when the shot was redone much later, this ‘bump’ is still in the final camera move, just because Rob Legato liked the feel of the recorded natural move, even with the bump. For the creative team, ironically given its causes, it just felt natural.
'''
There's a blog post from 6-8 years ago I can no longer find anywhere, where someone made node.js create similar ambient sounds and compared it to listening to hard drives doing mechanical work to understand what the computer was doing. Apparently it was actually useful to audibly hear when, for example, a request failed due to an authentication problem which had its distinct sound emerging from the pattern of code execution on that branch.
Does anyone have a link to what I'm talking about?
I recall something like that. You might be talking about baudio: https://www.npmjs.com/package/baudio . I also recall seeing some website demonstrating the idea of soundifying a stream of incoming metrics/events so you could listen to it and your brain could sift through and automatically pick our patterns, changes, and anomalies. I think there was a demo using the twitter firehose as an example? But for the life of me I can't remember anything more than that.
Unrelated, but GPU coil whine can give unintentional ambience to LLM inference.
In some llama.cpp versions on my home inference rig it would manifest as a sort of squelchy sound that would match the generation of characters on the screen, reminiscent of the effect often used during dialog in 8-bit era video games.
I've actually noticed something similar on my GPU. Not sure if it's the same thing but whenever I'm playing Minecraft it makes a weird sound I don't really know how to describe, but as soon as I navigate away the sound goes away (some performance mods probably halt it's execution/rendering when not in focus).
I'm very curious about what exactly makes the sound and how it works. Glad to know it isn't only me because for some time I was afraid my GPU was defective :^)
A lot of the time it can happen if you're rendering a ridiculous number of frames. I used to get it in game menus where the FPS would spike into the hundreds. Other than that, just general high load will do it.
The noise comes from inductors in the GPU core power supply. Under the right load conditions, the coil in the inductor begins to resonate. The magnetic fields change so rapidly and contain so much energy that they physically push the coil until it resonates at some small fraction of the switching frequency. It's a pretty neat, if extraordinarily annoying phenomenon
• This sounds very similar to what Calm Technology¹ is trying to achieve. Their first example was a freely dangling string tied to a motor, directly connected to the Ethernet. The string then spun soothingly in a corner with some auditory and visual indication of network activity.
• I seem to recall that the developers of the new computer in The Birth of a New Machine connected (old, analogue) oscilloscopes to the CPU pins for register content, directly driving the X and Y coordinates of the oscilloscope beam, and thereby got a visual image of how the execution usually looked in different situations, and when they looked “wrong”, it was time to slow down and analyze step-by-step.
• Many older machines were not very well shielded against RF interference, so in olden times hackers used to turn on a radio to the most interfered channels to listen to the CPU executing their program, which helped with debugging. (Later, many home computers were similarly not very good at compositing a TV signal, so the computer’s activities would partly bleed into the sound and/or video output. This also similarly aided debugging.)
I was somewhat disappointed that the artificial sounds in the video were actually old game machine sound effects tied to events, not the actual sounds of the high-frequency events turned into audio.
This happens in nature too, even for weirdly specific things. I was very surprised that my work laptop makes a high pitched hissing noise when updating graphics in a remote x11 program. Nothing else seems to trigger it...
Good idea! There are various other extensions possible.
Unfortunately there's a bit of an observer effect, the sounds are made with python code running in the same process, and the more complicated I made the sound generation logic, the more garbage it produces itself .
This is wonderful and I'd like to see more examples of multi-sensory experiences for working with computers or algorithms.
Are brains, especially the subconscious are powerful at finding and learning patterns - so being able to utilize that seems so useful!
> - the two different coin sounds are played when the JIT produced a new chunk of machine code.
... as a thing for a human to listen to, it sounds like a lot is going on. But as an indicator of what's happening inside a VM in 2024, is this surprisingly slow? Especially at the intro, if you were paying close attention, you could manually count these events.
The point of a JIT is to optimize code that is, roughly, the largest percentage of program time. Most programs follow a power distribution where there is some very important code that is ran a lot, with a lot of code that isn't ran very often (or worse, only ever once). If you JIT compile the N highest impact parts of the code, then you get the highest impact without actually needing to process that much code - which is extremely important, because the act of JIT compiling is very slow and only should be done when it's likely profitable in terms of execution time to waste so many cycles compiling code, with the expectation that you will make it back by making it N cycles faster every time it runs in the future.
The 'empty' parts of the beginning of the program are actually a 'configure' phase, where the program runs a bunch of subprocesses (C compiler with small query programs) and the vm is mostly waiting for the results of that. That's why there are pauses in the coins.
On the other end, I also artificially limit the samples to 10 coins a second (iirc), because if the jit is really compiling a lot it would be just too annoying.
