Roy van Rijn here, the guy that got an honerable mention for being “first”. Ended up 7th or something.
If you’re interested in even more in-depth tips and tricks that were used in this contest, I highly recommend this three hour (!) long deep dive I did with Thomas Wuerthinger (the winner of 1BRC):
Last January a challenge was posted online by Gunnar Morling:
How fast can you parse a file with one billion rows of weather data using Java?
Little did I know this deceivingly simple question would lead me down a path that taught me all about: parallelism, memory mapped files, SWAR techniques (SIMD as a register), bit twiddling, branchless code, mechanical sympathy, Graal native compilation and finally... I even turned to the dark side: using sun.misc.Unsafe.
Join me in this deep dive where I'll explain all the code changes and tricks that took me from the reference implementation which processes the billion records in less than 4 minutes, to processing everything in under two seconds.
Who knew Java could be this fast?
These fully customizable developers are powered by the advanced GPT-4 model. Digital Developers™ offer a cost-effective alternative to traditional Java developers, with a host of exceptional capabilities.
There are many ways to properly implement content-id, for example using fingerprinting (like Shazam does). These methods are able to detect matches even with a lot of noise and changes...
I hope they implement something like this, to make detecting duplicates and blocking "revenge porn" much easier and to have regression against re-uploading.
If you’re interested in even more in-depth tips and tricks that were used in this contest, I highly recommend this three hour (!) long deep dive I did with Thomas Wuerthinger (the winner of 1BRC):
https://youtu.be/_w4-BqeeC0k?si=p4NsRFPe6Jtq7HvA
There is so much valuable content in that talk, some things that I didn’t even realize until long after the contest was over.
It’s probably the best and most detailed conference talk I’ve ever done.