not the OP but the heaviest charges against Swartz didn't stem from his IP infringement, it was wire fraud and computer fraud. He used a network access and MIT guest account he didn't have legal access to because he wasn't affiliated with the institution, which enabled the CFAA charges.
Qwen’s 30B models run great on my MBP (M4, 48GB) but the issue I have is cooling - the fan exhaust is straight onto the screen, which I can’t help thinking will eventually degrade it, given the thermal cycling it would go through. A Mac Studio makes far more sense for local inference just for this reason alone.
I have 24GB VRAM available and haven't yet found a decent model or combination.
Last one I tried is Qwen with continue, I guess I need to spend more time on this.
im currently running a custom Gemma4 26b MoE model on my 24gb m2... super fast and it beat deepseek, chatgpt, and gemini in 3 different puzzles/code challenges I tested it on. the issue now is the low context... I can only do 2048 tokens with my vram... the gap is slowly closing on the frontier models
It's a MoE model so I'd assume a cheaper MBP would simply result in some experts staying on CPU? And those would still have a sizeable fraction of the unified memory bandwidth available.
I haven’t tried this myself yet but you would still need enough non-vram ram available to the cpu to offload to cpu, right? This is a fully novice question, I have not ever tried it.
You're correct. If you don't have enough RAM for the model, it can still run but most of it will run on the CPU and be continuously reloaded from the SSD (through mmap).
A medium MoE like 35B can still achieve usable speeds in that setup, mind you, depending on what you're doing.
If you are comfortable with a hand grinder, Porlex grinders [1] [2] are excellent. I use one to make a coarse grind for the french press. It's gotten used 4-7 times per week for the last 5+ years and still going strong.
On topic of best purchases under $100, suppose you regularly boil water for coffee or pasta but don't own an electric kettle, consider investing in a cheap white plastic kettle for $5. Fast and energy efficient way to turn electrical energy into boiling water.
I think _good_ depends on your expectations. We have the eureka mignon hacked with a bigger dial and custom burs. Still not amazing consistency. Looking to upgrade in the next few years.
Merry Christmas, friends. And a special cheers to everyone that makes this community possible. It’s one of the last bastions of all that is good about this internet.
I'm using this with a Bun project for my testing needs. I spin PGLite at the beginning, throw it all away at the end. It's not as nice as transactionally isolated testing (a la Ruby on Rails, or Elixir), but it's a fine replacement until I have time to replicate it.
I'd rather use SQL than an ORM. Kysely is basically a wrapper for the SQL query syntax inside typescript which has the benefit of composability and incredibly rich typescript support
I checked it out and it looks good on paper but it only runs on Bun.
Don't get me wrong, I use Bun and I'm happy with it, but it's still young. With Hono/Drizzle/Zod I can always switch back to Node or Deno if necessary.
