Batching lowers that, since the model is read once from memory. Activation accum... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

eurekin 3 months ago | parent | context | favorite | on: DeepSeek 4 Flash local inference engine for Metal

Batching lowers that, since the model is read once from memory. Activation accumulation doesn't scale as nicely

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact