Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

One big one is speculative decoding for larger models, the more “accurate”* the smaller model, the more the speed up on the bigger model

* as in matches the token that the larger model would output



It would be better if they also had a 70B model for this.

They would need the same vocabulary, etc. What else?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: