Chat-tuned ones do, but the base models don't. For example, Llama doesn't, but A... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

sp332 on July 24, 2023 | parent | context | favorite | on: Attention Is Off By One

Chat-tuned ones do, but the base models don't. For example, Llama doesn't, but Alpaca has "### Instruction:", "### Input:", and "### Response:".

int_19h on July 24, 2023 [–]

Base LLaMA still has dedicated tokens for beginning/end of string. What you're describing is the instruction format, which is separate.

sp332 on July 24, 2023 | [–]

Oh, I had misunderstood something.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact