Hacker News new | past | comments | ask | show | jobs | submit login

“Nearly perfect” is cherry-picked from the sentence

> Even models trained on just 32K tokens, such as the Qwen2.5-7B-Instruct, achieve nearly perfect accuracy in passkey retrieval tasks with 1M-token contexts.

Which is pages after the graph and table you mentioned, which are clearly introduced as

(Graph)

> First off, we evaluate the Qwen2.5-1M models on the Passkey Retrieval task with a context length of 1 million tokens. The results show that these models can accurately retrieve hidden information from documents containing up to 1M tokens, with only minor errors observed in the 7B model.

(Table)

> For more complex long-context understanding tasks, we select RULER, LV-Eval, LongbenchChat used in this blog.

That you went so deep into the post to find your “clever” phrase to complain about tells me you’re probably being intentionally misleading. Most readers won’t read that far and ones that do certainly won’t leave with an impression that this is “nearly perfect” for complex tasks.




> “Nearly perfect” is cherry-picked from the sentence

You're attempting to imply the rest of the sentence adds context that makes pulling out "nearly perfect" incorrect. Can you explain?

> ...

I'm not sure what the rest of the quotes are implying, as you just copy and paste and don't provide any indication of what you're communicating by sharing them. Can you explain more?

> That you went so deep into the post

It's the 587th word, less than 2 minutes reading at average reading speed.

> you’re probably being intentionally misleading.

!?!?!

#1) I'm certainly not intentionally misleading.

#2) What is misleading about "they say nearly perfect and then the highest # I can steelman from the table is 84%?"

#3) This is the first time in 15 years on HN that I've had someone accuse me of being intentionally misleading. Part of that is because there's numerous rules against that sort of dialogue. The remaining part is people, at least here, are usually self-interested enough to not make up motivations for other people feeling differently from them.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: