The big problem I see here is that longer comments are much more likely to include quotes and other contextual information that makes them seem 'better' compared to shorter comments. The author mentions this, but underestimates its importance.
For example, using HN karma as an approximate value, I left a one-line comment [1] that seems uninteresting by itself, but added significant information to the discussion, compared to another comment [2] that is very long. The thing is, it's just a gigantic quote. I think that these two cases are the norm, not outliers.
On the other hand, I think it would be very interesting (and more accurate) to compare comment length and karma value on HN.
Yes, using ratings that are made in the context the comment originally appears would probably be better. /. is another site which could be used as source material for such an analysis.
For example, using HN karma as an approximate value, I left a one-line comment [1] that seems uninteresting by itself, but added significant information to the discussion, compared to another comment [2] that is very long. The thing is, it's just a gigantic quote. I think that these two cases are the norm, not outliers.
On the other hand, I think it would be very interesting (and more accurate) to compare comment length and karma value on HN.
[1] http://news.ycombinator.com/item?id=830887
[2] http://news.ycombinator.com/item?id=814591