Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You shouldn't care about UTF-8/UTF-16/UCS-4 except for performance, and that totally depends on what you are doing with what data-sets.

As someone who speaks multiple languages and has written a fair amount of language-processing code, the simple truth is that if you are not using a vetted unicode framework for writing your application--and if you're asking if UTF-16 should be considered harmful, you're probably not--you are almost certainly introducing massive numbers of bugs that your own cultural biases are blinding you to.

Do not underestimate the difficulty of writing correct multi-lingual-aware programs. These frameworks exist for a reason, and are often written by professionally trained linguists.



Interfacing with third-party libraries is an obvious case where you do need to worry about encodings for reasons that are not performance.


My experience is that the harder a framework tries to "support" Unicode correctly, the more likely it's going to have bugs ranging from annoying to showstopper.

Simply passing UTF-8 through almost works if it wasn't for the fact that "pass through" allows illegal character combinations that can cause all sorts of trouble.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: