Python Unicode: Difference between revisions
Jump to navigation
Jump to search
m (links) |
m (Reverted edits by 60.27.15.11 (Talk); changed back to last version by Xavi) |
||
| (3 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
{{Translations}} |
|||
Python has good [[unicode]] support, but it is not necessarily ''easy'' to use. Some things to note: |
Python has good [[unicode]] support, but it is not necessarily ''easy'' to use. Some things to note: |
||
| Line 6: | Line 7: | ||
* The [http://python.org/doc/current/lib/module-codecs.html codecs] module has some helpers for reading unicode from files. |
* The [http://python.org/doc/current/lib/module-codecs.html codecs] module has some helpers for reading unicode from files. |
||
* You can't generally write unicode to a file or the console without setting up something with codecs or another wrapper. |
|||
==Resources== |
==Resources== |
||
| Line 16: | Line 19: | ||
* [http://www.tutorialsall.com/PYTHON/ascii-latin/ ascii to latin1] (and back again). A thread about searching unicode text, so that for example a search for "televisao" matches "televisão" |
* [http://www.tutorialsall.com/PYTHON/ascii-latin/ ascii to latin1] (and back again). A thread about searching unicode text, so that for example a search for "televisao" matches "televisão" |
||
* [http://unicode.org/faq/normalization.html Normalization FAQ] (not Python-specific) -- a single unicode string can be encoded multiple ways via "surrogates". This introduces ambiguity. This talks about some of that. |
* [http://unicode.org/faq/normalization.html Normalization FAQ] (not Python-specific) -- a single unicode string can be encoded multiple ways via "surrogates". This introduces ambiguity. This talks about some of that. |
||
* [http://pylonshq.com/docs/0.9.4/internationalization.html The Pylons Internationalization document] has a lot of information that is generally applicable to Python. |
|||
[[Category:Developers]] |
[[Category:Developers]] |
||
Latest revision as of 13:10, 5 June 2007
Python has good unicode support, but it is not necessarily easy to use. Some things to note:
- You must test your application with real Unicode (not ASCII-encodable) text. You can miss lots of bugs if you just use normal ASCII text (i.e., a-z, no accents).
- You should be careful not to confuse 8-bit strings (that contain binary data and are of type "str"), and text (that contains unicode data and is of type "unicode"). It's easy to substitute one for the other, until you use non-ASCII text, then you'll get a UnicodeEncode/DecodeError.
- The codecs module has some helpers for reading unicode from files.
- You can't generally write unicode to a file or the console without setting up something with codecs or another wrapper.
Resources
Some resources to learn about Unicode:
- The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky -- general Unicode information
- Unicode HOWTO by Andrew M Kuchling -- Python unicode help
- Unicode in Python by Jason Orendorff -- some more Python help
- ascii to latin1 (and back again). A thread about searching unicode text, so that for example a search for "televisao" matches "televisão"
- Normalization FAQ (not Python-specific) -- a single unicode string can be encoded multiple ways via "surrogates". This introduces ambiguity. This talks about some of that.
- The Pylons Internationalization document has a lot of information that is generally applicable to Python.