Using pytextile in Django – problem with unicode

I am coverting a J2EE application over to django. The existing app uses Textile as a simple markup for user text input. After installing pytextile (which is under ownership change at the moment), I found a problem using it:

The “textile” filter works for strings and strings retrieved from my database, but when I try to use it on a piece of string that I received from a web form, it blew up with an encoding error. This is the issue:

• Django nicely uses unicode string internally. When Django receive a string from a web form, it converts it from the browser encoding (utf8) to unicode.
• pytextile expects the string to be in either the system default encoding, or when called in the filter, with the Django default charset (usually utf-8).
• In my case pytextile choked on its own glyphs replacement code when trying to do a replace of latex style quotes on the string.

The solution is to convert the unicode string from forms input back to utf-8 first before giving it to pytextile:

for_textile_str = my_form_input_str.encode('utf8')
print textile(for_textile_str) # will now work.

Ownership change:

According to this webpage, I quote:

A couple of weeks I announced that I was looking for a new maintainer for pytextile, since I’m way too involved with my research and pydap. Well, as of today Silas Sewell is the new maintainer of pytextile.

Tagged as: django