PK Shiu
Ten Things to bring to College / College ...
Ten Things to bring to College / College Student Survival Kit
PK Shiu
I sold my car with CarMax
I sold my car with CarMax
PK Shiu
Creativity on a Paddle Board
Creativity on a Paddle Board
PK Shiu
Do your job well, thoughts on Martin Luther ...
Do your job well, thoughts on Martin Luther King Jr. Day
PK Shiu
Top 5 Podcasts for 2016
Top 5 Podcasts for 2016

django

Paul Bissex Presenting at the Boston Django Meetup 2010

Paul Bissex, author of "Python Web Development with Django", described how he use Django to replace two legacy desktop applications at hallmark.edu at the Boston Django meetup this month. Because of Django's ease of use and robustness, not only he replaced and deploy those apps easily, the apps has been running error free for a very long time!

Paul Bissex Presenting at the Boston Django Meetup 2010 from PK Shiu on Vimeo.

Jacob Kaplan-Moss on DevOps

Jacob gave a talk at the Boston Django Meetup this month on the topic of DevOps -- The role of the developer and the role of sysadmin are merging, and it is a good thing. This idea certainly resonate with me. I started my career as a DevOps by necessity -- I worked with a new and proprietary mini-computer, the Stratus. The OS was designed to be a developer's OS. All the operation support tools are designed for used by developers. It was a great OS to work with. When I moved to the *nix world, I met Ben, still one of the best sysadmin I know, who actually introduced me to Python. The world has come full cycle now. With utility computing and browser based clients, small team with small budgets launch large scale, fast growing web applications, developers need to be their own sysadmin as well. This is the video of Jacob's talk. I was a little bit late and missed a few minutes at the beginning.

Jacob Kaplan-Moss on DevOps at Boston Django Meetup 2010 from PK Shiu on Vimeo.

Django CSRF Migration

Like many of you, I am migrating all my Django sites to Django 1.2.1. For sites that are currently in production, I am doing the slow migration route. Just trying to get the site up with 1.2 without using any of the new features yet. One thing that I ran into is the new CSRF support. If you were not using it before, there really is no change with one exception -- all the generic views and admin views requires CSRF protection. This means that if you are using django login view django.contrib.auth.views.login , you have to make sure that any wrapper or custom templates support CSRF.

Specifically:

  1. If you use your own login template, you must add {% csrf_token %} to the end of the openning form tag.
  2. If you wrap the call to login with your own view, you must add the csrf decorator @csrf_protect to your view, after importing django.views.decorators.csrf.csrf_protect
  3. If you use the django.contrib.auth.logout view to redisplay a login form, you have to replace that with a wrapper because the auth.logout view does NOT add the csrf token. (Updated)

Otherwise django will send you a 403 error when you try to login.

My Accidental Connection with Senator Ted Kennedy

Being Chinese in Boston, I always hear positive stories about the late Senator Ted Kennedy helping immigrants. In addition, I agree with most of his political views, he seemed like a good Senator. After learning of his death, I felt we had lost a great man. I added one of his biographies to my Amazon wish list, to learn even more. Other than media, that is the only connection I have had to Ted Kennedy. Until Thursday.

I was instant messaging with my designer on a project, NPR news streaming in the background when a bit of white noise filtered in. “Did I just heard that the Kennedy Funeral is going to be held at the Basilica of Our Lady of Perpetual Help?” I am familiar with this church, often called the Mission Church, as it is located on Tremont Street, Mission Hill.

About a year ago, my friend, a supporter of the church, asked us to revamp their website. The previous site was, let's say, a pre FrontPage era website. Donating our services, we redesigned the site. With the limited budget, I photographed the site images myself. Corrie, our designer, touched them up digitally. You can read about the design process on her blog.

After hearing the Funeral location on NPR, I immediately checked the Google analytics for the website we had worked on. It went up over one thousand percent just yesterday, upping a few hits a day to 6,000. But that was before the radio announcements.

[*Update*] The site traffic spiked to a 14000 visits (about 30,000 hits) on Saturday. During the funeral I did embed a live stream from ustream.tv onto the news section, since I assume most visitor at that time wanted to watch it.

I quickly made changes to the site to ensure it could handle the spike in traffic. I notified our hosting company, Web Faction and they agreed to keep an eye out for us on the status of the church site. So far, we have received about 20,000 hits in the last eight hours, averaging one hit every 1.5 seconds. People are hitting the visit us page, and the news page looking for details on the funeral.

I am glad the site has stayed up, and that I caught the news about the Funeral location. We really cannot fault the Church for neglecting to notify us of the pending event so that we could better prepare the website. The church has their hands more than full and I am sure the web site is not top priority.

I am glad we were able to make a very, very small contribution to the Senator's legacy.

For the techies, the site is done in Django, our web application framework of choice. It uses a custom content management system from my company, Imperial Consulting. The dynamic content is pulled from a Postgresql database. While we provided the Church with bi-lingual capabilities, they have yet to create all the Spanish content. It is supported by some busy and dedicated church volunteers, after all. The pages are cached by memcached at the URL level by the Django cache framework. So far, it is handling the traffic with ease. Keeping our fingers crossed.

Some books on Ted Kennedy

If you can recommend a good biography let me know!

Django Production Error Handler

I am working on an application that, besides providing a dynamic website, also talks to an iPhone application. What happens when the iPhone, or a web visitor, triggered a bug in the application? Django actually provide a nice mechanism to report error in its "batteries included" goodness. You can easily setup the Django environment so that it will send you an email when a "server error" occurs. You just need to make sure the following is setup:

Outbound email working

The django environment must be able to send outbound emails. The actual requirement depends on your server environment, but you definitely need to have correct values setup for:

settings.EMAIL_HOST settings.EMAIL_PORT settings.EMAIL_HOST_USER settings.EMAIL_HOST_PASSWORD

Admin users

settings.ADMINS -- this is a list of lists (or more accurately tuple of tuples) settings.SERVER_EMAIL -- email address of the error reporting from address Debug Setup

settings.DEBUG=False

500.html and 404.html

Once DEBUG is off, Django will want to display your 500 or 404 page. Create these pages and make them available on one of our template directories.

Example

Here are some sample entries from my settings file:

EMAIL_HOST='smtp.webfaction.com' EMAIL_PORT=25 EMAIL_HOST_USER='my_mailbox_name' EMAIL_HOST_PASSWORD='my_mailbox_password' SERVER_EMAIL='webmaster@imperial-consulting.com' ADMINS=( ('PK Shiu', 'support@imperial-consulting.com'),) DEBUG=False

Reference

Serving favicon in an Django App using Apache

I got a free few minutes to work on my own site here. Since I migrated the site from all static pages to Django served, I still haven't put back the favicon icon back onto the site. The sites runs under a virtual host in apache2 at WebFaction. This is what you need to put in your httpd.conf file: 

alias /favicon.ico /home/your-home/your-app-etc/static/image/favicon.ico

<LocationMatch "\.(jpg|css|gif|pdf|ico)$">
SetHandler None
</LocationMatch>

The alias line tells apache to go look for the favicon.ico file at a static location of your choice.

The LocationMatch directives tell apache to not run those files thru the Django engine.

Resetting Django Admin Password

This barely qualifies for a blog post, but what to do if you loaded, via django-admin.py loaddataa full json file from someone during testing, and don't have their user's password?

Just run the django-admin.py shell, and by hand reset all the passwords:

from django.contrib.auth.models import User for u in Users: u.set_password('secret') u.save()

That's why you have to keep your shell login and settings files safe !!

J2EE to Django, slides for the Presentation at Cambridge Python Meetup

I gave a short presentation on Django to the Cambridge Python Users group earlier. Nate has a great writeup of the event and the other presentations that evening. I just want to share the slides here. The slides are just visual reminders and do not stand on their own. If you want more info free feel to shoot me an email. I switched from J2EE to Django as my sole web application platform two years ago and has not looked back since. It allows me to develop, and more importantly maintain, web apps faster and better. It is more time and cost effective for my customers and I.

Slides from J2EE to Django Presentations at Cambrdge Python Group from PK Shiu on Vimeo.

Django Tip: No leading slash for upload_to for FileField and ImageField

This is a common mistake. When defining a FileField or an ImageField, you need to specifywhere the files are stored. This is done by specifying a relative path in the upload_to argument. Django will then store your files in a subdirectory as named, under the MEDIA_ROOT directory. But, don't put a leading slash in the relative path. Otherwise it will try to store the file at the system root directory.

MEDIA_ROOT = '/home/myname/files/ upload_to='pictures' file: abc.jpg results in: /home/myname/files/pictures/abc.jpg

But

MEDIA_ROOT = '/home/myname/files/ upload_to='/pictures' file: abc.jpg results in: /pictures/abc.jpg

Also a side tip: You can add strftime style arguments to the upload_to argument, storing the files in dated sub directory. e.g.

MEDIA_ROOT = '/home/myname/files/ upload_to='/pictures/%Y/%b/%d' file: abc.jpg results in: /pictures/2008/May/09/abc.jpg

Django Tip: Outputting list of items separated by commas, but only if it has more than one item

How many times do you need to do this? You have a list of things to output. The list can be empty, has one element, or more. You want to separate each items with a separator for readability. What do you do? 1. The simple but not reader friendly way:

toppings = [ 'cheese','tomatos','pineapple' ] or toppings = ['cheese']

{% for t in toppings %} {{ t }} , {% endfor %}

that will output: cheese, tomatos, pineapple, or

cheese,

Note the ugly trailing comma.

2. This is the smart way using the template variables available in loops:

{% for t in toppings %} {% if not forloop.first %}, {% endif %} {{ t }} {% endfor %}

that will output: cheese, tomatos, pineapple or

cheese No more trailing commas, thanks to the built in forloop variables.

gettext on Leopard for Django Internationalization

I started working on one of my internationalized applications on the new Mac. I realized I did not install "gettext", which is required by the make-messages and compile-messages scripts. I want to avoid installing things into OS X if I can. Then I found the easy way out: 1. Install poedit for os x. I need this to edit po files anyway. You can download it here. 2. There is no step two -- poedit for OS X comes with (obviously) all the gettext tools.

o.k. There is a little step two. You want to include the path to those tools when you run compile-messages. Create a little script like this:

PATH=$PATH:/Applications/Poedit.app/Contents/MacOS/ python /Library/python/2.5/site-packages/django/bin/compile-messages.py

and run this script instead of running compile-messages directly.

,

Django on Leopard

,

Part of the reason I was waiting to switch from using Windows XP (gasp) as my Django development platform to OS X is that it is actually easier to install the platform on XP. Windows does not come with any of the tools, so it was a matter of installing the version that I need. For OS X, it had a older version of python, so I would have to have multiple versions of it running. But Leopard changes all that. So I switched! Running Django on os x becomes much easier with Leopard. Out of the box Leopard comes with almost everything I need :

Python is at version 2.5.1 • svn is at version 1.4.4 • SQLite is at version 3.4 with python library installed

For development, I use SQLite and the built in web server only. I run Postgres on the production boxes. Because I do not have to install Postgres, getting Django setup is just a matter of setting up the rest of the environment:

Install Django

You can just download the installation tarball and do a standard Python install. But to have a little more flexibility in switching Django versions, I download both the production and trunk versions of Django into my own directory. Then using a soft link I can switch between the two. Download Django into my own home directory, under extlib, where I put all the external libraries. Then link from the Python site-packages directory to my own Django directory. By switching where that link points, I can switch version.

# Make directory and download two versions of Django: cd $HOME mkdir extlib cd extlib svn co http://code.djangoproject.com/svn/django/tags/releases/0.96.1 django-0.96 svn co http://code.djangoproject.com/svn/django/trunk/ django_trunk

# Then symlink instead of installing the django code into the Python site-packages, which is at # at /Library/Python/2.5/site-packages on my machine.

sudo ln -s ~/extlib/django-0.96/django django

Note: You can find the location of the library by running a simple python statement:

python -c ' import sys; print sys.path'

and look for the site-packages directory.

Setup Shell

I need to setup my shell environment so that my (multiple) Django projects can find Django. If you need help in figuring out where to put things, bashrc vs bash_profile etc, this article explains how and where to put things very well. Most importantly, add the django bin directory to your path:

PATH=$PATH:/Library/Python2.5/site-packages/django/bin Note: I am adding the Django bin directory to the PATH via the python site-packages link. That way if I switch Django versions via the site-packages link, the PATH will switch automatically!

Python Image Library

I need to use this for image processing. I thought I could get away from installing the free Apple developer tools but I couldn't. I have to install the Apple XCodeTools set first, from the OS X installation CD. Then download the source files here .

Before running the install, you also need to get the libjpeg for JPEG support:

Dowload the source at http://www.ijg.org/files/jpegsrc.v6b.tar.gz untar it, cd into the jpeg-6b directory and then:

cp /usr/share/libtool/config.sub . cp /usr/share/libtool/config.guess . ./configure --enable-shared make sudo mkdir -p /usr/local/include sudo mkdir -p /usr/local/bin sudo mkdir -p /usr/local/lib sudo mkdir -p /usr/local/man/man1 sudo make install

The following the PIL build instructions:

edit the setup.py file, add the lines instead of setting them to None:

JPEG_ROOT = "/usr/local/include" ZLIB_ROOT = "/usr/local/include"

then run:

python setup.py build_ext -i python selftest.py sudo python setup.py install

Django 0.96 Internationalization

I am trying out Django's i18n support and found a few issues. This only applies if you are using newforms and Django 0.96. Order of Middleware

First, the order of the middleware is important. It is defined in the documentation, but I missed it the first time around. You have to put the localeMiddleware after SessionMiddleware:

MIDDLEWARE_CLASSES = ( 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.common.CommonMiddleware', )

Newforms label

There is a known bug in the newforms' handling of label text in the 0.96 release. It loads the label text at load time. So if you switch language after a form is rendered, it will not re-evaluate the display text for the new language. There is a patch at ticket number 4904. Apply this patch to your newforms.util.py

Then make sure you define label text as gettext_lazy:

from django.utils.translation import gettext_lazy class MyForm(forms.Form): myfield = forms.CharField(label=gettext_lazy('translatable_text'))

Newforms Label Trailing Colon

Another little problem with newforms at 0.96. The newforms engine constructs default HTML display label from the form's field names and always append a colon to it. While in the development version, it first checks for a trailing punctuation mark first. If a supplied label already ends in a colon, it will not add another colon. Perhaps because of this, the current locale files have many Django labels defined, with and without the trailing colon. Kinda strange huh?

Using pytextile in Django – problem with unicode

I am coverting a J2EE application over to django. The existing app uses Textile as a simple markup for user text input. After installing pytextile (which is under ownership change at the moment), I found a problem using it: The "textile" filter works for strings and strings retrieved from my database, but when I try to use it on a piece of string that I received from a web form, it blew up with an encoding error. This is the issue:

  • Django nicely uses unicode string internally. When Django receive a string from a web form, it converts it from the browser encoding (utf8) to unicode.
  • pytextile expects the string to be in either the system default encoding, or when called in the filter, with the Django default charset (usually utf-8).
  • In my case pytextile choked on its own glyphs replacement code when trying to do a replace of latex style quotes on the string.

The solution is to convert the unicode string from forms input back to utf-8 first before giving it to pytextile:

for_textile_str = my_form_input_str.encode('utf8')
print textile(for_textile_str) # will now work.

Ownership change:

According to this webpage, I quote:

A couple of weeks I announced that I was looking for a new maintainer for pytextile, since I'm way too involved with my research and pydap. Well, as of today Silas Sewell is the new maintainer of pytextile.

The Importance of the initial argument in Django newforms library

I like Django a lot. In fact I am in the middle of converting a large J2EE application to Django. Sometimes I run into a problem using it, and knowing Django, I know there is a good solution. To find the solution however, sometimes require very careful reading of the documentation. Case in point. The solution here is completely documented in the Django online documentation. But perhaps, at least to me, not so obvious.

The problem:

A typical usage: display a form to add some data to the application. I need to include a hidden field and populate it with a value (obviously, since it is a hidden field, the only value is going to come from the application).

If you simply put the value in the dictionary to the form's constructor in the "GET" code path of your view handler, you will likely trigger the form's validation logic, and complain if you have any other normal fields that are required field.

The Solution:

The solution is to use the "initial" argument to the form's constructor, not to use the bounded data dictionary. The documentation discuss the "initial" argument as part of the unbounded newforms support, and I (wrongly assumed) normally do not think of my forms are unbounded. But they are, in the GET part of the handler.

Code Sample:

Class MyForm(forms.Form):
    important_field = forms.CharField(max_length=32,required=True)
    my_secret_field = forms.IntegerField(widget=forms.HiddenInput(),required=True)

def my_view(request):
    if request.method == 'POST':
        # do post stuff...
    else:
        # GET, do this:
        form = MyForm(initial={'my_secret_field':42})
        # do NOT do this, because form will be displayed with
        # error: important_field is required on first display:
        # form = MyForm({'my_secret_field':42})

Django dumpdata does not work with numeric fields

I try to dump my database in Postgres to json so that I can load it into my test system. Executing manage.py dumpdata gives me an error: 

Unable to serialize database: Decimal("349.00") is not JSON serializable

Some googling finds ticket number 3324, a problem with the serializer. This is patched in the development tree already. Since I want to keep 0.96 in production, I just hand patched the json serializer source file from the diff, and it works now.

See the diff output here .

Django + Ajax != dojo

If you good for "django" and "ajax", you will quickly come across the "Django adopts Dojo as Ajax framework" article at Ajaxian. The important thing to note is that is it not (yet) true. Django 0.96, the current release, does not contain dojo. The dojo toolkit if definitely one of the better ones out there, but it is not part of the django distribution. Dojo itself is going thru a major change from 0.4x to 0.9. Perhaps it would make sense to wait until dojo is updated before making it the official ajax toolkit. Or, just let the developer pick her own.