Fighting Rabbits » Coding

Django on Snow Leopard

Richard Cooper — Sun, 04 Oct 2009 10:24:15 +0000

Having just been given an new MacBook Pro as an early birthday present, it didn’t take me long to try and install Django on it. Rather than use the MacPorts solution I decided to use the native python 2.6 and build the rest, apart from MySQL (use the Mac OS X 10.5 (x86_64) package from here)

Before proceeding make sure that you have installed XCode (It’s on the Snow Leopard install disc) as we need it to install libjpeg and PIL (all of my Django projects use this!)

Download libjpeg then:

tar -xzf jpegsrc-1.v7.tar
cd jpegsrc-1.v7
ln -s `which glibtool` ./libtool
set env MACOSX_DEPLOYMENT_TARGET 10.6
./configure --enable-shared && make && sudo make install

Now we can download the source and build PIL

tar -xzf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6
sudo python setup.py install

Last thing to do before we install Django is to install the MySqlDb connector – you will need to do add the following to your .bash_profile

PATH="${PATH}:/usr/local/mysql/bin"

Then:

sudo easy_install mysql-python

Now we can easy_install Django:

sudo easy_install django

and we are all set to go :)

Thanks to these guys for the help in figuring this out:
http://dryan.com/
http://www.brambraakman.com/blog/
http://colbypalmer.com/
http://www.agapow.net/

Nginx & Django on Webfaction (Part 1b) – Configuring

Richard Cooper — Mon, 07 Sep 2009 12:18:46 +0000

In Part 1a of this series we compiled Nginx and Installed Django in this next part we are going to configure nginx and Django and get our welcome page displaying.

First lets configure nginx, the default conf file looks like this:

#user  nobody;
worker_processes  1;
 
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
 
#pid        logs/nginx.pid;
 
 
events {
    worker_connections  1024;
}
 
 
http {
    include       mime.types;
    default_type  application/octet-stream;
 
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
 
    #access_log  logs/access.log  main;
 
    sendfile        on;
    #tcp_nopush     on;
 
    #keepalive_timeout  0;
    keepalive_timeout  65;
 
    #gzip  on;
 
    server {
        listen       80;
        server_name  localhost;
 
        #charset koi8-r;
 
        #access_log  logs/host.access.log  main;
 
        location / {
            root   html;
            index  index.html index.htm;
        }
 
        #error_page  404              /404.html;
 
        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
 
        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}
 
        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}
 
        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }
 
 
    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;
 
    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}
 
 
    # HTTPS server
    #
    #server {
    #    listen       443;
    #    server_name  localhost;
 
    #    ssl                  on;
    #    ssl_certificate      cert.pem;
    #    ssl_certificate_key  cert.key;
 
    #    ssl_session_timeout  5m;
 
    #    ssl_protocols  SSLv2 SSLv3 TLSv1;
    #    ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
    #    ssl_prefer_server_ciphers   on;
 
    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}
 
}

Now that we have seen the default file let’s back it up and start from scratch :)

cd ~/webapps/<myapp>/conf
mv nginx.conf nginx.conf.bak
nano nginx.conf

First we need to set our basic Nginx properties:

#2 workers x 1024 connections gives us some resiliance and should be enough for most sites.
worker_processes 2;
 
events {
    worker_connections 1024;
}
 
http {
    include             mime.types;
    default_type        application/octet-stream;	
 
    server {
        listen  myport;
        server_name mydomain;
        #You will need to change the next line to something bigger if you want to accept uploads over 5Mb
        client_max_body_size 5m;
    }
}

Remember to replace myport and mydomain with your actual values from Part 1a!

Next let’s setup our http server config to serve our static media (You could setup a static only app to serve your media from the main Apache Instance, but let’s do it this way as a learning exercise.) We are going to add one location for all our static media, and one location just to serve the favicon. Edit the http section to look like this:

http {
 
    include             mime.types;
    default_type        application/octet-stream;	
 
    server {
        listen  myport;
        server_name mydomain;
        client_max_body_size 5m;
 
        location /media/ {
            alias /home/<myusername>/webapps/<myapp>/<myproject>/media/;
        }
 
        location ~* /favicon.ico {
            root /home/<myusername>/webapps/<myapp>/<myproject>/media/img/;
            #Let's not waste time logging this access...
            access_log off;
        }
    }
}

Again remember to replace , and with your actual values.

Right now let’s setup nginx to talk to our fastcgi django process add this section into the configuration file after the favicon.ico location:

location / {
    proxy_set_header  	         X-Real-IP  $remote_addr;
    proxy_set_header             X-Forwarded-For $proxy_add_x_forwarded_for;
    fastcgi_pass                unix:/home/<myusername>/webapps/<myapp>/<myproject>/django.sock;
    fastcgi_pass_header          Authorization;          
    fastcgi_hide_header          X-Accel-Redirect;
    fastcgi_hide_header          X-Sendfile;
    fastcgi_intercept_errors     off;
    include                        fastcgi_params;
}

Our nginx.conf should now look like this:

#2 workers x 1024 connections gives us some resiliance and should be enough for most sites.
worker_processes 2;
 
events {
    worker_connections 1024;
}
 
http {
    include             mime.types;
    default_type        application/octet-stream;	
 
    server {
        listen  myport;
        server_name mydomain;
        client_max_body_size 5m;
 
        location /media/ {
           alias /home/<myusername>/webapps/<myapp>/<myproject>/media/;
        }
 
        location ~* /favicon.ico {
           root /home/<myusername>/webapps/<myapp>/<myproject>/media/img/;
           #Let's not waste time logging this access...
           access_log off;
        }
 
        location / {
            proxy_set_header  	        X-Real-IP  $remote_addr;
            proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
            fastcgi_pass                unix:/home/<myusername>/webapps/<myapp>/<myproject>/django.sock;
            fastcgi_pass_header         Authorization;          
            fastcgi_hide_header         X-Accel-Redirect;
            fastcgi_hide_header         X-Sendfile;
            fastcgi_intercept_errors    off;
            include                     fastcgi_params;
        }
    }
}

We need to make a slight change to the fastcgi_params file that is being included so that it works correctly with Django. Comment out the following line by placing a # in front of it:

fastcgi_param  SCRIPT_NAME		 $fastcgi_script_name;

and add this just above it:

fastcgi_param  PATH_INFO          $fastcgi_script_name;

You can download the complete sample files here zip / gz

Django configuration is pretty simple as we can use the default settings file for now:

# Django settings for myproject project.
 
DEBUG = True
TEMPLATE_DEBUG = DEBUG
 
ADMINS = (
    # ('Your Name', 'your_email@domain.com'),
)
 
MANAGERS = ADMINS
 
DATABASE_ENGINE = ''           # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
DATABASE_NAME = ''             # Or path to database file if using sqlite3.
DATABASE_USER = ''             # Not used with sqlite3.
DATABASE_PASSWORD = ''         # Not used with sqlite3.
DATABASE_HOST = ''             # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = ''             # Set to empty string for default. Not used with sqlite3.
 
# Local time zone for this installation. Choices can be found here:
# http://en.wikipedia.org/wiki/List_of_tz_zones_by_name
# although not all choices may be available on all operating systems.
# If running in a Windows environment this must be set to the same as your
# system time zone.
TIME_ZONE = 'America/Chicago'
 
# Language code for this installation. All choices can be found here:
# http://www.i18nguy.com/unicode/language-identifiers.html
LANGUAGE_CODE = 'en-us'
 
SITE_ID = 1
 
# If you set this to False, Django will make some optimizations so as not
# to load the internationalization machinery.
USE_I18N = True
 
# Absolute path to the directory that holds media.
# Example: "/home/media/media.lawrence.com/"
MEDIA_ROOT = ''
 
# URL that handles the media served from MEDIA_ROOT. Make sure to use a
# trailing slash if there is a path component (optional in other cases).
# Examples: "http://media.lawrence.com", "http://example.com/media/"
MEDIA_URL = ''
 
# URL prefix for admin media -- CSS, JavaScript and images. Make sure to use a
# trailing slash.
# Examples: "http://foo.com/media/", "/media/".
ADMIN_MEDIA_PREFIX = '/media/'
 
# Make this unique, and don't share it with anybody.
SECRET_KEY = '4#(%&p)^37k2ihr-q-k73qau44ec&8k==gyxb)1+$pck@4+qp='
 
# List of callables that know how to import templates from various sources.
TEMPLATE_LOADERS = (
    'django.template.loaders.filesystem.load_template_source',
    'django.template.loaders.app_directories.load_template_source',
#     'django.template.loaders.eggs.load_template_source',
)
 
MIDDLEWARE_CLASSES = (
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
)
 
ROOT_URLCONF = 'myproject.urls'
 
TEMPLATE_DIRS = (
    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.
)
 
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
)

Finally we need to start our servers:

cd ~/webapps/<myapp>/
./sbin/nginx
cd <myproject>
python2.5 manage.py runfcgi maxchildren=1 maxspare=1 method=prefork socket=$PWD/django.sock pidfile=$PWD/django.pid > /dev/null 2>&1 $

If you haven’t done it already mount your app in a domain from the webfaction control panel and then browse to the correct url and you should see the following:

Django Welcome Screen

So there you have it – we have installed and configured Nginx + Django however we still have a few thing’s to do… In Part 2 I’ll show you a couple of ways to make sure that our site isn’t down for long if the web-server is rebooted or Django crashes.

In Part 3 I’ll show you how to hand off file uploads to Nginx and get a fancy upload progress bar into the bargin :D

This post was brought to you by the letters L and A and the following websites.

http://forum.webfaction.com/viewtopic.php?id=1981

https://calomel.org/nginx.html

Nginx & Django on Webfaction (Part 1a) – Installing

Richard Cooper — Thu, 13 Aug 2009 22:23:31 +0000

This is aimed at Installing and configuring Django running as a FCGI Process behind Nginx on WebFaction shared hosting but can be altered to work with an almost identical setup on Ubuntu 9.04 (Jaunty).

For those who don’t know Nginx (Engine X) is a Free, open-source, high-performance HTTP server and reverse proxy with the added advantage for shared hosting that it has a low memory footprint – and Django is a high-level Python Web framework with added magic ponies :)

I’m creating this tutorial series so that in 3 months time when I need to set this up again for another project I haven’t forgotten where I put my written notes… (This is based off several threads in the WebFaction Forum, some research into Nginx, and a lot of trial and error getting the upload module to work with Django)

Later on in this series I will be showing how to create fancy ajax upload progress indicators with most of the work done by Nginx, and also how to hand off the work of large downloads back to Nginx.

Right, Let’s get started…

First we need to create a new app in the Webfaction control panel. Create a “new app listening on port” and make a note of the port number

Next we need to SSH into our account to configure and install (if you are using windows you can use the Putty SSH client.)
First we need to make some sub directories in our webapp folder and install Django and Flup (needed for the FCGI support later)

cd ~/webapps/<myapp>
mkdir -p bin lib/python2.5
PYTHONPATH=lib/python2.5 easy_install-2.5 -d lib/python2.5 -s bin http://www.djangoproject.com/download/1.1/tarball/ flup
./bin/django-admin.py startproject myproject

Now we need to download and untar the latest stable versions of Nginx, Nginx_UploadProgress_Module and PCRE (for the embeded perl in Nginx)
NB. If you need any other modules for Nginx now would be the time to wget them :)

wget http://sysoev.ru/nginx/nginx-0.7.61.tar.gz
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-7.9.tar.gz
wget http://wiki.nginx.org/images/8/83/Nginx_uploadprogress_module-0.5.tar.gz
tar -xzf nginx-0.7.61.tar.gz
tar -xzf pcre-7.9.tar.gz
tar -xzf Nginx_uploadprogress_module-0.5.tar.gz

Now lets configure, make and install Nginx

cd pcre-7.9
./configure --prefix=/home/<myusername>/webapps/<myapp>/pcre
cd ../nginx-0.7.61
./configure --prefix=/home/<myusername>/webapps/<myapp> --with-pcre=/home/<myusername>/webapps/<myapp>/pcre-7.9/ --add-module=/home/<myusername>/webapps/<myapp>/nginx_uploadprogress_module/

Assuming we don’t have any errors

make
make install

Now lets clean up before going any further

cd ..
rm -fr nginx-0.7.61
rm *.gz
rm -fr pcre-7.9

At this point we should have a working install of Nginx and Django, in Part 1b we will show how to configure them both and get our first page served.

WebFaction Easy-Install useage.

Richard Cooper — Sat, 01 Aug 2009 11:37:40 +0000

I’m posting this here so that I can remember it :)

If you need a Python library to be available to *all* of your Python apps, then install it in the $HOME/lib/python2.5 directory. A shorter version of the command is:
Code:

1	easy_install-2.5 -s $HOME/bin -d $HOME/lib/python2.5 the_module_name

If you need a Python library to be available to *one* of your Python apps, then install it in the $HOME/webapps//lib/python2.5 directory for that app. You must preface the easy_install command with the PYTHONPATH variable, like so:
Code:

1	PYTHONPATH=$HOME/webapps/<appname>/lib/python2.5 easy_install-2.5 -s $HOME/webapps/<appname>/bin -d $HOME/webapps/<appname>/lib/python2.5 the_module_name

StoutLabs MooTools Slider

Richard Cooper — Fri, 31 Jul 2009 10:51:59 +0000

Daniel over at StoutLabs has produced a really nice MooTools based image slider, with code that is really easy to understand even for a MooTools beginner like me.

We needed to use something similar for a new project at FrozenSkys, which required a few tweaks (like adding fading) to the slider class, and Daniel has been kind enough to include these back into his project.

You can find the code and a demo over at his site.

Microformats, X2V, and testing…

Richard Cooper — Tue, 28 Apr 2009 20:02:41 +0000

I’ve been playing around with Brian Suda’s X2V transforms from the microformats.org site, and as I needed to get my hand back in with .NET and Visual Studio 2008 I have created a couple of aspx pages and a webservice to provide X2V services. These pages will allow you to pass a url as a parameter

i.e http://frozenskys.co.uk/hcard2vcard.aspx?url=http://microformats.org/wiki/hcard

and if the page contains a valid hCard or hCalendar get back the corresponding vCard or iCal. Once I have tidied up the code and added one or two more formats, I’ll post a zip of it here for peeps to play with.

I now need some help testing them so if anybody would like to have a play they can be found here:

http://frozenskys.co.uk/hcard2vcard.aspx

http://frozenskys.co.uk/hcal2vcal.aspx

and the web service can be found here:

http://frozenskys.co.uk/x2v.asmx

Let me know if anything breaks :-)

Why free software is worth what you pay for it…

Richard Cooper — Mon, 13 Apr 2009 11:09:02 +0000

We have recently swapped from running our virtual machines in VMWare to Sun’s Virtual Box. The performance of virtual box is equal if not better than VMWare, and we will save quite a bit of money on licenses, as all our users would require the workstation version of VMWare. All was well with this plan until we needed to shrink some of the machines after restoring large DB backups onto them. The command to do this is given as “VBoxManage modifyhd MyMachine.vdi compact” unfortunatly the output from this is currently:- ”Error: Shrink hard disk operation is not available!” You can see the help ticket for this at http://www.virtualbox.org/ticket/2833 and it makes pretty horrific reading if you are a fan of open source and expect decent customer support.

This brings me onto the rant section of this blog. If this was paid for software I could influence the decision – I could take my cash elsewhere, I could ask to have it fixed as part of my support contract, or several other routes to resolution. As this is “Free” software I am reduced to arguing with someone in a defect log item as to whether my issue (and several other peoples as well judging by the defect log) is going to get fixed.

And with comments like “You are in no position to tell us what to do or not. No amount of complaints from your side will change the priority of this issue.” it’s nice to know we are valued customers…

At the end of it all however we are still going to use VirtualBox, and I am just going to have to hope that Sun will ask their support staff to be a little friendlier.