Steps to Initiate Django project in Eclipse

1. create project folder
cd ~/workspace
django-admin.py startproject myproject
cd myproject
python manage.py startapp myapp


<< result >>
~/workspace
   myproject
      myproject
         __init__.py
         settings.py
         urls.py
         wsgi.py
      manage.py
      myapp
         __init__.py
         models.py
         views.py
         tests.py


2. create project in Eclipse
Eclipse menu > File > New > PyDev Project
   Project name = myproject
   Don't configure PYTHONPATH <-- default settings
   Next >> Finish

<< result >>
myproject and myapp icon as folder


3. initialize project in Eclipse
Eclipse menu > Project > Properties > PyDev - PYTHONPATH > click "Add Source Folder"
   select "myproject" root folder

<< result >>
myproject and myapp icon as package

Eclipse PyDev Project Explorer > right-click package myapp > New > PyDev Package
   Source Folder = /myproject
   Name = myapp.models
repeat to create package myapp.views, myapp.utils


4. create database for project
   4.1 log in to MySQL as root
        mysql -h localhost -u root -p'rootpassword'
   4.2 create database with default character set and collation
        CREATE DATABASE mydb DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
   4.3 create user and grant privileges
        CREATE USER 'django_dbuser'@'%' IDENTIFIED BY 'djangodbpassword';
        GRANT ALL ON mydb.* to 'django_dbuser'@'%';
        CREATE USER 'django_dbuser'@'localhost' IDENTIFIED BY 'djangodbpassword';
        GRANT ALL ON mydb.* to 'django_dbuser'@'localhost';



5. modify DATABASES in settings.py for project database, e.g.
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'mydb',            # Or path to database file if using sqlite3.
        # The following settings are not used with sqlite3:
        'USER': 'django_dbuser',
        'PASSWORD': 'djangodbpassword',
        'HOST': '',                      # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
        'PORT': '',                      # Set to empty string for default.
    }
}



6. modify INSTALLED_APPS in settings.py to use south
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    # Uncomment the next line to enable the admin:
    'django.contrib.admin',
    # Uncomment the next line to enable admin documentation:
    # 'django.contrib.admindocs',
    'south',
)



7. syncdb
  python manage.py syncdb

Passing Request from AngularJS to Django

transform POST data before send from Angular to Django (with array of object)

// NOTE: this 2 lines for Django to receive POST data
$httpProvider.defaults.headers.post['Content-Type'] = 'application/x-www-form-urlencoded; charset=UTF-8';
$httpProvider.defaults.transformRequest = function(data) {

   if (data === undefined) {

      return data;

   }

   // NOTE: if transform this way -> at Django use json.loads(request.body)

   // can support array of object (no need to use getlist() at Django side)

   return JSON.stringify(data);

   // NOTE: if transform this way -> at Django use request.POST['key']

   // array of object will be sent with key like 'rows[10][id]', 'row[10][revenue]'

   // need to parse at Django side

// return $.param(data);

}

Passing Django's CSRF token through AngularJS

option 1:

var csrf_token = $('input[name="csrfmiddlewaretoken"]').val();

$http.post("/import_actual_simple_xls", postData, {
    headers: {'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', "X-CSRFToken" : csrf_token},
}).success(function(data, status, headers, config) {
    //other things to do when success
    ...
});

option 2:
ImportControllers.config(['$httpProvider', function($httpProvider) {
    // NOTE: this 2 lines for Django's csrf_token   
    $httpProvider.defaults.xsrfCookieName = 'csrftoken';
    $httpProvider.defaults.xsrfHeaderName = 'X-CSRFToken';
}]);

AngularJS - Any way for $http.post to send request parameters instead of JSON?

When post data using AngularJS' $http or $http.post(),

at Django side, request.POST is an empty dict but request.body has JSON string.

Use method in this page to transform request data JSON string --> request data parameter,

e.g.

{"param1":"value1","param2":"value2","param3":"value3"} --> param1=value1&param2=value2&param3=value3

Set up global transformRequest function:

var app = angular.module('myApp');

app.config(function ($httpProvider) {
    $httpProvider.defaults.transformRequest = function(data){
        if (data === undefined) {
            return data;
        }
        return $.param(data);
    }
});


Sample non-global transformRequest per call:

var transform = function(data){
    return $.param(data);
}


$http.post("/foo/bar", requestData, {
    headers: { 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8'},
    transformRequest: transform
}).success(function(responseData) {
    //do stuff with response
    ....});


For Hidden Field to Send Form Data

use <input name="stock_code" type="hidden" value="{{ actual.code }}">

Excel File Upload

After spending THE WHOLE DAY surfing internet, trying to find the way to send form data (including uploaded file) from AngularJS front-end to Django backend, here are what I get.
jQuery Form is my HERO.
My solution here is

- use jQuery Form to submit form data and file upload to backend.

- use dataType: 'html' for option passed to jQuery Form (I didn't know why but 'xml' didn't work).

- pass callback function to jQuery Form to update models when got response back.

- in the callback function, use $scope.$apply() to force AngularJS refresh its model after posted.
- no progress bar, no upload multiple files here because I don't need it now, but I really appreciate if you could guide me how to do.
NOTE: an example of upload progress here >> http://jquery.malsup.com/form/progress.html

html part (sample form which has both primitive data (multiplier) and file upload (input_excel))

<div id="div_load_xls" class="user-form col-lg-8" ng-app='myApp' ng-controller='importXlsController'>


    <form id="frm_load_xls" ng-submit='submitXls()' class="form-horizontal" method="post">

    {% csrf_token %}

        <input id="id_multiplier" name="multiplier" type="text" />

        <input id="id_input_excel" name="input_excel" type="file" />

        <input type="submit" name="validate_simple_actual" value="Validate" />

    </form>

</div>




Javascript part

<script src='/static/lib/js/angular-1.0.8.js'></script>    

<script src='/static/lib/js/jquery.form.js'></script>    

<script language="javascript" type="text/javascript">

    // get CSRF token from form's hidden field (generated from {% csfr_token %})

    var csrf_token = $('input[name="csrfmiddlewaretoken"]').val();




    // define AngularJS module and controller

    var myAppModule = angular.module('myApp', []);

    myAppModule.controller('importXlsController', importXlsController);




    // controller definition

    function importXlsController($scope, $http) {

        // function called when click submit button (ng-submit in the above form)

        $scope.submitXls = function() {




         // submit options

         var options = {

            dataType: 'html', // don't know why, but I used 'xml' and it didn't work

            url: '/import_actual_simple_xls', // to Django backend

            success: $scope.showResponse // callback function to process response defined below

         }




         // submit form data

         $('#frm_load_xls').ajaxSubmit(options);

      };




      // callback function to process response

      $scope.showResponse = function(responseText, statusText, xhr, $form) {

          // update model fields

          // use $apply to force AngularJS view to refresh

          $scope.$apply( function() {

               $scope.messages = jQuery.parseJSON(responseText);

         });

        };

    };

</script>




django backend (urls.py omitted)

def import_actual_simple_xls(request):
    // read primitive form data


    mul = request.POST.get('multiplier', None)

    if mul:

        messages.append('multiplier = %s' % mul)

    else:

        messages.append('No multiplier')




    // get file uploaded (handle error if needed)

    input_excel = request.FILES.get('input_excel', None)

    if not input_excel:

        messages.append('No Xls')

    else:

        book = xlrd.open_workbook(file_contents=input_excel.read())

        if ('Question List' in book.sheet_names()):

            messages.append('xls OK')

        else:

            messages.append('xls NG')

    return JSONResponse(messages)

Display Progress for Long Process

Situation

- upload file from a HTML form with submit button
- after submit, processing file take too long and timeout occur
- want to display progress after submit request

Solution 1

- use thread to process request, thus return response immediately
- keep progress data in session
- use Ajax to submit and display progress
Drawback
- no progress shown during submit big file

Solution 2

- use submit button to submit request
- use thread to process request, thus return response immediately
- keep progress data in session
- redirect to progress page, which use Ajax to display progress

Detail for Solution : use thread to process request

- for uwsgi, enable thread in uwsgi.ini by adding this row
    # enable thread
    enable-threads=True
- in Django, use code below to create thread
    import threading
    ...

    def func():
        ...
        args = [arg1, arg2]
        t = threading.Thread(target=do_something, args=args)
        t.daemon = True
        t.start()

    def do_something(arg1, arg2):
        ...



Detail for Solution : use Ajax to submit form data

- use jQuery Form Plugin to submit form data (especially uploaded file) from ajax
  detail here >> http://jquery.malsup.com/form/#getting-started
  detail for available options >> http://jquery.malsup.com/form/#options-object

    options = {
        type: "POST",
        url: "/import_ajax", // target url to submit form data
        validate: true,
        data: {'validate_question_ajax': '1'}, // additional data to be submitted with other form data
        dataType: 'json',
        success: function(response, textStatus, jqXHR){
            ...
            // start timer (to display progress) only if request is submitted successfully             init_timer();
        },
        // callback handler that will be called on error
        error: function(jqXHR, textStatus, errorThrown){
            ...
        }
    };
    $('#MyForm').ajaxSubmit(options);


Detail for Solution : save progress in session

- code below in django to get/set progress in session

    // enforce no cache for progress
    // redirect to this method from url, e.g. "/import_progress"
    @cache_control(no_cache=True)
    def get_progress(request):
        // use session key in request to get session data
        s = SessionStore(session_key=request.session._session_key)
        progress = s.get('progress', None)
        is_done = s.get('is_done', None)
        if is_done:
            return JSONResponse({'message': 'done %s records.' % progress, 'detail': s['detail']})
        else:
            return JSONResponse({'message': 'process %s records.' % progress, })

    def set_progress(session_key, progress, detail=None, reset=False):
        s = SessionStore(session_key=session_key)
        s['progress'] = progress
        s['is_done'] = (detail is not None)
        if detail:
            s['detail'] = detail
        elif reset:
            s['detail'] = []
        s.save()

    def do_something(request):
        // reset progress data immediately before process request
        set_progress(request.session._session_key, 0, reset=True)


Detail for Solution : use Ajax to display progress

- code below in javascript

    var progress_timer;

    // a function to initialize timer (called later)
    function init_timer() {
        // set timer to update status of importing Slam history
        progress_timer = setInterval(function() {
            $.ajax({
                url: "/import_progress", // url that return progress data, redirect to method get_progress() above
                ...
                success: function(response, textStatus, jqXHR){
                    // output progress
                    $("#import_progress").html(response.message);
                    if (response.is_done != null) {
                        // stop timer when process completed
                        clearInterval(progress_timer);
                    }
                    // emergency braker
                    i += 1;
                    if (i > 1000) {
                        clearInterval(progress_timer);
                    }
                },
                ...
            });
        }, 5000); // timer interval in millisec
    }

    ...

        // pass this as ajax submit option
        // this will be returned immediately after worker thread started
        success: function(response, textStatus, jqXHR){
            if (response.error != null) {
                ...
            }
            else {
                // if worker thread started normally, start timer to display progress
                init_timer();
            }
        },


Detail Solution : HTML page to show progress

- code below in javascript to initialize timer when loaded page
    $(function() {
        jQuery(document).ready(function(e) {
            init_timer(); // same function as above
        });
    });

Django Admin Site Customization

1) enable customized Admin class

from django.contrib import admin

class QuestionAdmin(admin.ModelAdmin):
  pass

admin.site.register(Question, QuestionAdmin)

2) customize fields to display at list page

class QuestionAdmin(admin.ModelAdmin):
    # fields to display at list page
    list_display = ('id', 'subject', )
 
    # at list page, fields that when click will open individual edit page
    # NOTE: when edit from list page, also call save_model()
    # if save_model() depends on custom fields in custom form (which not exists in list page),
    # one work around is to call super class method to handle model save
    # see save_model() below.
    list_display_links = ['question_html']
 
    # fields to show at filter side bar
    list_filter = ('status', 'subject', 'source', 'year')
 
    # fields that can edit from list page
    list_editable = ('solution', 'question_type', 'point', 'status', )
 
    # - fields used for search when click search button
    # - can search across relationship, e.g. subject__name
    # - by default, search for objects contain case-insensitive specified keyword
    #   (i.e. in MySQL : WHERE source ILIKE '%Ent%')
    # - for faster search performance
    #   + prefix field name with '=' to search exact match, e.g. WHERE year='2555'
    #   + prefix field name with '^' to match beginning of the field
    search_fields = ['subject__name', 'source', '=year', 'tags__name', 'solution_html']

    # specified fields that are read-only
    # (enhance performance when display individual edit page)
    readonly_fields = ['created_time', 'student', 'question']

    # field used as date, and display as link in list page that when click can filter by that date field
    date_hierarchy = 'created_time'

    # specify default order when display list page
    ordering = ['-id']

    def save_model(self, request, obj, form, change):
        # if not change from QuestionAdminForm, use default save_model()
        if not isinstance(form, QuestionAdminForm):
            super(QuestionAdmin,self).save_model(request, obj, form, change)
            return

Some sample screens below.

2.1) display link in list page

by using a function in list_display, which return url and set allow_tags = True. 

class ModelAdmin(admin.ModelAdmin):
    list_display = ('id', 'view_link')
    def view_link(self, obj):
            q = Question.objects.get(node_id=obj.id)
            return u'<a href="%s">%s</a>' % (q.get_absolute_url(),q.name)
    view_link.allow_tags = True


2.2) filter data shown in list page

by overriding queryset() as shown below.

def queryset(self, request):
qs = super(NodeAdmin, self).queryset(request)
return qs.filter(type__in=[2, 3, 4])


2.3) collapse list filter sidebar (with jQuery)

detail here >> https://gist.github.com/abyx/1017597

class QuestionAdmin(admin.ModelAdmin):
    list_filter = ['activity_flag']

    class Media:
            js = ['/static/js/list_filter_collapse.js' ] # path to JavaScript file

content of list_filter_collapse.js

(function($){
ListFilterCollapsePrototype = {
    bindToggle: function(){
        var that = this;
        this.$filterTitle.click(function(){
            that.$filterContent.slideToggle();
            that.$list.toggleClass('filtered');
        });
    },
    init: function(filterEl) {
        this.$filterTitle = $(filterEl).children('h2');
        this.$filterContent = $(filterEl).children('h3, ul');
        $(this.$filterTitle).css('cursor', 'pointer');
        this.$list = $('#changelist');
        this.bindToggle();
    }
}
function ListFilterCollapse(filterEl) {
    this.init(filterEl);
}
ListFilterCollapse.prototype = ListFilterCollapsePrototype;

$(document).ready(function(){
    $('#changelist-filter').each(function(){
        var collapser = new ListFilterCollapse(this);
    });
});
})(django.jQuery);

3) use inline to display/edit/add related object

# subclass from admin.TabularInline
class SectionResourceInline(admin.TabularInline):
    # model used in this inline
    model = SectionResource
    # fields display in this inline
    fields = ('type', 'seq_no', 'node')
    # specify if to let user input id instead of select from drop-down list
    # to improve performance
    raw_id_fields = ['node']
    # extra row displayed to add new data
    extra = 1


4.1) use custom form to display/edit individual object

from django.contrib import admin

class QuestionAdminForm(forms.ModelForm):
    ...

    class Meta:
        model = Question

class QuestionAdmin(admin.ModelAdmin):
    form = QuestionAdminForm
 
    # group fields in custom Form
    # 1st element in tuple = verbose name
    # 2nd element in tuple = dict with grouped fields, each inner tuple are fields in same line
    # e.g. group 'Information' has 2 lines,
    # first line has field difficulty, second line has source, year, seq_no
    # can specify custom fields together with fields in Model (e.g. new_chapter below)
    fieldsets = (
        ('Content', {'fields': (('subject', 'chapter', 'new_chapter'), ), }),
        ('Information', {'fields': ('difficulty',
                                    ('source', 'year', 'seq_no'),
                                   ),
                        }),
    )
    ...

Sample screen here.

4.2) keep original field value to use when save

in custom form : override __init__() to keep original field value somewhere.
class NodeAdminForm(forms.ModelForm):
    def __init__(self, *args, **kwargs):
        super(NodeAdminForm, self).__init__(*args, **kwargs)

        # initiate values for custom fields if change_form
        node = kwargs.pop('instance', None)
        if node:
            self.old_size = node.size

    class Meta:
        model = Node
in admin class : refer to original field value in form when save object
class NodeAdmin(admin.ModelAdmin):
    form = NodeAdminForm
    def save_model(self, request, obj, form, change):
        if form.old_size != obj.size:
            # do something
        super(NodeAdmin, self).save_model(request, obj, form, change)


4.3) use custom fields in Form

class QuestionAdminForm(forms.ModelForm):
    # queryset : to get available choices
    # widget : use admin.widgets.FilteredSelectMultiple to display 2 boxes : available choices <--> selected choices
    #          2nd parameter : True if display 2 boxes vertically, False if display horizontally
    #                          rows in attrs is approximate size of widget?
    internal_label = forms.ModelMultipleChoiceField(label='Internal Label', required=False,queryset=Label.objects.filter(type=Label.INTERNAL_LABEL_TYPE).order_by('name'),widget= admin.widgets.FilteredSelectMultiple('Internal Label', False, attrs={'rows':'10'})
                       
                       

    # show as dropdown widget (single select)
    # use ModelMultipleChoiceField for multiple select dropdown
    chapter = forms.ModelChoiceField(queryset=Chapter.objects.all().order_by('name'), required=False)

    def __init__(self, *args, **kwargs):
        # to display other fields in model
        super(QuestionAdminForm, self).__init__(*args, **kwargs)

        # set required fields
        self.fields['field_name'].required = True
        ...

        # use RelatedFieldWidgetWrapper to add green PLUS sign besides FilteredSelectMultiple widget
        # detail here >> http://dashdrum.com/blog/2012/07/relatedfieldwidgetwrapper/
        # 1st parameter = widget to be wrapped
        # 2nd parameter = a relation that defines the two models involved
        # 3rd parameter = a reference to the admin site (assigned from Admin class' __init__())
        #                 QuestionLabel is a class that has ForeignKey to both Question and Label
        self.fields['internal_label'].widget = admin.widgets.RelatedFieldWidgetWrapper(self.fields['internal_label'].widget,
QuestionLabel._meta.get_field('label').rel, self.admin_site)
    class Meta:
        # model related to Form
        model = Question

        # specify special widget for field
        # AdminFileWidget is a widget to load file
        widgets = {'question_html': admin.widgets.AdminFileWidget, }


class QuestionAdmin(admin.ModelAdmin):
    form = QuestionAdminForm

    def __init__(self, model, admin_site):
        super(QuestionAdmin,self).__init__(model, admin_site)
     
        # capture the admin_site
        # for use with RelatedFieldWidgetWrapper in QuestionAdminForm
        self.form.admin_site = admin_site

Sample screen here.

4.4) initiate default value in custom Form

class QuestionAdminForm(forms.ModelForm):
    ...

    def __init__(self, *args, **kwargs):
        ...

        # get Model object from kwargs
        question = kwargs.pop('instance', None)
        if question:
            # for simple fields, simply set value
            self.question_html = question.question_html

            # for single select choice field, simply set selected object id
            self.initial['chapter'] = question.get_chapter_id()

            # for multiple select choice field,
            # create a dict with key = selected object id, value = True,
            # then set as initial value
            selected_internal_labels = {}
            for r in Label.objects.filter(...):
                selected_internal_labels[r.id] = True
            self.initial['internal_label'] = selected_internal_labels


4.5) validate input data (before calling Admin class' save_model())

NOTE: method call sequence : ModelForm.clean() > Model.clean() > ModelAdmin.save_model() > Model.save()
so validation can occur at ModelForm.clean() or Model.clean()

class QuestionAdminForm(forms.ModelForm):
    ...

    def clean(self):
        # default validation first
        cleaned_data = super(QuestionAdminForm, self).clean()

        # custom validation follows
        # raise a ValidationError and error message will be displayed at top of edit Form
        # ValidationError raised in Model.clean() also displayed in the same way
        if some_condition:
            raise ValidationError('subject mismatch')

        # don't forget to return cleaned data
        return cleaned_data


4.6) save model from custom Form

NOTE: method call sequence : ModelForm.clean() > Model.clean() > ModelAdmin.save_model() > Model.save()
so setting default value can be done at ModelAdmin.save_model() or Model.save()

class QuestionAdmin(admin.ModelAdmin):
    ...

    def save_model(self, request, obj, form, change):
        # call super class' save_model() to handle changes from list page
        # (if we use custom form with custom fields not in list page)
        if not isinstance(form, QuestionAdminForm):
            super(QuestionAdmin,self).save_model(request, obj, form, change)
            return

        # obj = model object to be saved
        # can set default value from request
        # NOTE: use request.POST.get() to avoid exception when key not exists (i.e. not input from Form)
        # NOTE: when get a list from request object, use request.POST.getlist('a_key_to_list_value'),
        #       not simply call request.POST.get()
        obj.created_by_id = request.user.id

        # can set default value from form data
        obj.question_html = form.question_html

        # call super class' save_model() method for default process
        super(QuestionAdmin,self).save_model(request, obj, form, change)

        # then some special process follows
        ...

        # call message_user() to display some messages
        self.message_user(request, 'done...')

5) specific action in Admin Form

detail here >> https://docs.djangoproject.com/en/1.5/ref/contrib/admin/actions/

class QuestionAdmin(admin.ModelAdmin):
    actions = ['my_action']

    # queryset contain objects selected by user from list page
    def my_action(self, request, queryset):
        # for each selected object, do something
        for r in queryset:
            do_something(r)
     
        # show some message after completed
        self.message_user(request, "successfully do something")
    # a verbose description to show in page
    my_action.short_description = 'Do something for some purpose'



Django Login Redirect

at login.html
<input type="hidden" name="next" value="{{ next }}" />


at urls.py
urlpatterns += patterns('',
    url(r'^login$', 'user_login'),
    ...
)


at method to process login (e.g. user_login())
def apu_login(request):     ...
    if request.method == "GET":
        return render(request, "login.html", {'next': request.GET.get('next', '/default_page')})
    ...
    return HttpResponseRedirect(request.POST.get('next', '/default_page'))



at method that require login, add decorator e.g. like below (require user to be staff)
from django.contrib.auth.decorators import user_passes_test
...
@user_passes_test(lambda u: u.is_staff)
def import_excel_view(request):


    ...

Django Sample Template 500 & 404

detail here >> http://david.feinzeig.com/blog/2012/02/18/tips-for-creating-404-page-not-found-and-500-server-error-templates-in-django-plus-configuring-email-alerts/

500.html (for server error)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">

<head>
 <title>Page unavailable</title>
</head>

<body>
 <h1>Page unavailable</h1>

 <p>Sorry, but the requested page is unavailable due to a server hiccup.</p>

 <p>Our engineers have been notified, so check back later.</p>
</body>

</html>

404.html (for page not found)
{% extends "base.html" %}
{% load i18n %}

{% block title %}Page not found{% endblock %}

{% block content %}
<h1>Page not found</h1>

<p>Sorry, but the requested page could not be found.</p>
{% endblock %}


Create Script to Run Django Method

Sample script to run Django method from command line.
Assume there is
- /path/to/my/project/settings.py for Django settings
- /path/to/my/project/mymodule/models/mymodel.py
- Django class myclass in the above mymodel.py with method mymethod()

#!/usr/bin/python2.7

import os
import sys

sys.path.append('/path/to/my/project/')
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")

from mymodule.models.mymodel import myclass

myclass.mymethod()

Sample script that output result to a log file with date&time in file name as below (can use with Python 2.6+).
#!/usr/bin/python2.7
from __future__ import print_function
import os
import sys

sys.path.append('/path/to/my/project/')
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")

from datetime import datetime
from mymodule.models.mymodel import myclass

t = datetime.now()
fname = '/path/to/log/folder/%s.log' % t.strftime('%Y-%m-%d_%H%M%S')
f = open(fname, 'w')
print('done some task with result = %s' % myclass.method(), file=f)
f.close()

a log will be written to file like /path/to/log/floder/2013-07-09_151201.log

Created Timestamp and Updated Timestamp in Django

Added following fields in Django model for both timestamp
    created_time = models.DateTimeField(auto_now_add=True, blank=True)
    last_updated = models.DateTimeField(auto_now=True, blank=True)


NOTE: about timezone
There are 2 settings in settings.py
    TIME_ZONE = 'Asia/Bangkok'
    USE_TZ = False

if set like above, DateTimeField with auto_now and auto_now_add will use local time as indicated in TIME_ZONE, e.g. 17:00:00 (GMT+7:00)

however, if set USE_TZ = True, auto_now and auto_now_add will use UTC time, e.g. 10:00:00

detail here >> http://stackoverflow.com/questions/16377586/models-datetimefieldauto-now-add-true

Django ManytoMany Field

Sample model
class Chapter(models.Model):
    name = models.CharField(max_length=100)
    tags = models.ManyToManyField(Tag, blank=True, null=True)


Create Relationship - One by One
    new_chapter = Chapter.objects.create(name=chapter_name)
    new_tag = Tag.objects.create(name=tag_name)
    new_chapter.tags.add(new_tag)


Create Relationship - Multiple
    new_section = Section.objects.create(name=section_name, chapter=parent_chapter)
    new_section.tags = list(parent_chapter.tags)


Iterate ManyToMany relationship
    for tag in parent_chapter.tags.all():
        new_section.tags.add(tag)


Custom Field in Django with South Support

To define custom field in Django, here is an example:
detail here >> http://djangosnippets.org/snippets/2513/ and http://djangosnippets.org/snippets/1244/

class TinyIntegerField(models.SmallIntegerField):
    def db_type(self, connection):
        if connection.settings_dict['ENGINE'] == 'django.db.backends.mysql':
            return "tinyint"
        else:
            return super(TinyIntegerField, self).db_type(connection)

    def get_internal_type(self):
        return "TinyIntegerField"

in this example, db_type() tell Django to use field type tinyint if backend is MySQL

Using this custom field type, when try to migrate with South, this error happened.
 ! South cannot introspect some fields; this is probably because they are custom
 ! fields. If they worked in 0.6 or below, this is because we have removed the
 ! models parser (it often broke things).
 ! To fix this, read http://south.aeracode.org/wiki/MyFieldsDontWork

To fix this problem, add a method south_field_triple in your custom field class
detail here >> http://south.readthedocs.org/en/latest/customfields.html

    def south_field_triple(self):
        from south.modelsinspector import introspector
        field_class = self.__class__.__module__ + '.' + self.__class__.__name__
        args, kwargs = introspector(self)
        return (field_class, args, kwargs)



How to Use South in Django to Manage Database

Overview

South is a module to help managing database tables when you update models in Django.
Steps to start using South described below.

Described here is case with existing project + database just created + no table created yet
see detail for other cases here >> http://stackoverflow.com/questions/4840102/how-come-my-south-migrations-doesnt-work-for-django

Install South

detail here >> http://south.readthedocs.org/en/0.7.6/
  sudo easy_install South

Initiate South in Legacy Project + No Database

1. add South to INSTALLED_APPS in your settings.py file (not add your application at this step)
2. run python manage.py syncdb
    this will create South table south_migrationhistory in your database.
    skip this step, you will encounter error "Table 'app_name.south_migrationhistory' doesn't exist" when you run command python manage.py migrate app_name in step 5
3. now add your application to INSTALLED_APPS in your settings.py file
4. run python manage.py schemamigration app_name --initial
    this will create initial migration record
5. run python manage.py migrate app_name
If any error occur, you can reset South history as described below.

Reset South History 

detail here >> http://stackoverflow.com/questions/4625712/whats-the-recommended-approach-to-resetting-migration-history-using-django-sout
in case of any error, use following commands

  rm -r appname/migrations/ 
  python manage.py reset south << see NOTE below before run this command
  python manage.py convert_to_south appname 
then drop all related database tables, and initiate South again as described above.
NOTE:
- reset south will delete ALL migration history for all apps that you have installed.
  avoid using this if you only want to remove migration history of some specific app. selectively delete from table south_migrationhistory instead.
  e.g. delete from south_migrationhistory where app_name = 'homepage';
- last command, convert_to_south, use to avoid dropping all your tables. it is equivalent to these two commands
  python manage.py schemamigration --initial app_name_1
  python manage.py schemamigration --initial app_name_2
     ...
  python manage.py migrate --fake

Ongoing Maintenance

after modify models, run following commands
  python manage.py schemamigration app_name --auto
  python manage.py migrate app_name

Django Model Cheat Sheet

Field Types + Specific Options

<< boolean >>
- BooleanField << True/False
- NullBooleanField << null/True/False

<< number >>
- IntegerField
- SmallIntegerField
- PositiveIntegerField
- PositiveSmallIntegerField
- BigIntegerField << 64-bit integer
- AutoField << auto-incremental IntegerField
- DecimalField
  max_digits, decimal_places << 999.99 : max_digit=5, decimal_places=2
- FloatField

<< text >>
- CharField
  max_length << length in characters
- TextField
- SlugField
- CommaSeparatedIntegerField
- URLField
  verify_exists
- EmailField
  max_length=75
- IPAddressField
- GenericIPAddressField

<< date/time >>
- DateField, DateTimeField, TimeField
  auto_now=False << set current timestamp for every save
  auto_now_add=False << set current timestamp when first create

<< file >>
- FileField << file upload field
  upload_to << path appended to MEDIA_ROOT; can be callable object; can use strftime() formatting
  max_length=100
- ImageField << subclass from FileField
  height_field, width_field << auto-set
- FilePathField

<< relationship >>
- ForeignKey(one-sided model) << many-to-one relationship
  limit_choices_to << effect Admin/ModelForm choices
    e.g. limit_choices_to = {’pub_date__lte’: datetime.now}
  related_name << backward relationship (1 --> many)
  to_field
  on_delete=models.CASCADE << emulate ON DELETE CASCADE behavior
    e.g. user = models.ForeignKey(on_delete=models.SET_NULL) << when delete parent, set reference to NULL
    CASCADE = also delete child when parent is deleted
    PROTECT = cannot delete parent if child exists
    SET_NULL
    SET_DEFAULT = set to default value if parent deleted
    SET()
    DO_NOTHING
- ManyToManyField(other model) << many-to-many relationship
  related_name << same as ForeignKey
  limit_choices_to << same as ForeignKey
  symmetrical << for self-reference
  through << intermidiary model (for additional data)
  db_table << table to store many-to-many relationship
- OneToOneField << one-to-one relationship
  parent_link=False << True=link back to parent class (in subclass model)

(General) Field Options

- null=False << allow null in database
- blank=False << for validation
- default << default value
- choices << value stored + value displayed
    YEAR_IN_SCHOOL_CHOICES = (
        (’FR’, ’Freshman’),
        (’SO’, ’Sophomore’),
        (’JR’, ’Junior’),
        (’SR’, ’Senior’),
        (’GR’, ’Graduate’),
    )
    MEDIA_CHOICES = (
        (’Audio’, (
            (’vinyl’, ’Vinyl’),
            (’cd’, ’CD’),
        )),
        (’Video’, (
            (’vhs’, ’VHS Tape’),
            (’dvd’, ’DVD’),
        )),
        (’unknown’, ’Unknown’),
    )
- db_column << column name in database (override auto-generated column name)
- db_index << True = create index for this field
- editable=True << False = not displayed for edit in form
- unique << True = unique value in field
  unique_for_date, unique_for_month, unique_for_year
- primary_key << True = primary key for model (override auto-generated PK field)
- help_text << display with form widget
- verbose_name << human-readable name

Meta Options

- verbose_name, verbose_name_plural
  human-readable name for model
- unique_together
  combinated unique fields, e.g. unique_together = (("driver", "restaurant"),)
- abstract
  True=model for abstract class
- db_table
  database table used for model
- get_latest_by
  DateField/DateTimeField used to identify latest data (use in manager's latest() method)
- ordering
  default order when get, e.g. ordering = [’-order_date’] << order by order_date descending
- order_with_respect_to
  True=children has order related to its parent (add additional column suffixed _order)
  set at child model;
  parent model can get ordered children using get_<RELATED>_order()
  parent model can set children order using set_<RELATED>_order()
  child model can use get_next_in_order(), get_previous_in_order()
- app_label
  use when model exists outside standard models.py

Prepare Python + Django Development Environment

Here are steps I set up Ubuntu for web development using
- MySQL 5.5 + MySQL Workbench + phpMyAdmin
- Apache2
- Python 2.7.3 + Django 1.4.5
- Eclipse + PyDev + Color Theme
- South 0.7.6 (library to manage model sync with database in Django)
- Django Debug Toolbar

Install MySQL 5.5

detail here >> https://help.ubuntu.com/12.04/serverguide/mysql.html
  to install MySQL (root password = admin....)
    sudo apt-get install mysql-server
  to check if MySQL Server is running
    sudo netstat -tap | grep mysql
  to start/stop/restart MySQL
    sudo service mysql stop
    sudo service mysql restart
  to change config file
    config file at /etc/mysql/my.cnf
    after change, restart MySQL
  to change root password
    sudo dpkg-reconfigure mysql-server-5.5

Install MySQL Tuner

detail here >> https://help.ubuntu.com/12.04/serverguide/mysql.html
  to install
    sudo apt-get install mysqltuner
  to run
    mysqltuner

Install MySQL Workbench

detail here >> http://www.upubuntu.com/2012/12/install-mysql-workbench-5244-from-ppa.html?m=0
  to install
    sudo add-apt-repository ppa:olivier-berten/misc
    sudo apt-get update
    sudo apt-get install mysql-workbench
  to run
    mysql-workbench &

Install Apache2

  to install
    sudo apt-get install apache2
  to test
    http://127.0.0.1

  file location
    document root at /var/www
    config file at /etc/apache2/apache2.conf
    additional config at /etc/apache2 directories, e.g.
       /etc/apache2/mods-enabled (for Apache modules),
       /etc/apache2/sites-enabled (for virtual hosts), and
       /etc/apache2/conf.d.

    log file at /var/log/apache2

Install phpMyAdmin

detail here >> https://www.digitalocean.com/community/articles/how-to-install-and-secure-phpmyadmin-on-ubuntu-12-04
  sudo apt-get install phpmyadmin
    Select Apache2 for the server
    Choose YES when asked about whether to Configure the database
  edit /etc/apache2/apache2.conf to add following line
    Include /etc/phpmyadmin/apache.conf
  sudo service apache2 restart << restart Apache2
  test access to http://localhost/phpmyadmin

Install Python 2.7

No need for this step since Python 2.7.3 is installed by default on Ubuntu 12.04

NOTE: Python files at /usr/local/lib/python2.7/

Install Django 1.4

1. install mod_wsgi (for Apache2, similar to mod_php to link Apache with script)
  sudo apt-get install libapache2-mod-wsgi
2. install Python library for install Python package
  sudo apt-get install python-setuptools python-pip
3. install Django
  sudo pip install django==1.4.5
4. test Django
  python
  >>> import django
  >>> django.VERSION
  (1, 4, 5, 'final', 0)

NOTE: django files at /usr/local/lib/python2.7/dist-packages/django/

Install Eclipse

detail here >> http://www.linoob.com/2011/09/starting-with-python-on-eclipse-in-ubuntu/
detail for color theme here >> http://marketplace.eclipse.org/content/eclipse-color-theme#.UeCmfxcW3Zg
detail for template highlight here >> http://eclipse.kacprzak.org/

1. install Eclipse from Ubuntu Software Center (no need to install java before hand)
2. setup Pydev
   2.1 from Eclipse menu > Help > Install New Software > Add button
   2.2 input Name=PydevEnv(or what ever), URL=http://pydev.org/updates then click OK
   2.3 click Works With field, select Pydev from list below, then click Next, Next, and follow the steps
3. configure Pydev
  from Eclipse menu > Window > Preferences > PyDev > Interpreter Python > click AutoConfig > click OK for default selelction
  (no need to select the first one for Pydev)
4. [optional but nice to have] add color theme for eclipse
   4.1 from Eclipse menu > Help > Install New Software > Add button
   4.2 input Name=(what ever), URL=http://eclipse-color-theme.github.com/update then click OK
   4.3 select Eclipse Color Theme from list below, then click Next, Next, and follow the steps.

   4.4 to change color theme, go to menu Window > Preference > General > Appearance > Color Theme (for dark theme I use Sunburst).

5. [optional but nice to have] add highlight for Django template

   5.1 from Eclipse menu > Help > Install New Software > Add button
   5.2 input Name=(what ever), URL= http://eclipse.kacprzak.org/updates then click OK
   5.3 select Django Template Editor from list below, then click Next, Next, and follow the steps.
   5.4 to change settings, go to menu Window > Preference > Django Editor

Install MySQL for Python (MySQL connector for Python programming)

detail here >> http://blog.mattwoodward.com/2012/08/installing-mysql-python-module-on-ubuntu.html
1. (if not installed yet) Install pip
  sudo easy_install pip
2. (just for sure) upgrade pip
  sudo pip install pip --upgrade
3. build the dependencies for the python-mysqldb libraries
  sudo apt-get build-dep python-mysqldb
4. use pip to install the Python MySQL libraries
  sudo easy_install -U distribute
  sudo pip install MySQL-python



Install South (database migration in Django)

detail here >> http://south.readthedocs.org/en/0.7.6/
1. Install South (database migration in Django)
  sudo easy_install South
2. register to Eclipse
  go to menu Window > Preferences > PyDev > Interpreter-Python
  click button 'New Folder' to add
  /usr/local/lib/python2.7/dist-packages/South-0.7.6-py2.7.egg

Install Django Debug Toolbar (provide additional debug information)

detail here
>> https://github.com/django-debug-toolbar/django-debug-toolbar
     very detail in installation/configuration in this official site
>> http://www.packtpub.com/article/django-debug-toolbar
     explain screen image and usage

1. install debug tool bar
  sudo easy_install django_debug_toolbar
2. add 'debug_toolbar.middleware.DebugToolbarMiddleware', to the end of middleware classes in project settings
3. edit INTERNAL_IPS = ('127.0.0.1', ) in my django.global_settings.py
   by default, debug toolbar is displayed only when DEBUG is True and run from IP listed in INTERNAL_IPS.
4. add 'debug_toolbar', to the INSTALLED_APPS in project settings.py
<< optional >>
5. add a tuple called DEBUG_TOOLBAR_PANELS to settings.py to specify the full Python path to panels that are included in toolbar.
    # comment out unnecessary panels or re-order if needed
    DEBUG_TOOLBAR_PANELS = (
        'debug_toolbar.panels.version.VersionDebugPanel',
        'debug_toolbar.panels.timer.TimerDebugPanel',
        'debug_toolbar.panels.settings_vars.SettingsVarsDebugPanel',
        'debug_toolbar.panels.headers.HeaderDebugPanel',
        'debug_toolbar.panels.request_vars.RequestVarsDebugPanel',
        'debug_toolbar.panels.template.TemplateDebugPanel',
        'debug_toolbar.panels.sql.SQLDebugPanel',
        'debug_toolbar.panels.signals.SignalDebugPanel',
        'debug_toolbar.panels.logger.LoggingPanel',
    )
6. add a dictionary called DEBUG_TOOLBAR_CONFIG to settings.py to specify additional configuration
    DEBUG_TOOLBAR_CONFIG = {
        'INTERCEPT_REDIRECTS': False,
        'SHOW_TOOLBAR_CALLBACK': None,
        'EXTRA_SIGNALS': ['myproject.signals.MySignal'],
        'HIDE_DJANGO_SQL': False,
        'SHOW_TEMPLATE_CONTEXT': False,
        'TAG': 'div',
        'ENABLE_STACKTRACES' : True,
    }
7. add the debug_toolbar directory to your Python path (so far not needed)

NOTE: The debug toolbar will only display itself if the mimetype of the response is either text/html or application/xhtml+xml and contains a closing tag.

Create Sample Django Project

1. create project folder

  cd /home/somnuk/workspace

  django-admin.py startproject twc

  << ls to see result >>

  somnuk@somnuk-NB:~/workspace$ ls twc

  manage.py  twc

  somnuk@somnuk-NB:~/workspace$ ls twc/twc

  __init__.py  settings.py  urls.py  wsgi.py

2. create file /home/somnuk/workspace/twc/apache/django.wsgi with following content

  NOTE: make sure no indent in this file (indent in Python has meaning)

    import os

    import sys

    sys.path.append('/home/somnuk/workspace/twc')

    os.environ['DJANGO_SETTINGS_MODULE'] = 'twc.settings'

    import django.core.handlers.wsgi

    application = django.core.handlers.wsgi.WSGIHandler()

3. create necessary folders

  mkdir /home/somnuk/workspace/twc/static

  mkdir /home/somnuk/workspace/twc/media

  mkdir /home/somnuk/workspace/twc/django-static

  NOTE:

  - corresponding to file twc.conf created in step 4

  - if these folders not exists, Internal Server Error when test in step 6

4. create file /etc/apache2/sites-available/twc.conf with following content

  Alias /robots.txt /home/somnuk/workspace/twc/static/robots.txt

  Alias /favicon.ico /home/somnuk/workspace/twc/static/favicon.ico

  AliasMatch ^/([^/]*\.css) /home/somnuk/workspace/twc/static/styles/$1

  Alias /media/ /home/somnuk/workspace/twc/media/

  Alias /static/ /home/somnuk/workspace/twc/django-static/

  <Directory /home/somnuk/workspace/twc/django-static>

    Order deny,allow

    Allow from all

  </Directory>

  <Directory /home/somnuk/workspace/twc/media>

    Order deny,allow

    Allow from all

  </Directory>

  WSGIScriptAlias / /home/somnuk/workspace/twc/apache/django.wsgi

  <Directory /home/somnuk/workspace/twc/apache>

    Order allow,deny

    Allow from all

  </Directory>

5. enable site

  sudo a2ensite twc.conf

  sudo /etc/init.d/apache2 restart

6. access http://127.0.0.1

  if error, check apache log at /var/log/apache2