Tutorial: Using Django and Jquery

image001

I had some difficulty recently finding a web tutorial specifically aimed at the use of Django and Jquery. There is an immense wealth of information on each topic but little on using them together – how to pass arguments to Django, how to receive it and what data formats JQuery expects back. After playing around with them and seeing how they click together, I thought I’d write this tutorial.

Background

I recently had an idea for a little web app that could be useful for some guys I work with. Developing such a tool could save them time and possibly give them a better way of visualising a metric that they monitor. Personally, the main reason I pursued it was because it was an opportunity for me to learn a couple of new technologies (namely Django and JQuery; specifically the excellent DynaTree, nvd3 and DataTable JQuery packages).

Note that I worked on this entirely on my personal spare time and I fabricated my own data (and it ultimately was not adopted). However, I will only speak in this tutorial of my experience learning the technologies – the business details will remain ambiguous.

Versions used

The web app to be developed

Let’s start off with some objects with a tree-style relationship. If these objects were to be stored persistently, it would likely be as database rows, which can be difficult to visiualise. So why not present this in a tree?

1.table2Tree1

Additionally, let’s say each object in the tree had timeseries data for variations of a certain metric. It would cool to display this metric for the selected nodes in the tree, triggering the different variations depending on the option(s) selected.

2.tree2Chart

Better still, since I’ve already got the time series, why not present it in a paginated table that the user could download?

3.tree2Table

The web app would look like this…

4.all-together

To make it responsive, I didn’t want to the page to reload each time a setting changed. Wouldn’t it be cool to select various multiple nodes on the tree and get the chart and table sections of the page to automatically update? There are those variations in the timeseries data so there would need to be additional input widgets for those. The chart and table would also update when these are modified.

So let’s take sneak peek at what we’ll actually be building (with sensitive details omitted, of course)…
image001

Time to learn some Ajax!

Before we proceed, why Jquery and Django?

Firstly, the first requirement was the responsiveness of the web app. Being browser-based, the client-side processing would need to be done via Javascript; the choice to be made was over the JS library to simplify the coding. There are many options besides Jquery, say Node.js, but Jquery anecdotally was the most popular one in terms of available tutorial, plugins or developers I knew that used it.

Secondly, Django. I opted for that because I didn’t want to relearn a lanuage for this experiment. Python is particularly easy to program. Additionally, a web framework like Django would have all the elements in place such that I wouldn’t need to rewrite pre-existing patterns or learn multiple new modules.

Breaking down the work

There are three distinct parts:

  1. the tree, which will be implemented using DynaTree;
  2. the graph, which will be implemented using nvd3 and
  3. the table, which will be implemented using DataTables

Additionally, each of these frontend components will be sourcing their data from web services. I’ll call these web services as follows; including a description of their parameters.

  1. /getTree web service – returns the tree hierarchy
    • no parameters
    • returns the tree in the format DynaTree expects; specifically, as a JSON dictionary
  2. /getGraphData web service – returns one timeseries given the specific arguments
    • parameters are the flags that nail down the specific series in the plot
    • returns the specific timeseries in the format that nvd3 expects; specifically as a JSON dictionary with a data key containing the timeseries as an array; each date and value is a two-element array in the array
  3. /getTableData web service – returns all the timeseries matching the specific arguments
    • parameters are the flags that specify the multiple timeseries we are interested in
    • returns all the matching timeseries in the JSON format DataTable expects

The structure of this tutorial

I’ll group the sections of this tutorial in four parts.

  1. Initial set-up
  2. DynaTree widget and the corresponding web service
  3. nvd3 widget and the corresponding web service
  4. DataTable widget and the corresponding web service
  5. Bringing it all together

Initial Set-Up

This first part is setting up Django. All of this is covered in Django’s excellent documentation but is repeated here with minimal descriptions for completeness.

Install Django

Download it from the Django site, untar and install it. The usual setup.py install command will install it in your python modules directory.

    tar xzvf Django-1.5.1.tar.gz
    cd Django-1.5.1
    sudo python setup.py install

Initialise app

Now create the Django project instance that will hold our app.

    django-admin.py startproject tutorialWebServer
    # where tutorialWebServer is the name of the project and the directory that will be created
    python manage.py syncdb
    python manage.py startapp treePlotter
    # where treePlotter is the name of our app and the directory that will be created
    python manage.py runserver 8080

We won’t be using any Django’s model features.

HTML file

All the Jquery widgets will be contained in one HTML page. You can serve this via a web server or launch it locally.

Javascript libraries

Add the following JS libraries to the HTML head element.

    <script src='/media/jquery-1.10.2.min.js' type="text/javascript"></script>
    <script src='/media/jqueryui/ui/jquery-ui.js' type="text/javascript"></script>
    <script src='/media/jquery/jquery.cookie.js' type="text/javascript"></script>

I store all the supplementary files (JS, CSS, etc) in /media.

DynaTree widget and web service

DynaTree in HTML file

Edit the HTML
    <div id="tree"></div>
Edit the Javascript

Add the CSS and the JS to the HTML head

    <link rel='stylesheet' type='text/css' href='/media/skin/ui.dynatree.css'>
    <script src='/media/jquery.dynatree.js' type="text/javascript"></script>

Next, we add the JS to be executed once the entire document has loaded. This is typically done with a script element at the bottom of the document but jquery let’s us do it at the top with $(document).ready() or the anonymous function version: $( function() {} );.

    $(function() {

      // start tree load
      $("#tree").dynatree({
        checkbox: true,
        selectMode: 2,
        initAjax: {
          url: "/getTree/",
          dataType: "jsonp",
          data:  {}
        },
        onSelect: function(node) {
          //updateChart();
          //updateDataTable();
        },
        onActivate: function (node) {
        },
        persist: true,
        noLink: false,
        fx: { height: "toggle", duration: 200 },
        onPostInit: function (isReloading, isError) {
            if (getStringOfSelectedTreeNodes() != '') {
              //updateChart();
              //updateDataTable();
            }
        }
      });
      // end tree load


    }); // end function()

Note that onSelect (when the user ticks a node’s checkbox) and onPostInit (the function to run after the tree loads) contain updateChart() and updateDataTable(). We want the chart and the data table to be updated when a node is ticked. We’ll define these later.

Web Service

urls.py

This can be done in two places: the project urls file, tutorialWebServer/urls.py or the app urls files, treePlotter/urls.py. We’ll keep it simple and do it in tutorialWebServer/urls.py.

    from django.conf.urls import patterns, include, url
    import treePlotter
    urlpatterns = patterns('',
      url(r'^getTree', treePlotter.views.getTree, name='getTree'),
    )
views.py

This is where the function that accepts the HTTP request and returns the JSON dictionary is defined. In this case, we need to define getTree.

I get the tree information from a DB but in this case, I’ll hard code it. What’s important here is how DynaTree expects to get back a json object.

    def treetest(request):
      myTree = {'children': [], 'title':'Root', 'isFolder':True, 'hideCheckbox': True}
      myTree['children'].append( {'title':'Child 1', 'isFolder':'false', 'children':[] } )
      myTree['children'].append( {'title':'Child 2', 'isFolder':'false', 'children':[] } )
      myTree['children'].append( {'title':'Child 3', 'isFolder':'false', 'children':[] } )
      
      # Convert result list to a JSON string
      res = simplejson.dumps(myTree, encoding="Latin-1")
    
      # Support for the JSONP protocol.
      response_dict = {}
      if request.GET.has_key('callback'):
        response_dict = request.GET['callback'] + "(" + res + ")" 
    
      return HttpResponse(response_dict, mimetype='application/json') 
    
      response_dict = {}
      response_dict.update({'children': tree })
      return HttpResponse(response_dict, mimetype='application/javascript') 

The tree here (myTree) is far more simple than what I pulled from the DB. Here, it’s just a root node with three child nodes. In reality, it could be multiple levels with an aribrary number of descendants.

Changes to views.py will be automatically picked up by the Django process running. Load your HTML page on a browser it will load your DynaTree.

nvd3 Graph widget and web service

nvd3 Graph in HTML file

Edit the HTML

Add the div and the svg elements where the chart will be inserted.

    <div id="chart">
      <svg style="height: 500px;"></svg>
    </div>
Edit the Javascript

Add the JS and CSS files.

    <!-- nvd3 stuff -->
    <script src="/media/nvd3.lib/d3.v2.js"></script>
    <script src="/media/nv.d3.js"></script>
    <script src="/media/nvd3.src/tooltip.js"></script>
    <script src="/media/nvd3.src/utils.js"></script>
    <script src="/media/nvd3.src/models/legend.js"></script>
    <script src="/media/nvd3.src/models/axis.js"></script>
    <script src="/media/nvd3.src/models/scatter.js"></script>
    <script src="/media/nvd3.src/models/line.js"></script>
    <link href="/media/nvd3.src/nv.d3.css" rel="stylesheet" type="text/css">

We want to get the nodes from the tree so we write a function to do so. It returns it as a string, which will be used to pass it on to the web service as part of the GET query string.

    function getStringOfSelectedTreeNodes() {
      var returnString ="";
      var nodes = $("#tree").dynatree("getSelectedNodes");
      for (i in nodes) {
        returnString = nodes[i].data.title + "|" + returnString;
      }
      return returnString.substring(0, returnString.length - 1) ;
    }

Now add a function to initialise/update the chart.

    serialize = function(obj) {
      var str = [];
      for(var p in obj)
         str.push(encodeURIComponent(p) + "=" + encodeURIComponent(obj[p]));
      return str.join("&");
    }
    
    function updateChart() {
      var returnDict = getDictOfAllCheckboxes();
      returnDict['nodes'] = getStringOfSelectedTreeNodes();
      var argQueryString = serialize(returnDict);

      var request = $.ajax({
        url: "/getGraphData?" + argQueryString,
        type: "GET",
        dataType: "json",
        success: function(result) {
          // start nvd3 code
          nv.addGraph(function() {
            var chart = nv.models.lineChart();
          
            chart.xAxis
                .tickFormat( function(d) { return d3.time.format('%x')(new Date(d)); } );
          
            chart.yAxis
                .tickFormat(d3.format(',.3f'));
          
            d3.select('#chart svg')
                .datum(result)
              .transition().duration(500)
                .call(chart);
          
            nv.utils.windowResize(chart.update);
          
            return chart;
          });
        }
      });
  
    }

Note that getDictOfAllCheckboxes() is not defined here but it just takes the value of other form inputs and returns it in a dictionary. On the other hand, serialize() is defined here – it just takes an object and encodes it for use with a GET query (using JS function, encodeURIComponent).

Web Service

urls.py

Now, we add a URL pattern to urls.py so that requests to that url get picked up by the correct function.

    urlpatterns = patterns('',
      url(r'^getTree', treePlotter.views.getTree, name='getTree'),
      url(r'^getGraphData', treePlotter.views.getGraphData, name='getGraphData'),
    )
views.py
  def getGraphData(request):
    if request.method == 'GET':
      formData = dict(request.GET.iterlists())
    elif request.method == 'POST':
      formData = dict(request.POST.iterlists())
  
    # split the string to get the nodes from the dynatree
    nodes = (formData['nodes'])[0].split('|')
  
    # get other values from the query string using the formData variable
    # do other processing
  
    finalResults = []
    for node in nodes:
      timeSeries = getTimeSeries(node) # not defined here
      # getTimeSeries returns an array of tuples: [(datetime, decimal)]
  
      results = []
      for [(x,y) in cursor.fetchall():
        results.append({ 'x': time.mktime(x.timetuple())*1000 , 'y': y})
        # nvd3 expects time to be in milliseconds
  
      resultDict = { 'key': node, 'values': results }
      finalResults.append(resultDict)
  
    # Convert result list to a JSON string
    res = simplejson.dumps(finalResults, encoding="Latin-1")
  
    # Support for the JSONP protocol.
    response_dict = {}
    return HttpResponse(res, mimetype='application/json') 
    if request.GET.has_key('_'):
      response_dict = request.GET['_'] + "(" + res + ")" 
      return HttpResponse(response_dict, mimetype='application/json') 
    else:
      print request.POST.keys()
      response_dict = request.POST['_'] + "(" + res + ")" 
      return HttpResponse(response_dict, mimetype='application/json') 

Note that getTimeSeries() is not defined here. It will be database call that picks up the time series for that node.

DataTable widget and web service

DataTable in HTML file

Edit the HTML
    <table cellpadding="0" cellspacing="0" border="0" class="display" id="dataTable">
    	<thead>
    		<tr>
    			<th>Col 1</th>
    			<th>Col 2</th>
    			<th>Col 3</th>
    			<th>Col 4</th>
    			<th>Col 5</th>
    		</tr>
    	</thead>
    	<tbody>
    		
    	</tbody>
    	<tfoot>
    		<tr>
    			<th>Col 1</th>
    			<th>Col 2</th>
    			<th>Col 3</th>
    			<th>Col 4</th>
    			<th>Col 5</th>
    		</tr>
    	</tfoot>
    </table>
Edit the Javascript

Add the JS and CSS for the data table.

    <!-- table tools stuff -->
    <link rel='stylesheet' type='text/css' href='/media/tabletools/css/TableTools.css'>
    <link rel='stylesheet' type='text/css' href='/media/tabletools/css/TableTools_JUI.css'>
    <script src='/media/tabletools/js/TableTools.js' type="text/javascript"></script>
    <script src='/media/tabletools/js/ZeroClipboard.js' type="text/javascript"></script>

Add the JS to initialise the data table.

    // initialise datatable
    $('#dataTable').dataTable({
      "bProcessing": true,
      "bJQueryUI": true,
      "sPaginationType": "full_numbers",
      "sDom": 'T<"clear">lfrtip',
      "oTableTools": {
        "sSwfPath": "/media/tabletools/swf/copy_csv_xls_pdf.swf",
        "aButtons": ["csv"]
      }
    });
    // end initialise datatable

The above doesn’t actually populate the table. We write the function to do that like so.

    function updateDataTable() {
      var returnDict = getDictOfAllCheckboxes();
      returnDict['nodes'] = getStringOfSelectedTreeNodes();
      if (returnDict['nodes'] == '') {
        return;
      }
      // also need logic to check the from and to fields (parse fields and see if they're valid)
      $('#dataTable').dataTable().fnReloadAjax("/getTableData?"+  serialize(returnDict) );
  
    }

As above, getDictOfAllCheckboxes() is not defined but used here to get other inputs.

With the two functions defined (updateChart() and updateDataTable()), they can now be uncommented in the DynaTree initialisation. Each time new nodes are selected/deselected, the chart and the table will update.

Web Service

urls.py

Now, we add a URL pattern to urls.py so that requests to that url get picked up by the correct function.

    urlpatterns = patterns('',
      url(r'^getTree', treePlotter.views.getTree, name='getTree'),
      url(r'^getGraphData', treePlotter.views.getGraphData, name='getGraphData'),
      url(r'^getTableData', treePlotter.views.getTableData, name='getTableData'),
    )
views.py
    def getTableData(request):
      if request.method == 'GET':
        formData = dict(request.GET.iterlists())
      elif request.method == 'POST':
        formData = dict(request.POST.iterlists())
    
      nodes = (formData['nodes'])[0].split('|')

      # get other values from the query string using the formData variable
      # do other processing

      timeSeries = getTimeSeriesAllOptions(node) # not defined here
      # getTimeSeriesAllOptions returns an array of arrays; each inner array being a row of the table.
      
      finalResults = { 'aaData' : timeSeries }
      # Convert result list to a JSON string
      res = simplejson.dumps(finalResults, encoding="Latin-1")
    
      # Support for the JSONP protocol.
      response_dict = {}
      return HttpResponse(res, mimetype='application/json') 
      if request.GET.has_key('_'):
        response_dict = request.GET['_'] + "(" + res + ")" 
        return HttpResponse(response_dict, mimetype='application/json') 
      else:
        print request.POST.keys()
        response_dict = request.POST['_'] + "(" + res + ")" 
        return HttpResponse(response_dict, mimetype='application/json') 

Note that getTimeSeriesAllOptions() is not defined here, just as with getTimeSeries() above. It will be a database call that picks up the time series for all the selected nodes for all the options selected.

Bringing it all together

All the Jquery requests and the Django services to receive and respond to these are in place. Specifically, I’ve shown the different ways that DynaTree, nvd3 and DataTable make ajax requests and what JSON format they need back.

What remains to be done is the skin the html by cleaning it up and presenting it with some nice CSS and add JS functions for input validation and further interactivity. Hope this was helpful.

8 Comments

  1. Hi Jose, can you provide the datasource of this article to download?
    I’m learning Django and for practice i’m trying to read some data from my database and show some charts.

    I did everything of this tutorial and my project didn’t show any errors, but my webpage keeps blank.

    Reply
  2. Greate Job!I like this content.I am learning django and I want to develop a Django Project that can help me show charts and data tables from Mysql database.I have tried several chart tools,but failed.When seeing this content,I am very exited.You konw it is what I want.I downloaded all JS and try to code according to the process.Up to now ,I still fail.So,thanks a lot if you can send me some example code.

    Reply
  3. I tried your tutorial.
    But i run in some troble … so It will be very great if you provide us with Demo source code..

    Thank You

    Reply
  4. Thanks for your time.
    This is really help. Could you send me the project zip file ? or share a link for downloading it, many thanks!

    Reply
  5. JB, thanks – helped crystallise some early hen scrathings on a small botanical data collection and display system – I had contemplated Django/JQuery before I read this – good confirmation.

    Have you any experience with the Google Maps API? I want to overlay location data, a la http://www.ala.org.au/‎
    Regards RobP

    Reply

Leave a Comment.