Visualizing The Census With #D3js

Not long ago D3js came on my radar as a library I needed to look into.  At first I dismissed it as ‘just another jQuery wannabe’.  There is some overlap, but having played with it a little, I’m coming to think of D3 more as jQuery’s nerdy cousin.

I still have some learning to do, but my first foray into it is a quick census data visualization.  Truth be told, I janked one of the examples and tweaked it to allow choosing any census measure, adjust the colors, automatically determine the scale, and fade between color changes.

var width = 960,

    height = 500,

    centered,

    quantize;

var path = d3.geo.path();

var svg = d3.select("body").append("svg")

    .attr("width", width)

    .attr("height", height);

var color = d3.scale.linear()

 .domain([0, 20])

 .range(["#75B7CF", "#00114F"])

 .interpolate(d3.interpolateLab);

queue()

    .defer(d3.json, "/content/us.js")

    .defer(d3.csv, "/content/censusquickfacts.csv")

    .await(ready);

var us, census;

function showData() {

    var rateById = {};

    var maxVal = 0; var minVal = 0;

    var vals = [];

    census.forEach(function (d) {

        var fips = parseInt(d.fips);

        if (fips != 0 && fips.toString().substr(3) != '000') {

            var key = document.getElementById('measure').value;

            var val = parseFloat(d[key]);

            if (val > maxVal) maxVal = val;

            if (val < minVal) minVal = val;

            rateById[fips] = +val;

            vals.push(val);

        }

    });

    quantize = d3.scale.quantile()

     .domain(vals)

    .range(d3.range(20).map(function (i) { return color(i); }));

    svg.select('g').selectAll('path')

         .transition()

        .duration(750)

         .style("fill", function (d) { return quantize(rateById[d.id]); });

}

function prepareMap() {

    svg.append("g")

          .attr("class", "counties")

        .selectAll("path")

          .data(topojson.object(us, us.objects.counties).geometries)

        .enter().append("path")

          .style("fill", function (d) { return '#ccc' })

          .attr("d", path);

    svg.append("path")

         .datum(topojson.mesh(us, us.objects.states, function (a, b) { return a.id !== b.id; }))

         .attr("class", "states")

         .attr("d", path);

}

function ready(error, usData, censusData) {

    census = censusData;

    us = usData;

    prepareMap();

    showData();

}

The result is pretty astounding.  With just a few lines of code, we grab some CSV census data and county map vectors, do the math to map the census value to a color, and draw it on the page. 

The ajax and selection are nice, but technically are things jQuery already does pretty well. Similarly, D3’s databinding model is powerful, but not enough to convince me to pick it over Knockout.  What D3 brings that is fairly unique are a slew of useful math, array, and scaling functions.  For example, this would be much more complex without D3:

//Create a function of 20 colors fading from 75B7CF to 00114F

var color = d3.scale.linear()

        .domain([0, 20])

        .range(["#75B7CF", "#00114F"])

        .interpolate(d3.interpolateLab);

As would this:

//Take the values from the CSV and map them to one of the 20 colors.

quantize = d3.scale.quantize()

        .domain(vals)

        .range(d3.range(20).map(color));

Perusing the examples, there’s still a fair amount to learn about this library, but I think it’s safe to say I’ll be putting it in the toolbox alongside jQuery and Knockout.

SharePoint Collaboration, Visualized

I recently developed some custom web analytics reporting on top of SharePoint 2010 for a customer.  Among the data shown are number of page requests for each department, broken further into department.

It turns out as a metric, this isn’t necessarily the best.  For example, departments with sites that require lots of clicking have more ‘hits’ than departments with more efficient sites.  Still, it is _a_ metric. 

One interesting pattern I’ve seen repeatedly looks like the example above.  The sparkline shows number of requests per day.  For some departments, it routinely comes out looking like this.  Note one employee spiking one day, followed by another employee the next day, and another the next.   I could be over-analyzing, but I believe what this shows is that Employee1 does something, which Employee2 then sees and does something else with, that Employee 3 sees, and so on. 

In other words, by charting the page hits over time by employee, we’ve visualized collaboration within a department.  With a little more smarts, this could be turned into more valuable data- suggesting colleagues, or helping provide insight into business process improvement possibilities.