At a glance
- See the finished data visualisation here
- Get the source code for the data visualisation over on GitHub
- Get the original open data from Geelong Regional Libraries Corporation
Introduction
Geelong Regional Libraries Corporation (GRLC) came on board GovHack this year, and as well as being a sponsor, opened a number of datasets for the hackathon. Being the lead organiser for GovHack, I didn’t actually get a chance to explore the open data during the competition. However, while I was going for a walk one day – as it always does – I had an idea around how the GRLC datasets could be visualised. I’d previously done some work visualising data using a d3.chord layout, and while this data wasn’t suitable for that type of layout, the concept of using annulars – donut charts – to represent and compare the datasets seemed appealing. There was only one problem – I’d never tackled anything like this before.
Challenge: accepted
Understanding what problem I was trying to solve
Of course the first question here was what problem I was trying to solve (thanks Radia Perlman for teaching me to always solve the right problem – I’ll never forget your LCA2013 keynote). Was this an exploratory data visualisation or an explanatory one? This led to formulating a problem statement:
How do the different Libraries in the Geelong region compare to each other in terms of holdings, membership, visits and other attributes?
This clearly established some parameters for the visualisation: it was going to be exploratory, and comparative. It would need to have a way to identify each Library – likely via colour code, and have appropriate use of shapes and axes to allow for comparison. While I was tempted to use a stacked bar chart, I really wanted to dig deeper into d3.js and extend my skills in this Javascript library – so resolved to visualise the data using circular rings.
Colour selection
The first challenge was to ensure that the colours of the visualisation were both appealling and appropriate. While this seems an unlikely starting place for a visualisation – with most practitioners opting to get the basic shape right first, for this project getting the colours right felt like the best starting point. For inspiration, I turned to the Geelong Regional Library Corporation’s Annual Report, and used the ColorZilla extension to eyedropper the key brand colours used in the report. However, this only provided about 7 colours, and I needed 17 in order to map each of the different libraries. In order to identify ‘in between’ colours, I used this nifty tool from Meyerweb, which is super-handy for calculating gradients. The colours were then used as an array for a d3.scaleOrdinal object, and mapped to each library.
var color = d3.scaleOrdinal() .range([ "#59d134", "#4CCA62", "#40C28F", "#33bbbd", "#36AFC7", "#3b98da", "#427DC9", "#5148a6", "#8647A8", "#BC47A9", "#f146ab", "#F03E85", "#F0355E", "#f0431e", "#F8880F", "#FFCC00", "#ACCF1A" ]) .domain([ "Geelong", "Belmont", "Corio", "Geelong West", "Waurn Ponds", "Ocean Grove", "Newcomb", "Torquay", "Drysdale", "Lara", "Bannockburn", "Queenscliff", "Chilwell", "Highton", "Mobile Libraries", "Barwon Heads", "Western Heights College" ]);
Annular representation of data using d3.pie
The first attempt at representing the data was … a first attempt. While I was able to create an annular representation (donut chart) from the data using d3.pie and d3.arc, the labels of the Libraries themselves weren’t positioned well. The best tutorial I’ve read on this topic by far is from data visualisation superstar, Nadieh Bremer, over on her blog, Visual Cinnamon. I decided to leave labels on the arcs as a challenge for later in the process, and instead focus on the next part of visualisation – multiple annulars in one visualisation.
Multiple annulars in one visualisation
The second challenge was to place multiple annulars – one for each dataset – within the same svg. Normally with d3.js, you create an svg object which is appended to the body element of the html document. So what happens when you place two d3.pie objects on the svg object? You guessed it! Fail! The two annulars were positioned one under the other, rather than over the top of each other. I was stuck on this problem for a while, until I realised that the solution was to place different annulars on different layers within the svg object. This also gave more control over the visualisation. However, SVG doesn’t have layers as part of its definition – objects in SVG are drawn one on top of the other, with the last drawn object ‘on top’ – sometimes called stacking . But by creating groups within the BaseSvg like the below, for shapes to be drawn within, I was able to approximate layering.
var BaseSvg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height)
.append("g")
.attr("transform", "translate(" + (width / 2 - annularXOffset) + "," + (height / 2 - annularYOffset) + ")");
/*
Layers for each annular
*/
var CollectionLayer = BaseSvg.append('g');
var LoansLayer = BaseSvg.append('g');
var MembersLayer = BaseSvg.append('g');
var EventsLayer = BaseSvg.append('g');
var VisitsLayer = BaseSvg.append('g');
var WirelessLayer = BaseSvg.append('g');
var InternetLayer = BaseSvg.append('g');
var InternetLayer = BaseSvg.append('g');
var TitleLayer = BaseSvg.append('g');
var LegendLayer = BaseSvg.append('g');
At this point I found Scott Murray’s SVG Primer very good reading.
I was a step closer!
Adding in parameters for spacing and width of the annulars
Once I’d figured out how to get annulars rendering on top of each other, it was time to experiment with the size and shape of the rings. In order to do this, I tried to define a general approach to the shapes that were being built. That general approach looked a little like this (well, it was a lot more scribble).
By being able to define a general approach, I was able to declare variables for elements such as the annular width and annular spacing, which became incredibly useful later as more annulars were added – the positioning and shape of the arcs for each annular could be calculated mathematically using these variables (see the source code for how this was done).
var annularXOffset = 100; // how much to shift the annulars horizontally from centre var annularYOffset = 0; // how much to shift the annulars vertically from centre var annularSpacing = 26; // space between different annulars var annularWidth = 22; // width of each annular var annularMargin = 70; // margin between annulars and canvas var padAngle = 0.027; // amount that each segment of an annular is padded var cornerRadius = 4; // amount that the sectors are rounded
This allowed me to ‘play around’ with the size and shape of the annulars until I got something that was ‘about right’.
At this stage I also experimented with the padAngle of the annular arcs (also defined as a variable for easy tweaking), and with the stroke weight and colour, which was defined in CSS. Again, I took inspiration from GRLC’s corporate branding.
Placing dataset labels on the arcs
Now that I had the basic shape of the visualisation, the next challenge was to add dataset labels. This was again a major blocking point, and it took me a lot of tinkering to finally realise that the dataset labels would need to be svg text, sitting on paths created from separate arcs than that rendered by the d3.pie function. Without separate paths, the text wrapped around each arc segment in the annular – shown below. So, for each dataset, I created a new arc and path for the dataset label to be rendered on, and then appended a text element to the path. I’d never used this technique in svg before and it was an interesting learning experience.
Having sketched out a general approach again helped here, as with the addition of a few extra variables I was able to easily create new arcs for the dataset text to sit on. A few more variables to control the positioning of the dataset labels, and voila!
Adding a legend
The next challenge was to add a legend to the diagram, mostly because I’d decided that the infographic would be too busy with Library labels on each data point. This again took a bit of working through, because while d3.js has a d3.legend function for constructing legends, it’s only intended for data plotted horizontally or vertically, not 7 data sets plotted on consecutive annulars. This tutorial from Zero Viscosity and this one from Competa helped me understand that a legend is really just a group of related rectangles.
var legend = LegendLayer.selectAll("g")
.data(color.domain())
.enter()
.append('g')
.attr('x', legendPlacementX)
.attr('y', legendPlacementY)
.attr('class', 'legend')
.attr('transform', function(d, i) {
return 'translate(' + (legendPlacementX + legendWidth) + ',' + (legendPlacementY + (i * legendHeight)) + ')';
});
legend.append('rect')
.attr('width', legendWidth)
.attr('height', legendHeight)
.attr('class', 'legendRect')
.style('fill', color)
.style('stroke', legendStrokeColor);
legend.append('text')
.attr('x', legendWidth + legendXSpacing)
.attr('y', legendHeight - legendYSpacing)
.attr('class', 'legendText')
.text(function(d) { return d; });
Again, the positioning took a little work, but eventually I got the legend positioned well.
Responsive design and data visualisation with d3.js
One of the other key challenges with this project was attempting to have a reasonably responsive design. This appears to be incredibly hard to do with d3.js. I experimented with a number of settings to aim for a more responsive layout. Originally, the narrative text was positioned in a sidebar to the right of the image, but at different screen resolutions the CSS float rendered awkwardly, so I decided to use a one column layout instead, and this worked much better at different resolutions.
Next, I experimented with using the Javascript values innerWidth and innerHeight to help set the width and height of the svg element, and also dynamically positioned the legend. This gave a much better, while not perfect, rendering at different resolutions. It’s still a little hinkey, particularly at smaller resolutions, but is still an incremental improvement.
Thinking this through more deeply, although SVG and d3.js in general are vector-based, and therefore lend themselves well to responsive design to begin with, there are a number of elements which don’t scale well at different resolutions – such as text sizes. Unless all these elements were to be made dynamic, and likely conditional on viewport and orientation, then it’s going to be challenging indeed to produce a visualisation that’s fully responsive.
Adding tooltips
While I was reasonably pleased with the progress on the project, I felt that the visualisation needed an interactive element. I considered using some sort of arc tween to show movement between data sets, but given that historical data (say for previous years) wasn’t available, this didn’t seem to be an appropriate choice.
After getting very frustrated with the lack of built in tooltips in d3.js itself, I happened upon the d3.tip library. This was a beautifully written addition to d3.js, and although its original intent was for horizontal and vertical chart elements, it worked passably on annular segments.
Drawbacks in using d3.tip for circular imagery
One downside I found in using this library was the way in which it considers the positioning of the tooltip – this has some unpredictable, and visually unpleasant results when data is being represented in circular format. In particular, the way that d3.tip calculates the ‘centre’ of the object that it is applied to does not translate well to arc and circular shapes. For instance, look at how the d3.tip is applied to arc segments that are large and have only small amounts of curvature – such as the Geelong arc segment for ‘Members’. I’ve had a bit of a think about how to solve this problem, and the solution involves a more optimal approach to calculating the the ‘centre’ point of an arc segment.
This is beyond what I’m capable of with d3.js, but wanted to call this out as a future enhancement and exploration.
Adding percentage values to the tooltip with d3.nest and d3.sum
The next key challenge was to include the percentage figure, as well as Library and data value in the d3.tip. This was significantly more challenging than I had anticipated, and meant learning up on d3.nest and d3.sum functions. These tutorials from Phoebe Bright, and LearnJS were helpful, and Zan Armstrong’s tutorial on d3.format helped me get the precision formatting correct. After much experimentation, it turned out that summing the values of each dataset (in order to calculate percentage) was but a mere three lines of Javascript:
var CollectionItemCount = d3.nest()
.rollup(function (v) { return d3.sum(v, function(d) { return d.Items})})
.entries(CollectionData);
Concluding remarks
Data visualisation is much more challenging than I thought it would be, and the learning curve for d3.js is steep – but it’s worth it. This exercise drew on a range of technical skills, including circular trigonometry, HTML and knowledge of the DOM, CSS and Javascript, and above all the ability to ‘break a problem down’ and look at it from multiple angles (no pun intended).