Australian Internet Governance Forum 2016

The Australian Internet Governance Forum – #auigf – was held at the Park Hyatt, Melbourne, October 11th-12th, 2016. This was the first time I’d had an opportunity to attend the #auigf, and I wasn’t sure what to expect. Internet users are a diverse cohort – and auDA – regulator for the .au namespace, and the body which auspices #auigf classifies members into supply class – those providing internet services – and demand class – those consuming services.

My first impression was one of surprise. The #auigf theme for the forum was ‘a focus on a competitive digital future for Australia’  – and given the significant influence that digital technology, policy and communities will play in an era of digital disruption, I couldn’t help but wonder why more key players weren’t passionate about driving the future of the internet in Australia.

 

Stuart Benjamin, Chairman of auDA

The regulator has been the subject of criticism in recent years, particularly around its engagement and consultation practices, and long-serving CEO Chris Disspain left the organisation in March, being replaced by former Liberal state parliamentarian, Cameron Boardman. This #auigf was therefore a symbolic opportunity for Boardman to signal to stakeholders the organisation’s new focus.  auDA chairman Stuart Benjamin in his opening address tackled this head on, outlining a renewed focus on stakeholder engagement, particularly in the area of building international partnerships, and relatedly, cybersecurity. He framed this strategic shift as auDA ‘growing up’ – moving from adolescence into maturity. In particular he flagged a shift from reactive approaches to domain administration, to more proactive approaches, underpinned by stronger relationships, renewed processes and systems and more innovative thinking. Linking board performance as critical to the success of the organisation, he introduced new Board Directors, Michaella Richards and Dr Leonie Walsh. Continuing the theme of advancing women in the organisation, Benjamin congratulated lawyer Rachael Falk on her appointment as Director of Technology, Security and Strategy, a newly created role tasked with catalysing auDA’s new directions. Acknowleding that auDA needs to win back the trust of the community it serves, Benjamin emphasised higher expectations of auDA – both externally from stakeholders and driven internally by the organisation itself, announcing he will be “seeking a lot more”.

Prof Paul Cornish, former Professor of International Security at Chatham House and independent consultant and author

Prof Cornish outlined how auDA is heading towards a more international posture and developing a number of partnerships. His main argument was that the future of the internet – and the digital economy – needs to be secured. Cybersecurity needs to evolve as the internet does, using a capability maturity model.

Cybersecurity Plenary – Chaired by Rachael Falk, with Alistair MacGibbon, Laura Bell, Prof Chris Leckie, Simon Raik-Allen, Craig McDonald

Rachael Falk opened by drawing attention to the National Cyber Security Strategy, urging attendees to become familiar with it. The discussion quickly turned to why there wasn’t more focus on cyber security, and Prof Cornish had a very incisive response – “interest follows money”. Money is starting to flow to cyber security, and interest will follow. Prof Leckie outlined challenges getting cyber security research from the lab into mainstream commercialisation. Researchers are challenged by the rate of change – for example, hypothetical attacks are quickly becoming reality. Academia is also confronted by getting business and industry to recognise the threat that cyber security presents. The other challenge is getting boards to recognise that cyber security is many different problems – which need many solutions. This is overwhelming for small businesses who “just want it to work”.

One of the best insights on the plenary came from Laura Bell – @lady_nerd on Twitter – who recounted the example of big corporations acquiring smaller firms – who may have a very different security posture, thus putting the larger corporation at risk.

The plenary used the term “happy clickers” to denote people who click on phishing emails without critically assessing their validity. This was the first time I’d heard that term, but it captures the psychological state accurately. Interesting, there was discussion around how people who are disengaged in their roles being more likely to be ‘happy clickers’ – because the phishing email represents a welcome distraction – another reason to ensure positive employee engagement.

Another very interesting discussion thread in this plenary was the paradox of cyberware – people personal information freely with services like Google and Facebook, but resent government intrusion as seen recently with the census. This may come down to the compulsion element – it’s about giving information freely versus being compelled to disclose. There’s an element here for government design of online services – another job for the DTO! – around information design. Imagine a census that was voluntary rather than mandatory, but got people to participate because of the social good involved. I think it would be a much more positive process.

This led into a discussion around corporate use of data – and whether consumers understand the value of their own data – essentially we’re trading our data for ‘free products’. For many online services we have to consent to data disclosure to get access to the service, but in the background there’s data matching going on – there’s a ‘creep factor’. The link was drawn from ‘creep factor’ behaviour to band value – trust and transparency are linked to the public’s view of the brand.

Key takeaway: The pub test for data use – “is it creepy?” If so, don’t do it.

This plenary also covered the practice of ‘hacking back‘ – where individuals or businesses use information security counter-measures to retaliate. The consensus in the room is that this is a poor response, largely because identifying the aggressor is so difficult. The group also highlighted that Australia has an offensive cyber capability – again linking cyber security to an international, nation-state based context. The lack of a standard response protocol for dealing with hacking incidents was also covered – many businesses are afraid of disclosing and are reluctant to do so, but having a standard response protocol would allow businesses to respond in a mature way.

In summary, cyber security is hard – there’s lots of layers and issues to consider, there’s a lack of general awareness in business and industry, the field is rapidly changing and no defined response protocols for business to use.

Women in STEM Plenary – Dr Rowan Brookes, Renee Noble, Dr Catherine Lang, Dr Leonie Walsh, Luan Heimlich

Dr Brookes introduced the plenary with an apology for not being able to include more women of colour and from the LBGQTI spectrum, particularly on Ada Lovelace Day. The key themes of needing to address systemic issues and create a pipeline for women in STEM were prevalent throughout the conversation.

What struck me first up with this plenary was the range of initiatives, groups and organisations that are working to further women in STEM, and I wondered whether this fragmentation is actually a disservice – so many voices have less volume.

Key takeaway: Are there too many women in STEM groups that are too fragmented? Do we need an Australia ecosystem map of women / females in STEM / ICT

Luan Heimlich opened the plenary by asking the audience who young girls look up to; met with responses of pop stars, sports celebrities and models. Not a science or technology role model in sight! She followed up by questioning whether these role models are going to solve the problems of tomorrow – digital disruption, climate change and public health, and let the audience ponder on the gap.

Dr Leonie Walsh covered efforts to help encourage early to mid career researchers to further their careers, noting that it’s difficult for women to step out of their careers to have a family – as this often puts them several years behind. She also noted that employers are looking for candidates with more well rounded skills, and her program provides exposure to work environments. Dr Catherine Lang highlighted the influence of pre-service teachers in promoting STEM. Another key thread in this discussion was that professions are socially constructed, and that this can be changed – but it’s an uphill battle because ICT careers are not even on the radar as a career choice for young women.

While programs are having localised success, there are still major gaps at a systemic level, and better consistency and co-ordination is required at a national level.

Behavioural insights panel – Kirstan Corban, Dr Alex Gyani, Christian Stenta, Helen Sharpley

This panel was a series of vignettes centred around how behavioural insights had led to social change. The standout piece was by Alex Gyani, who ran the audience through examples of where minor changes had a major impact – using a framework of

  • Easy – interventions should be easy for people, but this is hard to do
  • Attractive – the intervention has to be attractive for people
  • Timely – try something, see if it works – don’t be caught in analysis paralysis
  • Social – social norms are a powerful influencer for change

A key concept from Gyani’s talk was the concept of cognitive budget – we have so many choices to make every day we need to think critically about choice architecture.

The other three speakers, from health and government, highlighted case studies that showcased design thinking, co-design, and approaches to difficult problems.

Key takeaway – minor changes can make a big impact

Internet of Things Plenary – Pablo Hinojosa, Matthew Pryor, Phil Goebel, Lorraine Tighy, Dr Kate Auty

Hinojosa opened proceedings by outlining how the internet has reached 3.5 billion users – half of this volume in Asia – and there are double the number of internet connected devices than people. We’re on the cusp of a revolution.

Matthew Pryor outlined the use of IOT in agriculture and agribusiness, and emphasised how IoT helps with decision making. He highlighted how it’s hard to scale infrastructure in regional and rural areas – and questioned whether we should be investing in networks that connect people or devices or both? He gave the example that as soon as farmers leave the farmhouse, they have no internet – they need to go back to the farmhouse to make better decisions, and this reduces their ability to deliver economic benefit. We need to consider the principle of universal access as we build out infrastructure.

Phil Goebel used the Disneyland Magic Band example to highlight how IoT has taken a purely physical experience and used connectivity to enhance that – leading to “augmented experience”. For example, the band allows Disney to know where the longest queues are, how the park is being used, what facilities are important for which demographics – very granular marketing data. He outlined that there are multiple users of the data – different actors in the ecosystem – administration, marketers and the users themselves – using the data gathered by wearables for different purposes. He flagged the issue that there are no guidelines around how the data is being used – for instance is it being sold on – we need to consider transparency.

Lorraine Tighe is the Smart City and Innovation Manager at City of Melbourne, and outlined how vendors she mets present the IoT as a silver bullet. She outlined the use cases for IoT in smart cities, including parking sensors – to reduce traffic that is searching for a car park – leading to traffic efficiencies. She positioned local government at the coalface of the community, and bringing the community along on the journey – using the City Lab as a vehicle to test and prototype solutions. As part of this, the City of Melbourne made the decision to go open by default with their data, encouraging smart people to co-create with the City.

 

Dr Kate Auty spoke on projects like RedMap and Atlas of Living Australia providing citizen scientists with tools to protect biodiversity. She related how ‘super science’ projects like AURIN and NECTAR are important for understanding how cities work.

Scott Seely had the quote of the panel though;

 

Conclusions

In summary, the #auigf reflected many of the contemporary themes of digital society. Digital disruption and digital society are changing at a rapid pace, and we have a dearth of tools, approaches, standards and response protocols to handle them. We need to start by clearly defining the problems we’re trying to solve, and approach solving them with new types of problem solving approaches, such as design thinking, co-creation and open data. Many of the problems we’re trying to solve require national and international co-operation to build ecosystems, standards and agreed approaches – and the #auigf is a good starting point.

Save

State of my toolchain 2016

In July, I transitioned from a 16-year career in digital and IT with a regional university to setting up my own digital consultancy. This meant that I no longer had a Managed Operating Environment (MoE) to rely on, and instead had to build my own toolchain. Both to document this toolchain, and to provide a snapshot to compare to in the future, this post articulates the equipment, software and utilities I use, from hardware up the stack.

Hardware

I have three main devices;

  • Asus N76 17.3″ laptop – not really a portable device, but a beast of a work machine. I’ve had this since January 2013, and it hasn’t let me down yet. It has 16GB of RAM, 4 dual core Intel(R) Core(TM) i7-3630QM CPU @ 2.40GHz CPUs, so 8 cores in total, and it basically needs its own power station to run. This machine is a joy to own. It speeds through GIMP and video processing operations, and has plenty of grunt to do some of data visualisation (Processing) work that I do. The NVIDIA graphics are beautiful. The only upgrade in this baby’s near term future is to swap out the spinning rust HDD (x2) with some solid state goodness.
  • Asus Trio Transformer TX201LA – a portal device, useful for taking on trains and to meetings. I’ve had this for around 18 months now, and while it’s a solid little portable device, it does have some downsides. This is a dual operating system device – the screen, which is a touchscreen, and detaches, runs stock Android (which hasn’t had an update since 4.2.2 – disappointing), while I’ve got the base configured via Grub to dual boot Win10 and Ubuntu 16.04 LTS. Switching between the mobile OS and desktop OS is generally seamless, but I’ve had some glitches switching between Ubuntu and Android – in ASUS’ defence, they did tell me that Linux wasn’t supported on this device, and of course you all knew what my response that was, didn’t you? Challenge: accepted. The hardware on this device is a little less grunty than I’d like – 4GB RAM and Intel® Core™ i7-4500U processor. It just isn’t enough RAM, and I have to pretty much limit myself to running 3-4 apps at a time, and less than 10 Firefox tabs. But, that said, I *do* like the convenience of having the Android device as well – and the screen is a joy to work with. One little niggle is that VGA / HDMI out are via mini display port – and only a VGA adaptor was provided in the box. I’ll have to get a mini display port to HDMI adapter at some stage, as the world embraces digital video out. For the meantime, I’ll have to party like it’s 1999 with VGA.
  • LG Nexus 5X – my mobile phone. Purchased in January 2016, it’s running stock Android Marshmallow, and I’ve been super happy with how fast Android OTA updates ship to this device. For non-RAM-intensive operations it’s pretty snappy, and the quality of the camera is fantastic. The battery life is pretty good compared to my old Nexus 4, and I can usually go a full day on a charge, if I’m not Ingressing. This device has some pretty major downsides though. The USB-C charging cable is frustrating, given everything else I own charges on micro USB, so I’ve had to shell out for new cables. The RAM on this device just isn’t enough for its processor, and I’m constantly experiencing lag on operations, making for a frustrating user experience. The camera is buggy as hell, and there’s more than once I’ve taken a great shot, only to find it hasn’t been saved. I’ll be looking for a different model next time, but I can’t justify replacing this at the moment – it’s only around 8 months old.

My hardware overview wouldn’t be complete without these other useful peripherals:

Wearables

The two key wearables I have are the Pebble Time and Fitbit. As Pebble Time’s GPS and fitness tracking capabilities increase, I’m expecting to be able to decom my Fitbit. I can’t imagine living without the Pebble now – it’s a great wearable device. The battery life is pretty good – 3-4 days, and the charging connector is robust – unlike my poor experiences with the Fitbit – both with the device battery itself degrading over time, and having been through 5-6 chargers in 3 years. I’ve Kickstarted the Pebble Core, and can’t wait to see where this product line goes next.

Software

At the operating system level, both my laptops dual boot both Windows 10 and Ubuntu LTS 16.04, with my preference to be to use Ubuntu if possible. This generally works well, but there are some document types that I can’t access readily on Ubuntu – such as Microsoft Project. Luckily, most of the work I do these days is web-based. I still need Windows for gaming, because not all the titles I play are delivered via Steam – with the key one being The Secret World. Total addict 🙂

Office productivity

  • LibreOffice – my office suite of choice is LibreOffice. OpenOffice is pretty much dead, and the key driver of that is being umbrella’d by Oracle. Open source communities don’t want to be owned by large corporates who purchase things, like, oh I don’t know, MySQL, to simply gain market share rather than ascribing to the open source ethos.
  • Firefox – my browser of choice. Yes, I know it’s slower. Yes, I know it’s a memory hog. But it’s Firefox for me. I really like the Sync feature, meaning that the plugins and addons that I have on one installation automatically download on another – very useful when you’re running essentially four machines. My favourite and most used extensions would have to be LeetKey, Awesome Screenshot, Zotero, ColorZilla and of course Web Developer tools.
  • Thunderbird – I run Thunderbird with a bunch of extensions like Enigmail, Lightning (with a Google Calendar integration for scheduling) and Send Later – so that if I write a bunch of emails at 2am in the morning, they actually send at a more humane hour.
  • Zotero – I used Zotero, and its LibreOffice plugin for referencing. It’s beautiful. And open source.
  • Slack – Slack is the new killer app. I use it everywhere, on all the things. The integrations it has are so incredibly useful. In particular, I use an integration called Tomato Bot for Pomodoro-style productivity.
  • Xero – Yes, I have a paid account to Xero for accounting and bookkeeping. It’s lovely and simple.
  • Trello – For all the project management goodness. I got some free months of Trello Gold, and I’ve let it lapse, but will probably buy it again. It’s $USD 5 per month and has great integration with Slack. Again, if there were an open source alternative I’d give that a go, but, well, there just isn’t.
  • GitHub and Git – If your office is about digital and technology, then GitHub is an office productivity tool! I use Git from the command line, because it’s just easier than running another application on top of everything else.

Social media and radio

  • Hootsuite – Yes, I have a paid account to Hootsuite. There just isn’t a comparable open source alternative on the market yet. It has some limitations – such as lack of strong integration with newer social media platforms such as Instagram and SnapChat, but you can’t go past it for managing multiple Facebook pages or Twitter accounts at once.
  • Pandora – I stream with Pandora, but I really, really, really miss Rdio.

Quantified self

Over the years, I’ve found a lot of value in running a few quantified self applications to get a better idea of how I’m spending my time – after all, making a problem visible is the first step toward a solution.

  • RescueTime – the visualisations are beautiful, and it runs on every device I have, including Linux. It provides great insights, and makes really clear when I’ve been slacking off and not doing enough productive work. One of the features that I appreciate most is to be able to set your own categorisations. For examples, Ingress in my RescueTime is categorised as neutral – yes it’s a game, but I only play it when I’m walking – so that’s something I’m aiming to do more of.
  • BeeMindr – this nifty little app puts a sting in the tail of goals – and charges you money if you don’t stick with strong habits. I’ve found it’s started to help change my behaviour and build some better habits, such as more sleep and more steps. It has a huge range of integrations with other tools such as RescueTime and Fitbit.

Coding, data visualisation and other nerdery

  • Atom Editor – this is my editor of choice, again because it works on both Windows and Linux. The only downside is that plugins – I run many – have to be individually installed. If Atom had something like Firefox Sync, it would be a killer product. It’s so much lighter than Eclipse and other Java-based editors I’ve used in the past.
  • D3.js – this is my go to Javascript visualisation library. V4 has some pitfalls – namely syntax changes since v3, but it’s still a beautiful visualisation library.
  • Processing – I’ve used Processing a little bit, but I’m frustrated that it’s Java-based. Processing.js is a library that attempts to replicate the Java-based Processing, but the functionality is not yet fully equivalent – particularly for file manipulation operations. The concept behind Processing – data visualisation for designers, not programmers – is sound, but I feel that they’ve made an architectural faux pas by not going Javascript right from the start. I haven’t really gotten in to R or Python yet, but I can see that on the horizon.

Graphics, typography and design

  • Scribus – in the past year I’ve had to do quite a few posters, thank you certificates and so on – and Scribus has been my go to tool. The user interface is a little awkward in places, but it provides around 60% of the functionality of desktop publishing tools like QuarkXPress and InDesign – for free.
  • InkScape and GIMP – my go to tools for vector and raster work respectively. Although, I have started to experiment a little with Krita lately. One of the things I’ve found a little frustrating with both InkScape and GIMP is the limited range of palettes that they ship with, so I started writing some of my own.
  • Typecatcher – for loading Google fonts on to Linux.

Next steps

Thin client computing seems to be taking off in a big way – virtualised desktops are all the rage at the moment, but I don’t think they would work for me, primarily because I tend to work in low bandwidth situations. My home internet is 4-5Mbps, and my 4G dongle gets about the same, but is pre-paid, so data is expensive. For now, I’ll have to manage my own desktop environment!

What do you think? Are these choices reasonable? Are there components in the stack that should be replaced? Appreciate your feedback 😀

Valentine’s Day: A data visualization to learn Chords in d3.js

One of my learning goals this year was to really understand d3.js, and become more proficient in creating interactive data visualizations. In turn, this lead me to attempting to learn and analyse Chord diagrams. Chord diagrams visualize relationships either unilaterally or bilaterally. For example, they have been used to show capital flows inbound and outbound in financial visualization.

Learning how Chord diagrams are constructed in d3.js

Firstly, I wanted a solid primer on how Chord diagrams are constructed in d3.js. Steven Hall’s excellent blog post for Delimited provided the best overview, and clearly articulated elements of a Chord diagram such as the Matrix, the map of flows, arcs and paths. Using this approach, I decided to construct my own scenario and see if I could visualize it. The scenario had to have:

  • Bi-directional data flows which may be asymmetric (x has a relationship with y, but y may have a different relationship with x)
  • A small enough dataset that I could manually construct it (without having to do a lot of CSV or json processing – this exercise was about learning Chords, not about grokking data loading in d3.js)
  • A dataset that could be easily understood by a layperson

I settled on the concept of Valentine’s Day crushes, because they satisfy the above criteria. Next, I constructed a number of statements that were to be visualized. They assumed that one person expressed a crush on one other person, and that this may or may not be mutual. After doing an initial list, I had to call myself out – I’d assumed hetero-normative relationships (male attracted to female and vice-versa), but of course that’s simply not diverse or inclusive thinking.

  • Bob (male, hetero) likes Emily (female, hetero)
  • Giovanni (male, hetero) likes Emily (female, hetero)
  • Kevin (male, hetero) likes Poh (female, hetero)
  • Art (male, hetero) likes Viva (female, hetero)
  • Pyotr (male, hetero) likes Viva (female, hetero)
  • Rohan (male, hetero) likes Rachel (female, hetero)
  • Sasha (male, same-sex attracted) likes Pyotr
  • Emily likes Bob
  • Rachel likes Rohan
  • Poh (female, hetero) likes Sasha
  • Viva likes Pyotr
  • Lee (female, same-sex attracted) likes Poh

The next step was to convert these statements into a matrix.

The matrix

Matrices are usually built from spreadsheet or other tabular datasets. Therefore, it was helpful for me to represent the above relationships in a table.

Valentine’s Day preferences
Name Bob Giovanni Steve Kevin Art Pyotr Rohan Sasha Emily Rachel Poh Viva Lee
Bob No No No No No No No Yes No No No No
Giovanni No No No No No No No Yes No No No No
Steve No No No No No No No No Yes No No No
Kevin No No No No No No No No No Yes No No
Art No No No No No No No No No No Yes No
Pyotr No No No No No No No No No No Yes No
Rohan No No No No No No No No Yes No No No
Sasha No No No No No Yes No No No No No No
Emily Yes No No No No No No No No No No No
Rachel No No No No No No Yes No No No No No
Poh No No No No No No No Yes No No No No
Viva No No No No No Yes No No No No No No
Lee No No No No No No No No No No Yes No

The matrix is an inherent part of Chord diagrams. Chord diagrams are based on a symmetric matrix  – that is, there are as many rows in the matrix as there are columns. One of the first mistakes I made in this exercise was not to have the columns and rows in the same order – I ordered the names of people in the columns differently to the rows. When this was implemented as a Chord layout in d3.js, it was an incorrect representation.

Trap: Ensure that in your data matrix, that the data in rows and columns is in the same order. If you don’t have your data in the same order, the Chord diagram will assume that your row data is in the same order as your column data.

From this table, I was then able to declare a matrix variable:

var matrix = [
  [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0], // Bob
  [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0], // Giovanni
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0], // Steve
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0], // Kevin
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0], // Art
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0], // Pyotr
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0], // Rohan
  [0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0], // Rachel
  [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], // Sasha
  [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0], // Emily
  [0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], // Lee
  [0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0], // Viva
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]  // Poh
]

Chords and ribbons

The next step in this process is to calculate the flows in the matrix in both directions. In this example, it means calculating the relationship between each of the people, and that relationship may not be equal or symmetric. For instance, Rachel likes Rohan, and Rohan likes Rachel this is a symmetric relationship. However, Kevin likes Poh, but Poh likes Sasha. The relationship is asymmetric. This is reasonably simple to do (and Steven Hall provides an excellent pseudocode example in his blog post). d3.js being the knock-your-socks-off piece of awesome that is provides an inbuilt function for this:

chords(matrix)

This function calculates several values needed to calculate the chords in a Chord diagram, including the starting and ending angle of the chords, as well as the values (inbound and outbound) of the chords.  See the d3.js manual entry for more information.

Arcs

A Chord diagram often has labels around the outside of the diagram, and these are produced in d3.js by passing the subgroups (from the matrix() function) to the

d3.arc()

generator. See the d3.js manual entry for more information.

Problems encountered

One of the key issues I encountered in getting this far with the data visualization was the syntax changes between version 3 of d3.js and version 4. There are a number of changes to method names, and the way that they are called under version 4. Many of the examples I used as a jumping off point were done prior to mid-2016 (when version 4 was released), using the older syntax. My example used version 4, and this resulted in a number of syntax errors if I was ‘copying and pasting’ code.

The way I handled this was to have a good read through the ChangeLog for version 4, noting in particular the changes to Chord and Ribbon methods.

Another issue that occupied some brain cycles was the different way that the matrices were calculated. Many earlier examples and tutorials included a ‘mapper’ function where series data was mapped to a matrix. In my example, the matrix() function did mapping as well.

As someone who’s familiar with OO-style programming, I’m still getting used to the way that d3.js modifies the document object model, first by selecting, entering and then modifying one or more DOM elements. This is something I’m just going to have to “get used to” as I use d3.js more.

Trap: Ensure you’re using the correct method calls for the version of d3.js you’re using

Visual design aspects

Data is only one part of the data visualization lifecycle. In order to be useful, it has to be visualized in a meaningful way.

Colours

The first major choice was how to represent the different people in the visualisation. It made sense to use different colours for men and women, and following (traditional, socialised, boring, gender-normative – I get it) what people are expecting, I chose blue for men and pink for women (from the Pantone Colours of Spring 2016 palette. Because Pantone). This provided a pleasant looking graphic, but data visualization is about telling a story.

I decided to add in two more colours to represent the same-sex attracted people in the data series (Sasha and Poh). This added visual interest, and made it easier to interpret some of the interesting details about the whole that were not apparent from the initial statements.

Chord diagram with solid colour in ribbons
Chord diagram with solid colour in ribbons

Using gradients in the ribbons

As you can see from the above, the solid colours (well, solid colours with a transparency applied) don’t really narrate the story of this visualisation very well. The pink colour dominates, and nuances (such as the unrequited love triangle between Poh, Sasha and Lee) in the data are less obvious.

Visually, I wanted the ribbons in the diagram to have gradients. At first glance this looked incredibly complex, and I was about to give up, when I found an excellent article by Nadieh Bremer, one of the gurus of d3.js, on this exact topic. Nadieh’s article provides the mathematical basis for visually appealing gradients in ribbons, including how the direction of the gradient is calculated, based on the position and direction of the ribbon. It’s very well articulated, and you don’t even need basic trigonometry skills to get it – it’s visually explained.

In a nutshell, Nadieh’s code calculates the gradient start and stop points for each ribbon, and the angular direction in which the gradient should be applied.

Using Nadieh’s code, I then applied gradients to the ribbons, for a much more informative and meaningful visualization.

Chord diagram with gradients
Chord diagram with gradients

Arc labels

The next tricky piece visually was adding the name of each person to the arc. For this, I relied on code from this Chord example from AndrewRP.  This included applying CSS styles to the svg text, which I hadn’t done before. Because you’re styling text within an svg element, you need to prefix the selector with the svg element:

svg .titles {
  font-size: 180%;
  font-family: "Abel", sans-serif;
  font-weight: bold
}

This wasn’t something I’d done before, so it was a great learning experience.

I’m still not entirely happy with the labels in the arcs – I would much prefer them to be larger, bolder and centred within the arc segments themselves. A good extension activity for another day.

Telling the story

Of course a key point with a data visualization is for the data to tell a story.

Visualizing the Valentine’s day sentiments that we started off with allows us to derive a lot more meaning from the data overall:

  • We can see a tragic unrequited love triangle. Lee is attracted to Poh, who is attracted to Sasha, who is attracted to Pyotr, but Pyotr and Viva are mutually attracted
  • No-one is attracted to Lee, Giovanni, Kevin or Art
  • We can see that Rohan and Rachel, Bob and Emily and Viva and Pyotr have mutual attraction.
  • Rachel and Viva are both liked by two men and thus are the most popular women

Next steps

There were some additional elements I would have liked to have added to this visualization, but ran out of time to implement – but I’m noting them here as extension activities if I come back to this in the future.

  • As mentioned, I’d like to clean up the arc titles and visually enhance them
  • Being able to click on a ribbon and learn more information about a particular relationship would be useful and visually pleasing. Nadieh Bremer again has a worked example of how to achieve this, however the code is quite complex and requires a large code base for the ToolTip functionality.
  • It would also be ideal to isolate a particular ribbon – especially given that so many ribbons are overlapping in the centre, making it harder to follow visually who is attracted to who. This would use some form of opacity change to ‘fade’ the ribbons not selected.
  • A number of the variables are statically declared in the code, as arrays. While this is totally fine as a learning example, for reusability I’d much prefer to put them into CSV or JSON files, then use Javascript to read them in.

Get the code

See the final visualization at:

http://blog.kathyreid.id.au/valentines/

And get the source code on GitHub at:

https://github.com/KathyReid/valentines-dataviz