In praise of insets

Sky News reports on the Scottish politician Tavish Scott (real name) as he proposes a ban on maps that use an inset to show Shetland. Utter nonsense.

His reasoning is that the 'islands should be in the right place on the map' and 'to ensure that in future that government publications and documents do reflect the reality of Scotland in terms of its geography, and not something that fits neatly on an A4 sheet of paper.'

Well let's just break down (sorry, I mean utterly destroy) his preposterous statements. Shetland (The Shetland Isles) is a part of Scotland. It's also part of Great Britain...and the British Isles... and the United Kingdom (and at the time of writing at least, Europe). The following usefully clarifies the terminology:

So in that sense it has a rightful place on any map of which any of those jurisdictions is the focus.

If you're not aware of where Shetland is in the world then this should help (map to scale, UK National Grid):

In simple geographical terms Shetland is approximately 130 miles from mainland Scotland though it's also on the same line of latitude as Bergen on the west coast of Norway at a distance of 200 miles. That's where it sits. At a nice northerly 60° 9' 11" N and 1° 8' 58" W. We can argue all you like whether it should be part of Scotland or, perhaps, a bit of waste left over from Slartibartfast's design of the Norwegian coastline but for now, it gets put on maps of Scotland and any other that includes Scotland. And that makes it a pain in the arse for cartographers.

By the way, did anyone spot I deleted France from the map above? Guess that'd annoy the French too but whatever, I doubt the scots or Scott cares much about that little cartographic editorial decision.

Anyway, as the scot, Scott, says, many maps are made on A4 (or any A series piece of paper where the length and width are in the same proportion). So here's the above map proportioned as A4:

The area outside the yellow line is superfluous to the map as the Republic of Ireland is irrelevant on many maps that show thematic data for Great Britain and Northern Ireland (or UK etc). So as a rough estimate the page only uses approximately 60% of its overall space to show the mapped content.

Put another way, we need a fifth of the entire page (above the green line) just to get Shetland on. That's Shetland, population 22,000. Or, in geographical terms, an island that is 566 sq miles which is 8x larger than the City of Glasgow but which has 27x fewer people. So you're making a decision to allow geography to not only influence the design but take an inordinately exaggerated status simply by virtue of position.

Now, admittedly there's lots of lovely space for titles and legends and all the other crap we put on a map but, nevertheless, it's wasteful. But that's what you'd have to do if you want Shetland on the map, on that proportion of paper, in its correct geographical position. Alternatively, as Scott bemoans, cartographers will often use an inset and you'd end up with a map like this:

This fits the map to the paper (not the paper to the map). And there's far less wasteful space. Far less prominence to unpopulated swathes of water. And yet Shetland gets its own little special place on the map, with the addition of a neat border that clearly demarcates it. Often, Shetland is even exaggerated in scale to make the inset worthwhile. I bet you didn't notice but in the example above Shetland is about 25% bigger than it really is. So, if Scott wants Shetland back in its proper geographical location then he can have it reduced back to its real size too.

Insets are a neat solution and one that has served print mapping well for centuries. It's also a solution that people understand. You could add a small arrow and distance marker to point to where the inset exists in reality. Many even use a marked graticule to show clearly the lines of latitude and longitude that apply to the inset to make it clear that it differs from that of the main map. Even the Sky News article showed an historical example that clearly uses this technique (and note how exaggerated in size Shetland is on this too):

There's other considerations...

Returning to the City of Glasgow. In fact, any relatively populous place. They suffer horrendously on any maps of thematic data because large areas, perhaps relatively uninhabited or sparsely populated take visual prominence. Scotland is a great example. Its total population is around 5.3 million yet 1 million of them live in Glasgow and Edinburgh. Their population densities are far more than the far greater share of Scottish land, including Shetland. So why give such visual prominence to sparsely populated areas?

Insets are not just used to move geographically awkward places. They are commonly used to create larger scale versions of the map for smaller, yet more densely populated places. Often they are positioned over sparsely populated land to use space wisely. I'm guessing Scott would have an objection to an inset that, to his mind, would exaggerate the geographical importance of Glasgow compared to Shetland. Yet...in population terms it's a place of massively greater importance so one could argue it deserves greater relative visual prominence on the map. Many maps are about people, not geography.

In addition to moving Shetland south to make better use of map space, you could very well argue that you should use a cartogram to give far better relative visual prominence to the places where more people live and work. Now that will utterly delight Scott as it completely distorts geography. Not only could you have Shetland moved, but squished to an almost unrecognisable shape. Here you go Tavish...enjoy this beauty of population totals morphing geography (courtesy: worldmapper.org):

Whatever your view of insets (and Scott's is incorrect), there's so many valuable uses for them that counter the problems of geography making it awkward to make maps. Generations of cartographers have come up with novel solutions to many, if not all, of these dilemmas about what to show, where and how. And if the map has an overarching location map showing everything in its correct position then there can be absolutely no confusion whatsoever.

I would guess Scott would equally be horrified if Shetland was seen poking outside the graticule or neatline on an atlas page too - another common way in which maps break the rules of either geography or design in a creative way to simplify and communicate. He'd be delighted by this classic Times Atlas of the World page showing Shetland in its correct position as part of The British Isles but horrors of horrors...Rockall (also a part of Scotland so has equal rights on a map based on Scott's nonsense...but ignore the more northerly Faroe Islands, not part of Scotland) slips off the left edge:

And what of digital maps? Scott seems to be stuck in the age of print cartography because insets are rarely, if ever, a requirement in digital cartography. Everywhere exists where it is. The map is slippy and you can pan and zoom to your heart's content. Want to see a densely populated area? Zoom right on in. In fact, whisper it quietly in case Scott is listening but...if he uses the standard Web Mercator web map he not only gets Shetland in its rightful position AND it's also exaggerated in size compared to the southerly latitudes of mainland England by virtue of the projection. Now isn't that the map he really wants?

Update  4th October 2018

As BBC News reports Mr Scott has got his way and a bill has been passed that includes a 'mapping requirement' that Shetland not be placed in a box. Fortunately there's a clause allowing cartographers to use an inset if they have a case to do so...so that's OK then because there's always a case to do so UNLESS you're making a navigational chart where distance, direction and bearings are obviously the paramount need for the map's properties and design to support. But all this really means is time wasted and taxes used up as map-makers submit their case to use an inset for Shetland.

So, where do we go from here? If Alaska and Hawaii find out about this nonsense then it's going to make future US maps rather interesting. And please don't get the Falkland Islands worked up and starting to assert their right to be included on maps of the UK. As a British Overseas Territory, they, along with the rest, may very well start insisting they should be on the map, in the correct location. So, here's your updated map Mr Scott. Shetland is in the right place but you can't see it any more because we've had to include all the British Overseas Terretories as well. Sorry about that but they all have an equal right. I do hope you appreciate it's a useless map for showing important geographies of the UK though. That's what the new 'mapping requirement' law promotes - bloody ridiculous mapping.

Dotty election map

Well that escalated quickly...

While I've been working on the forthcoming book and mooc I've been doing some data wrangling in the background at work. For the 2012 Presidential election I made a gallery of maps that illustrated diverse styles of cartography along with some comments on the map types. Each map can tell a different story of the election. I've been in the process of updating this with a new gallery of the 2016 election results (currently around ten maps but more to come) and I got to the tricky one - the dasymetric dot density map. It requires quite a bit of manipulation of data so here is the map, and in this blog I'll explain a little of the process.

--------------------------
Update: There's now a web map which shows the data at 6 scales in much more detail than the screengrab above. Check it out here or below:

---------------------------

In 2012 I made a similar map for the Obama/Romney election. It was a product of the web mapping technology of the time. Made using ArcMap (full disclosure for those who don't know I work for Esri - who make ArcGIS). At the smallest scale 1 dot = 1,000 votes. At the largest, 1 dot = 10 votes and if you printed the map out it would be as large as a football field. It took 3 months to cajole the largest scale map onto the web!!! I wanted to update the map and the four years that have intervened have brought new software capabilities. For 2012 I had to generate up to 12 million points and position them. Now, using ArcGIS Pro I can use the dot density renderer and let the software take the strain and if I were going all out then why not try and make a map where 1 dot = 1 vote. So, for me, the map is a technical challenge. Part of what I do at work to push the software to see what it is capable of, to test it and to show others what capabilities it affords.

So how to make the map? Well, it's a product of a number of decisions, each one of which propagates into the map. I'll be doing a proper write-up on the ArcGIS blog in due course but, in summary, a dasymetric map takes data held at one spatial unit (in this case counties) and reapportions it to different (usually smaller) areas. It uses a technique developed by the late Waldo Tobler called pycnophylactic reallocation modelling. Those different areas are, broadly, urban. The point of the map is to show where people live and vote rather than simply painting an entire county with a colour which creates a map that often misleads [Waldo sadly passed away recently and I was running the model when I heard of his death a couple of weeks ago. I met him a few times and his legacy to computational geography and cartography is immense].

I used the National Land Cover Database to extract urban areas. It's a raster dataset at 30m resolution. I used the impervious surface categories and created a polygon dataset with three classes, broadly dense urban, urban, and rural. I then did some data wrangling in ArcGIS Pro (more of that in a different blog) to reapportion the Democrat and Republican total votes at county level into the new polygons. There's some weighting involved so the dense urban polygons get (in total) 50% of the data. The urban get 35% of the data and the rural polygons get 15% of the data. Then I got the dot density renderer in ArcGIS Pro to draw the dots, one for each vote resulting in a map with nearly 130 million dots.

The result is a map that pushes the data into areas where people actually live. It leaves areas where no-one lives devoid of data. It reveals the structure of the US population surface. Most maps that take a dasymetric approach will all end up like this but I think there's value in the approach. To me it presents a better visual comparison of the amount of red and blue that the standard county level map that maps geography, not people, and overemphasises relatively sparsely populated large geographical areas.

So the map I saw on my desktop late Tuesday afternoon took 35 minutes to draw. Technical challenge achieved. ArcGIS Pro nailed it. This is a map that I couldn't have made in the previous election cycle. I was excited and so I took a quick screengrab, sent out a tweet and went home to walk Wisley the dog.

And that, I thought, was that. I'd put the map on the backburner and return to doing layout reviews for the book and doing last-minute work on the mooc over the next couple of weeks. But then something unexpected happened. My phone started pinging. Slowly at first but then a little more during the evening as people began to see the map on Twitter and like or re-tweet it. That's nice, I thought. I went to bed. Wednesday morning I woke to a relative avalanche of likes and retweets. I spent the day in Palm Springs at our Developer Summit and my phone never stopped. By the end of the day it had received around 3,000 likes and had been retweeted 2,000 times. I'm writing this Thursday morning and it's currently at 7,000 likes and a little over 3,000 retweets. The side-effect of this 15 minutes of map fame is I've picked up an extra 1,000 followers (25% increase) on my nearly 10 year old Twitter habit.

But there's a problem. The screengrab was quick and dirty and while there have been many and varied comments on the 'map' it's by no means the finished article. I want to create a hi-res version and also make a web map like the 2012 version. I don't have time to do this in the next couple of weeks but it will happen. But be assured, I am aware of a number of issues. Some have already spotted them and commented.

The symbols - I chose a very default red and blue. Each dot has 90% transparency so overlapping dots at this scale will undoubtedly coalesce into clumps. The impression will appear to bleed across the map. I need to tweak the colours (less saturated) and adjust the transparency to get a better effect. I will also likely do what I did for the 2012 map and classify the data so that at small scales 1 dot = 100 or 1,000 etc. To remove visual 'noise' at those scales. I'll also check for too many overlaps and overprinting. I actually think there's a problem in some areas with blue dots overprinting red. There should be more mixing and more purple. And no, there's no yellow dots. The map only displays Democrat and Republican votes in what remains, effectively, a binary voting outcome.

The data - it's county data, reapportioned. Dot maps convey a positioning that is a function of the processing, not where people actually live or vote. Dots are positioned randomly. Some have, quite reasonably, interpreted the map as showing where votes are and this is a fundamental drawback of the approach. No personal information is in the map at all. I also need to double-check a few areas where people have pointed out apparent anomalies in the map, compared to their personal knowledge of the areas. There may be errors. I need to check. That said, it's a function of the way I've used the NLCD so that data is the basis for reapportionment.

The geography - yes, I hold my hand up. There's no Alaska or Hawaii. I apologise. I'm not sure I'll go back as it requires doing some movement of those states to position them around the lower 48 and put them back in. It's easy but a non-trivial task when you're working in a GIS but I'll think about it. I understand this is unpalatable for some and I accept that criticism.

The interpretations - many have offered some fascinating insights into the gaps and the patterns through Twitter replies. I'll be going through these more carefully when the hullabaloo dies down and teasing out some. But more than anything I've been blown away by the nice things that have been said about the map. It shows the election result in a different way. It tells a different story. One of my favourite responses was this by Thomas de Beus...a lovely mashup and play on the classic photo of Trump's preferred view of the data to hang on the wall of the White House by Trey Yingst.

And this is the point of making a map like this. It presents the SAME data in a different way. It leads to different insights, different interpretations and a different perception. Neither of the above are right or wrong. They are different. Of course, we all have out own view on which serves our needs and which we prefer but that's for us as individuals.

My only regret is that I excitedly tweeted a rough version. I should have waited until I made the map properly. I'll do that but I suspect this is my one viral 15 minutes of fame and I regret it doesn't reflect the quality I know the final version will exhibit. A finished map likely won't get the same traction but we'll see. At the very least it has ignited a discussion. It brings different cartographic eyes to the dataset. Will it ever be hung in the White House? Unlikely.

Thanks for your interest and comments thus far!

Ken

Hurriedly written from a hotel in Palm Springs during which time the map's had many more likes, 11 more mentions and I've picked up another 86 followers. I can only apologise to them when they realise I tweet just as much about beer and football as I do about maps.