Dog watching or dog's dinner?

Watch_Dogs WeareData alleges to be the first web site to gather publicly available data about Paris, London and Berlin, in one location.  They map the data and create a visualisation of the soup we live in. Now I'm not going to be picky...oh, ok, that's the point of the blog!

So first off...it's not the first. CityDashboard by my friends at CASA has been around a good while and does a great job. Clean, simple, not over-selling what it does, informative. There are others.

So is the new one any good?

Well it looks 'nice'. it has a really strong aesthetic and is definitely very strong visually. It uses colour very well, particularly in receding contextual information while highlighting the detail using highly saturate colours with excellent use of shadows, glows and transparency.  It's one of those web maps you look at and genuinely thing...wow, that's actually 'different'. Beyond that is where the problems arise so I'll just offer some thoughts in no particular order...

Buildings are rendered very well with just a hint of outline and shadow to create the illusion of depth on a sphere from the central focal point. As you zoom in we increasingly move away from planimetric to a 3D panoramic view. The landscape gets a carbon fibre style treatment and we have building walls in transparent gradient fills. Very few names though and at large scale we need labels to determine where we are. Buildings aren't clickable. More labels would help.

Is there too much going on? There's movement, interaction and a lot of data being displayed.  Note I said data...but with a quick click we can reveal information for some data types for instance down to the one-thousandth of a second until a particular tube train arrives at the next station.  Really?  Does publicly available data really give us accuracy down to that degree of precision? Once you click on a moving train symbol though, the train becomes the centre of the map which pans to its destination...nice touch.

As you click other objects (e.g. CCTV camera symbols) and a Bladerunner style set of lines appears, shudders and then persists, glowing.  What? Are the cameras connected? Why can't I see a web stream? What is this connectivity showing?

Mobile cellphone masts are similarly clickable and seemingly connected.  But in reality they're not are they? Neither are ATMs or any other class of object other than purely as a map category.

Boris Bikes are symbolised with a green circle and give details on the number of bikes available but why not use the proper symbols? Yes, I know...copyright, but geometric shapes lose much of the power of mapping when common objects are mapped. Same for the London Underground stations. Some symbols are so engrained; so ubiquitous that seeing new representations is awkward and unfamiliar.

Traffic lights are included but...why? There's no roads mapped (though some road labels are visible) and no data associated with the lights. There's no information on traffic problems, accidents, heavy traffic or anything like that. If we could control them then maybe it would be fun but this is an example of mapping data because they managed to acquire/scrape it rather than having a purpose to mapping it.

What are the green polygons? Parkland? Again...not clickable and only a knowledge of the real world enables you to figure out what you're seeing.

Then we have the standard social media feeds...tweets, instagram, foursquare, Flickr. Really. Isn't this all getting a bit tiresome?  I clicked one instagram post and saw a picture of an empty bowl of sweets.  In the context of the city...what precisely is this telling me? Maybe that Rasheeb has run out of Quality Street? That's all. Nothing more. Pointless noise.

With all the nice geographically accurate buildings why do we have a stylised Underground network? Straight lines connecting stations? I'll tell you why...because that's how the data exists from the source used. But it jars, visually.

We get area stats that include a latitude, longitude to 6 decimal places...for an area? Where in the area? Many of the data fields are blank for some areas. Others appear static such as electricity consumption....oh, they are static...it's the average consumption for an entire year. So we have tube trains position reported to thousandths of a second and electricity consumption for a huge area of a city averaged over a year.  How is any of this comparable temporally when presented on the same map?

White text clashes with white symbols etc etc. And what are the annoying white micro-dots that seemingly hover over the city? Non are clickable. Non have any explanation? Argh.......

Heck it even has audio. Stick your headphones on and you're immediately in the trash compactor scene in Star Wars Episode IV...no, wait...there's some trancy music. It IS Bladerunner!

I could go on but you get the gist.  Apparently we can "discover more about the data that controls modern cities". Oh really? I hate to break this to you but you actually can't. Visual eye candy but beyond that, this is a random collection of data sets thrown on a map loosely and with no sense of purpose.

Answers on a postcard

Here's a map:

What does it show? It uses a rainbow colour palette that doesn't quite cover the basemap. It has a load of dot symbols of varying, equally bright colours. Some of the dots have a halo symbol behind them. Some of these halos exist without other overlying dots. Did someone spill a packet of Skittles on the map? Should it be titled 'a rainbow of fruit flavours'?

I have no idea what this map shows (sorry!). Answers on a postcard?

The dottiness of dot maps

Dot density mapping seems to be the new hexagon in mapping. We've seen iPhone vs Android neighbourhoods, locals vs tourists and languages mapped. The US census has been mapped as one dot per person. I had a bash at mapping the results of the 2012 Presidential election. There's plenty of others and Eric Fischer writes a nice discussion on some of the issues you have to contend with when making such maps (picking up some threads of a discussion I had with him after my previous blog).

And then we have The Guardian's "Every person in England and Wales on a map".

There isn't much information about the map's construction except each person in England and Wales has been represented with a dot. The data source is the 2011 census. That equates to 56,075,912 dots on the map.

According to The Guardian's Chris Cross it creates a 'beautiful picture of population density across the country" and you can " zoom in to the highest level to see the individual dots". OK then...here goes:

At it's proper size this is the maximum scale which is 1:72,224. I struggle to see individual dots. The reason is simple...the dots are too large so in the areas of most people they are coalescing into an amorphous blob. We have no way of seeing any variation amongst the most densely populated areas. All we get is a black fill for the underlying polygons.  This also creates the illusion of 'totality' in the sense that the area is absolutely rammed with people to the point of there being no room left for anyone else (is this the political point the map maker wanted to make?). Black is never a good colour to use on a map for 'fills'. Leave it for linework and labels....or make your dots sufficiently small so we can see some detail in these areas.

That's not the biggest problem though...

The map above clearly shows that the data has been mapped into boundaries.  These look like wards...the primary unit of English and Welsh electoral geography. It's not the finest scale of geography and it's conceivable they've used data reported at sub-units (e.g. Output Areas). And so the dots are placed randomly within the areas. That's a fairly standard technique for dot density mapping but let's not get carried away. This does not create a picture of population density. Using wards or other arbitrary boundaries means the data is constrained by the pattern that those boundaries create and NOT the pattern of where people live. So I don't find this a beautiful picture of population across England and Wales; I find it a visual misrepresentation of that statement.

The standard technique for mapping census data in wards is a choropleth map which, through the use of graduated colours, would allow us to compare the population density of areas across the country. It might be considered a 'boring' technique, particularly for the media, but it would be useful in this context. Instead we've got randomly placed dots across the entire country including all those areas only populated by parkland...or airports...or reservoirs and lakes...or sheep..or well, the map suggests that people exhaust space but in different densities. People don't.  In fact, many areas on the map will have far lower densities (practicaly zero) and many areas will actually be far more populated.

Here's a few of those land uses in the map extract above...

Whenever people look at dot maps, particularly those that suggest each dot is an individual data item (person in this case) the presumption is that the dot is the actual location.  The danger of imputing the characteristics of an area to a finer resolution of data is termed the ecological fallacy and this is manifest in The Guardian's map. They've taken data reported in areas and created a map that at least 'suggests' a finer scale of mapping than the data is capable of providing.  At best we know there are a lot of people in one area compared to another but the map suggests people are everywhere. The arbitrary boundaries of wards create the pattern we see, not the populated surface.

The map really should use a dasymetric technique which takes into account a secondary data set that allows us to position the source data more in line with where we logically know the data exists.  In this case, using urban land use would help. Using residential land use (because that's where the census reports location) would be even better. Additionally, mask out all the parks, airports and unpopulated areas and THEN use the dot density technique to distribute the dots in those new areas.  That's the technique I used for the dasymetric dot density map of the  2012 Presidential election data. I also avoided the ecological fallacy by deliberately NOT using 1 dot to symbolize 1 vote because I have no idea of voting patterns at that scale.  I left it at 1 dot is 10 votes.  I perhaps stretched things even then and maybe should have left it at 1 dot is 100 votes.  Either way...the map shouldn't suggest more than the data is capable of showing.

The point here (no pun intended) is that accurate mapping requires quite a bit of data processing.  It requires thought and, often, further data than simply the item to be mapped. It requires a knowledge and understanding of what you're doing and the limitations inherent in the technique and how you're crow-barring the data into it.

And the biggest problem of this sort of maverick cartography is the fact it gets huge readership.  It appears in a national publication, on a web site with large readership and is promoted by twitter accounts and blogs with large followers.  I've said before that one of the big dangers in modern cartography is that we're seeing even more maps made by people who are unaware of basic techniques who produce maps consumed by the uninitiated.  Cartography is important.  The point of cartographic techniques is to marshal the way we map in a way that avoids misinterpretations.  Cartography is not a set of rules to constrain innovation or stifle experimentation...it's a guide to ensure that the map's meaning will be interpreted accurately.  The Guardian have failed massively.  While what they've created isn't in itself wrong, the way it's presented is what causes the confusion.  Maps like this should have a series of caveats to accompany them so the rest of us don't make false assumptions. Honest cartography would be a good start.  Instead, these sort of maps seem to be made because the author has found a way to make them...thus solving a technical challenge.  In so doing they may get a viral map; they may be seen as innovative. Very few question the value of their cartographic approach but if we're to improve the honesty in modern cartography then we need to see these sort of maps be published with a greater sense of caution about what they show rather than the fanfare of technical triumphilism

A small tale of academic vs corporate behaviour

I had an interesting experience recently.  Someone made a map (and before anyone starts...this was not someone at the company I work for). It was going viral. I contacted said author privately and had a good conversation about the issue.  I'd faced a similar issue about 6 months ago and was able to give him some advice on how to tackle it. He then published a blog that paraphrased the conversation and explained how important it was to blah blah blah yackity schmackity (you get the idea). His blog on the issue is currently going viral. His 'insight' is taking his worth to new levels. Good luck to him, but he didn't even mention any conversation we had or the fact that he was both wrong and lacked understanding prior to someone reaching out to help. It wasn't necessary to acknowledge my help but the fact that he didn't somehow feels bad.

Yesterday I was asked to help an academic friend of mine with another map.  I experienced a few technical issues and couldn't quite get done what I had hoped but I offered a few pieces of technical advice which he then went off and implemented (expertly). When he published a blog post with the resulting work he was gracious to acknowledge my very small part in the process.  It wasn't at all necessary but it felt good. It's nice to be appreciated.

In the academic world, doing research and publishing ideas is based on learning, doing and reporting.  This is normally done objectively, with peer review and with proper attribution for work that went before including citations for previously published work and so on. It takes time. Often too much time... but you progress your research interests. You begin collaborating with like-minded people and the total often becomes more than the sum of the parts. You become an acknowledged expert in a niche area and you make a name for yourself based on your expertise. Your friends listen and it feeds future collaborations.  You become interested in quality over quantity. The art of an academic is keeping your friends close and working with them.

In the corporate world, it's more important to come first, be seen to be first, even if the ideas are old and someone got there before you anyway. You learn where to steal ideas and how to then pass them off as your own. You act as if you're engaged and interested but really the only thing you're interested in is bleeding someone of ideas and then pretending you know it all to the next person you meet. This doesn't take much time and you can iterate through various different types of work with speed and efficiency.  You become an expert at being a non-expert in the next best idea. You make a name for yourself regardless of any expertise and everyone listens. You become interested in quantity over quality. The art of being corporate is keeping your friends apart so they never find out they can bypass you entirely.

Having moved from academia to the corporate world I brought many good friends along with me.  Finding out who the new ones might be still poses a challenge every now and then.