How the west was really won: by manipulative data vizualization.

One of most well-known historical theories about America is the “frontier thesis,” advanced by Frederick Jackson Turner in 1893. It states that the long existence of a “frontier,” a zone between the settled and unsettled regions of the United States, is responsible for distinctive aspects of the American character: independence, self-reliance, egalitarianism, a certain disdain for high culture and learning, etc. The United States was, at independence, a fairly narrow strip of land east of the Mississippi, but the frontier was pushed continually west until eventually it reached the Pacific and vanished altogether.

You can see this movement on a sequence of population density maps, based on decennial census results, published in the 1898 Statistical atlas of the United States. Here’s the 1790 to 1860, when they didn’t even both showing the west:



Introduction to Wikidata using SPARQL

You’re certainly familiar with Wikipedia, but you may not be aware of Wikidata, which is an ongoing effort to structure some of the data underlying Wikipedia. Traditionally, facts (e.g. the population of New York City) are embedded in the text of a wiki, and there’s no easy way to automatically extract them. Wikipedia has a little more structure than this, but it’s still really designed for humans rather than machines.

Wikidata is the opposite – designed for machines, not humans.

It’s part of the broader semantic web movement, which aims to make the web more and more machine readable. Most of the time you don’t notice this, but when you run a query like “spouse of George Washington” and see this, rather than just a collection of links, that’s Google taking advantage of semantic web data (probably – they might also be using machine learning to infer it from unstructured text).

Screenshot 2017-09-11 16.45.07


Calculating Purchasing Power Parity is easy (but doing it well is difficult)

Many economic statistics seem conceptually simple, but are rather intricate and technical as actually measured (or more accurately, constructed). You can’t just line up dollars of GDP and count them. Price indices are similarly fiendish.

Conceptually, the purchasing power parity exchange rate between two countries is simply the relative cost, in local currency units, of buying the same basket of goods in each country. This can be very different from the market exchange rate, but is generally a better way to convert, say, GDP per capita, if you’re interested in cross-country comparisons of welfare (GDP? welfare? we’re already on shaky ground…)

Last week another Australian I know in DC was surprised to see that the International Comparison Program’s (ICP) 2015 PPP exchange rate for Australia is 1.487. Are things really 50% more “expensive” (dollar for dollar) in Australia, he wondered?