The William and Flora Hewlett Foundation recently approached us to help them explore ways of visualizing the distribution and impact of their grantmaking efforts over the last ten years.

Their goal was to make a tool with three functions: provide insight into where the Foundation has made the largest impact; provide grant seekers context for their applications; and help the Foundation’s officers make decisions about new grantmaking efforts, based on their existing portfolio.

They had one request: No maps.

Exploring the Data

The data arrived, as it so often does, in the rough: An Excel document compiled quickly, by hand, with the primary goal of providing an overview, rather than complete accuracy. Most of our projects start out exactly like this, and during our initial assessment and acclimation process we paint with broad brushes. We learn the data’s characteristics, determine which facets are interesting, and prototype visualization ideas.

At the beginning of a project, we always explore a few simple visualization techniques to get a feel for the data. For example, simple bar charts as shown in Figure 1, scatter plots, and choropleths, are great ways to get a visual sense of what the data is saying.

Our main tools for this process are d3.js, R — ggplot2 in particular — and Tableau. For the Hewlett Foundation’s project we used ggplot2 (version 0.9 came out halfway through) and the CRAN package ‘beanplot’.

Idea: Concentric Circles

Once we have a feel for the data, we start brainstorming, and trying out ideas.

An early idea led us to explore using concentric circles to represent the tree of geographic categories (Hemisphere, Continent, Country, Region, County, City), and then filling an arc of a circle with a scatter plot to show individual grants. You can see this idea sketched, with mostly fake data in the example below.

We ultimately decided this technique didn’t use space effectively enough for what we needed to convey.

Idea: BeanPlots

Our next idea was to use modified beanplots to succinctly describe the distribution of various quantities at the same time. These were made with the beanplot package available on CRAN. We denormalized them — meaning we hacked the beanplot function to make the total area of the beanplot proportional to volume.

With traditional beanplots, the total area of each bean is always the same, since they represent probability distributions rather than counts. This is counter-intuitive if the viewer is unfamiliar with statistics. We actually went as far as developing a working tool using these modified bean plots.

The width of the bean at a given dollar amount shows the probability the next dollar falls at the given amount. After extensive user testing, this proved too high a cognitive hurdle for the casual viewer. Users liked the visual presentation, but were confused as to their meaning, even with a detailed page showing how to interpret the beanplot.

Idea: Heatmaps

We decided to consider alternatives to the beanplot that still accomplish the same goals. We also wanted a very simple technique that could be explained in a phrase. After a few iterations, we agreed that interactive heatmaps would be a good solution. The data was easily viewable at a glance, sorted easily, worked on many scales, and allowed us to easily show binning of dollar ranges by using saturation.

R provides an ideal toolkit to explore methods to visualize data distributions. Between specialized packages and comprehensive toolkits like ggplot2, a wide range of techniques are available to the analyst. In particular, the transparent structure of most R functions make them easy to pull apart and put back together again, lending great flexibility to the patient programmer.

Final Interface

The final interface evolved the heatmap idea and includes a series of filters for the data. We also include a “key findings” section that allows visitors to jump to interesting sortings of the data, and allows the Hewlett Foundation to highlight the stories they felt were most compelling.

More:

Explore the entire visualization on Hewlett.org

See more details about the project including a video demonstration and screen caps