If you think about it, architects don’t build buildings, we make pictures of them. So one of the byproducts of an architectural training is an expertise in graphical representation. In fact, when I went to architecture school everyone had to have a book called Graphic Standards– around 1000 pages– each paper thin like a Bible– that describes how to represent all things related to the built environment so that building owners and contractors (like plumbers, electricians, roofers, carpenters) and the myriad vendors involved with the building process would be able to understand and act on the information.

So, the way that architects use different kinds of drawing conventions like plans and sections and axonometric and perspective– these are used to explain or communicate the various conditions of the building. And this is exactly what we do with data: we articulate conditions in data– you can think of visualization as different slices and perspectives of a dataset– so that human beings can understand and act on them.

Designing with data– infographics, visualization, decision support or whatever term suits your fancy (data-driven user experience is mine)– is inherently a right and left-brained competency involving creative and analytic processes– just like the design of buildings. the architect designs for a set of programmatic needs, but the building must stand up and withstand environmental conditions. But, ultimately the architect ensures the right experience of the building– is it beneficial to human needs and goals?

This is the approach we take to the challenges we see in the age of Big Data. But beyond the design of a single user interface into a dataset– like the design of an individual building– there is the design of the surrounding neighborhood and city supporting those pieces. We have some infrastructure and certain utilities for data available to us, but what is the built environment of data going to look like? Are we headed here (Washington DC, Daniel Burnham’s 1901 design):

Or here? (Kowloon’s Walled City, demolished in 1993, where lack of regulation enabled 33,000 people to build over and live within the space of one city block):

Listen to the podcast

This reminded me of a phenomenon I have struggled to describe, but which I have been interested in for some time. There is a state of heightened awareness we are all familiar with.  It happens after you see something interesting or intriguing that you can’t quite figure out.  For example, as you walk by a building, you happen to look through a glass door and see a car idling in what looks like the lobby of the building.  Wait a minute, you think after taking another step or two, what’s a car doing in the lobby of that building?  You stop and step back for another look and realize that what you thought was a lobby is in fact the courtyard of the building.  These misapprehensions occur every day and we probably for the most part easily forget them.  As you were walking by the building, maybe you had been thinking about some emails you have to answer, people you need to call, things to get at the store on your way home, whatever.  As soon as that odd visual presented itself to you, however, nothing was as interesting as figuring out what was going on.  During the moment when you were trying to resolve that confusion, everything else flew out of your mind.

What was going on in that example?  The context– the way the building presented itself to the street, the style and size of the glass door, etc.– all suggested to you that what you would see when you looked through the door is a typical building lobby.  What you did see, however, violated that expectation.  Or perhaps more precisely, your brain thought it had enough information to make a decent assumption, but you suddenly learned that it did not have sufficient information to make the particular assumption that a lobby stood behind the glass door.  If reality had aligned with our expectation– if you had seen an ordinary lobby behind the door– you wouldn’t have given it a second thought and may not even have noticed that you had seen the door or lobby at all.  On the other hand, if the door hadn’t looked like such a lobby door, or if the building hadn’t looked so much like other buildings on that street– all of which have lobbies– your brain might have reckoned that it didn’t have enough information to make any assumption at all about what was behind the door.  You wouldn’t have had any expectation that the space was anything in particular.  So seeing a car there probably wouldn’t have surprised you.  It was the disconnect between your assumption of what you would see, and what you did see, that created that state of heightened awareness.

Why spend so much time on an example like this?  Three reasons:

1. A state of heightened awareness is valuable, so it’s helpful to understand what may produce it
2. I have an unconfirmed suspicion that this example is related to effective reporting and visualization of data
3. I like odd and uncanny stuff like this

For now, I’ll focus on #2.

I think the link between my example above and data visualization has to do with pattern recognition.  Consider these propositions:

Case 1: Too much data. This  is chaotic and overwhelming.  We are unable to discern a recognizable pattern, so we do not make any assumptions about what the data tells us:

Case 2: Too little data. This is boring.  We barely notice that there is useful information there at all, and don’t even bother to discern a pattern or make assumptions.

Somewhere in the middle– where the amount of data is about right– I suggest there are two possibilities:

Case 3: The amount of data is about right. The pattern is clear and easy to recognize. We instantly understand the situation:

Case 4: The amount of data is about right. There seems to be a pattern. Maybe the pattern is not what we expected to see.  Maybe it’s difficult to make out or understand the pattern.  Either way, we want to learn more to figure out what’s going on:

How does this relate to reporting and visualization?  Here are my thoughts:

Case 1 & Case 2 = Not useful

Case 3 = Useful reporting.  This gives you an answer to a question you have asked.

Case 4 = Useful data visualization/exploration.  This gives you insight, allowing you to ask more questions.

Fast Company Magazine profiles Angela Shen-Hsieh in its annual Masters of Design issue of next generation rising stars.

Read the article

People expect that a product would be developed through a design process before it was manufactured and released into the market, or that an architect would be hired to design a building before ground was broken. Making data fit for human consumption requires a design process as well: it demands left and right brain thinking that unites the two ends of the problem– the available data, and the human analytical processes that enable informed, confident decision-making.

Read how we bring analytics to everyone in Information Management Magazine

Someone said to me recently “We’ll prepare 300 slides for management—they only end up looking at 10, and they ask for things we didn’t do. We can never anticipate how the conversation will go so we are always going back to the drawing board.”

That’s a lot of inefficiency and a lot of cycles at a time when all businesses are trying to do more with less. This loop between management questions and analyst responses has created a culture of ‘analysis interruptus’. Another senior business analyst told me “I have seen over the years that senior managers have trained themselves NOT to ask too many questions because they’ve seen how off-hand remarks can send analyst teams down a rat hole and burn weeks of time.”

There are two lines of attack—one is to enable management and business users with easier to use tools for analysis, reducing report requests and decision cycle times.  The other is to better equip analysts and report writers to have more answers at their finger tips to reduce the manual efforts and lead times of one-off reporting projects—less need to go back to that drawing board. Both of these strategies rely on visualization (to convey complexity and spot patterns quickly) and interactivity (to ask and answer questions and iterate analysis on-the-fly). When these two capabilities are brought together in a fluid and intuitive way, analytics can transcend the technology of cubes and queries and algorithms and become a dialogue or conversation with data. This is what I think of as conversational analytics.

Well, not exactly. Visualization actually requires a whole different set of skills than UI and web design. Fundamentally, visualization is about decision making—understanding the information and its context better so that you can ask better questions, get better answers, and make better choices. UI design is workflow—like a data entry form, a website shopping cart, or trying to figure out how to reset the bullet formatting in Powerpoint.

Both have an important element of navigation, which may be why the two seem interchangeable. But one is navigation based upon the way the data is interpreted via the visualization (you can’t say where you are going but you’ll know it when you get there). The other is navigation to accomplish a task—fairly straight-forward to map with predictable parameters.

And visualization is data-driven. This, it turns out, also demands a different approach and skills than UI design. That’s because UI designers are used to being able to control the way something looks, but visualization renders based on the data driving it. It’s hard to get away from the legacy of the graphic design thinking that has dominated screen-based design—but a fundamentally print-based approach demands control over almost every pixel and the way any page or view will render. Visualization is not about designing information graphically, it’s really about designing graphical systems that will render unknown and unpredictable data into consistent, intuitive, visual metaphors.

Angela Shen-Hsieh and Mark Schindler have also developed the proprietary technology, Visual i|o, which provides on-the-fly deployment of interactive visualization for data exploration and discovery. Their team uses this technology, as well as other industry standard and off-the-shelf tools in their technology solutions.

Watch the video at ZD Net.

Read Angela Shen-Hsieh’s answer