In Part One of this post, I mentioned Reach Advisors’ report on visitors’ preferences for computers. In their nationwide surveys of 40,000+ from over a 100 museums, they found that only 11% of visitors liked using computers to receive information. In response to the crowing of the Luddites online, I felt compelled to offer up a little analysis of their analysis. Just remember, I’m not a professional evaluator, just a consumer of evaluations. You’ve been warned.
I ended my last post with the question “What’s going on here?” Their “findings” and the way they’ve chosen to present them are pretty sensationalist. But I have some problems with their methodology. The big one is their sample.
Sampling
40,000 people sounds like an impressive amount, and it is. But if you look closely at their report, you see that their response rate across all the museums they worked with hovers around 5%. That’s a tiny amount of the visitorship of any museum, and that 5% were not a random sample of visitors. They are exclusively people who volunteered to fill out a survey from the museum. And if you’ve ever read visitor comment cards from your museum you know that by and large they are written exclusively by people with strong feelings one way or the other. The Museum of Science has been conducting random visitor surveys for some time, and one of the things we’ve learned is that members’ opinions differ greatly from non-members when it comes to satisfaction. Without a random sample, it’s not really possible to make broad claims about museum visitors, or even “core” visitors as they call them. With a 5% uptake rate, they can’t even say how their sample differs from the museum-going population as a whole.
Definitions (or “a car is a car, right?”)
My main beef is the way they’ve defined “computer interactives” to be the most boring, vanilla implementation of computers in museums — the lonely kiosk. I agree with their reasoning that it would’ve been too hard to explain to visitors all the ways that computing technology might be embedded in an experience that doesn’t look like a computer. For solid results I think you’d need to have an evaluator on hand explaining it. But then, I wouldn’t be quite so liberal with the proclamations about their results. I’m not a big fan of kiosks either. But my beef with them is usually a design problem, not an inherent quality of their “computerness” which is how it is possible to interpret Reach Advisors’ report.
I’ve always taken it as a given, but maybe it needs saying. In the hierarchy of interactions, the pinnacle has always been person to person. If we could station an interpreter at every exhibit, there’d be no need for people who do what I do. Given that we can’t have people everywhere, we settle for interactive exhibits to give visitors as rich an experience as possible. Which takes me to the part of the report I actually love, which I mentioned in the first part of this post.
“Conclusions”
Looking back over the report, it almost feels to me like two different people wrote it. After starting out with some grand statements about visitors and computers, they wind up saying what every exhibit designer or developer hopefully learns on Day One; that every situation is unique, and that reflective practitioners focus on the ideas and experiences they want the visitor to have, and then design an experience that delivers on that,using the best technologies to achieve it. I wrote about this some time ago, in a post called Listen! What’s the Work Telling You?
The takeaway message of the report seems to be, “Be a thoughtful developer and don’t just use a tool or technique for it’s own sake!” This is terrific advice, and one that more people should take, but it’s not really a conclusion in the classical sense. It doesn’t necessarily flow from their data; it’s just good practice. A thoughtful designer chooses the right tools for the job. And there are some experiences that are best handled by a computer, like manipulating data, and displaying dynamic responses to changing inputs.Some experiences are best handled as mechanical interactives. And many only work as staff-mediated experiences. It’s the developer’s job to pick the right one.
And finally…
Lest you think I’m a complete ingrate, sneering at the huge amount of work Reach Advisors has done and shared for free, be assured that is not the case. I think this report should be a great conversation starter. I’ve already had some great talks with colleagues at other museums about the report itself, about what else we could/should ask visitors, and more. I just wouldn’t use the report as anything more than suggestive data.
What did you think of the report?
Thinking about Museums 
I’m going to just pick up on this –
See I think this is the wrong purpose for interactive exhibits – and we need to come up with a better justification. They shouldn’t be there to replace people – in fact when I’ve visited the Miraikan in Tokyo they have actually had human explainers on every exhibit station. They’ve also had interactive experiences which are there to enhance and compliment the visit. This is what the Star Wars ones in the Boston show did well.
I’m thinking the ideal place to be is a kiosk-less museum but one still rich with digital experiences. It’d be interesting to check the Reach results against our own research where we do ask about different types of ‘computerised experience’.
LikeLike
I’d love to see what kind of comparative data you might come up with. I think many of us are sitting on interesting piles of findings that might help move us past the kiosk phase, and into whatever the next phase is. Might be an interesting conference session in there…
LikeLike
A quick Luddite response:
The general portrayal of particular groups (the Amish, the Luddites) as being “anti-technology” is nearly always simplistic. The original Luddites, for instance, were loom operators and as such were most certainly not against technology insofar as tool use is how they earned their living. The Amish, as another case, do not typically reject the use of telephones, they simply do not like them in their homes. A good idea if I’ve ever heard one.
Rather, we Luddites reject the indiscriminate use of technology. Nowhere is this more the case than with computers. Too often, the computer is a solution in search of a problem. I met a woman once who explained – at length – the wonderfulness of a new program she had bought her son. It simulated blocks. She explained that when one virtual block was placed on another, it made a clicking sound and how that if the virtual blocks were not properly stacked, they would virtually tip over. You can see what’s coming: why not just buy some blocks? They have a wonderful texture and smell. They seep into your child’s memories as they become wonderfully worn around the edges.
When I was a physics teacher, my colleagues would do computer simulations of falling objects. I – by contrast – would take my students out to the bleachers and we would instead drop all manner of real objects from varying real vertical displacements. I’m sorry, but I win.
I do not reject computers. There are five of them in my house, one of which was used to type and post this little note. But my personal, somewhat limited, and undeniably anecdotal experience with computers in museum exhibits has been this: yawn. Why? To me, the reason is clear: they have computers in their house. And their workplace. And their cars. And their phones. And their school. When they get to the museum, please, show them something else.
Hey Ed!
LikeLike
Hey!
Guilty on all counts, Mr. Rogers. Thanks for calling me on it. The tension between brevity and clarity is always tough to get right. I think we’re all saying similar things, including the original report’s authors. Use the right tool for the job, not just because it’s “the new thing.”
So here’s my question for you. Given that we’re all opposed to gratuitous use of computers in exhibits, do you have an example or a situation in mind that embodies an appropriate use of the tool, a non-yawn inducing use?
LikeLike
Excellent question. Remember the “Build a Speaker” activity we were considering? For your readers: The main components of a speaker are the magnet, the coil, and the cone. We considered having an activity wherein the visitor could combine various instantiations of these components (large or small, different materials, etc.) and then see how the speaker they had built would behave. Great idea. The problems, though, were non-trivial. The components were small (easy to walk away with), difficult to fabricate (expensive), and – especially concerning the cones – fragile (high maintenance).
But imagine this as a computer activity: A screen with a bin of virtual components, an area on the screen in which the visitor could “drag and drop” the components to build the speaker, and – perhaps at the bottom – slides controls for both frequency and volume so the visitor could test out their design. This activity simply begs for a touch screen.
As a designer prone to gadget-making, I must remain aware of my inclination to perhaps too quickly dismiss the computer option. The example above – I think – illustrates that the computer is sometimes the way to go.
LikeLike