Oh no! I feel another multipart post coming on… How many parts, I don’t yet know, maybe just two. I’ll strive for brevity.
Reach Advisors have come out with a new finding from the large museum study they conducted in 2010, and I love it, though maybe not for the reasons that others will. You should go read the post. It’s short (2 pages) and isn’t weighed down with charts or facts. It’s just findings and lots of helpfully highlighted bits ready to be tweeted and emailed broadly.
The gist of this post is that computer interactives are one of the least attractive ways visitors prefer to receive content, coming in at only 11%. This is across a big national sample of visitors (n=40,000+) to all kinds of museums. This is pretty earth-shattering stuff, right? Well, yes and no…
The first time I read this post, I sat there and said, “Whoa! That doesn’t jibe with my own experience or evaluations I’ve seen previously, so what’s up? If computer interactives suck so bad, that’s a big deal.” Better go dig into it a bit more and see what’s there.
The first thing I did was look at our own findings. Our Star Wars exhibition was evaluated at two venues, and the results for both venues are largely the same. Go read the report. At MOS the percentage of diligent visitors (%DV) was 51% and the sweep rate was 208, what Beverly Serrell would classify as an exceptionally
thoroughly used exhibit. It was also full of computer interactives, two of which were in the top ten exhibits in terms of holding power. Considering their competition included Luke’s Landspeeder, Darth Vader, and Yoda, their inclusion in the top ten doesn’t jibe that well with Reach’s finding.
Now before you get all snippy and make rude comments about people who come to Star Wars exhibitions, let me say that the exhibition drew a broad family audience. There might be a dork in the group visiting, but there were more non-fans than fans. But I agree it is not a typical case, being a paid exhibition. People who have plunked down a wad of cash to see a timed-ticketed show are more likely to use as much as they can.
So what other data are out there? Luckily for us, we can all look at the findings from a study Smithsonian did in 2010 of preferences and expectations for information and electronic services by Smithsonian visitors. It’s a pretty thoroughgoing look at what average SI visitors (not just the ones who volunteer to fill out an online survey) felt about technology. They asked over 1,000 visitors, and found that 26% felt that computer kiosks were useful tools. Not surprisingly, text (brochures and labels) comes out on top, followed by live guides, and films. Are Smithsonian’s visitors’ more than twice as interested in computers than the U.S. visitors polled by Reach?
Next part – What’s going on here?
Visitors’ preference for computers, Part Two
In Part One of this post, I mentioned Reach Advisors’ report on visitors’ preferences for computers. In their nationwide surveys of 40,000+ from over a 100 museums, they found that only 11% of visitors liked using computers to receive information. In response to the crowing of the Luddites online, I felt compelled to offer up a little analysis of their analysis. Just remember, I’m not a professional evaluator, just a consumer of evaluations. You’ve been warned.
I ended my last post with the question “What’s going on here?” Their “findings” and the way they’ve chosen to present them are pretty sensationalist. But I have some problems with their methodology. The big one is their sample.
Sampling
40,000 people sounds like an impressive amount, and it is. But if you look closely at their report, you see that their response rate across all the museums they worked with hovers around 5%. That’s a tiny amount of the visitorship of any museum, and that 5% were not a random sample of visitors. They are exclusively people who volunteered to fill out a survey from the museum. And if you’ve ever read visitor comment cards from your museum you know that by and large they are written exclusively by people with strong feelings one way or the other. The Museum of Science has been conducting random visitor surveys for some time, and one of the things we’ve learned is that members’ opinions differ greatly from non-members when it comes to satisfaction. Without a random sample, it’s not really possible to make broad claims about museum visitors, or even “core” visitors as they call them. With a 5% uptake rate, they can’t even say how their sample differs from the museum-going population as a whole.
Definitions (or “a car is a car, right?”)
a car by Flickr user indy_slug
a car by Flickr user Peter Martin Hall
My main beef is the way they’ve defined “computer interactives” to be the most boring, vanilla implementation of computers in museums — the lonely kiosk. I agree with their reasoning that it would’ve been too hard to explain to visitors all the ways that computing technology might be embedded in an experience that doesn’t look like a computer. For solid results I think you’d need to have an evaluator on hand explaining it. But then, I wouldn’t be quite so liberal with the proclamations about their results. I’m not a big fan of kiosks either. But my beef with them is usually a design problem, not an inherent quality of their “computerness” which is how it is possible to interpret Reach Advisors’ report.
I’ve always taken it as a given, but maybe it needs saying. In the hierarchy of interactions, the pinnacle has always been person to person. If we could station an interpreter at every exhibit, there’d be no need for people who do what I do. Given that we can’t have people everywhere, we settle for interactive exhibits to give visitors as rich an experience as possible. Which takes me to the part of the report I actually love, which I mentioned in the first part of this post.
“Conclusions”
Looking back over the report, it almost feels to me like two different people wrote it. After starting out with some grand statements about visitors and computers, they wind up saying what every exhibit designer or developer hopefully learns on Day One; that every situation is unique, and that reflective practitioners focus on the ideas and experiences they want the visitor to have, and then design an experience that delivers on that,using the best technologies to achieve it. I wrote about this some time ago, in a post called Listen! What’s the Work Telling You?
The takeaway message of the report seems to be, “Be a thoughtful developer and don’t just use a tool or technique for it’s own sake!” This is terrific advice, and one that more people should take, but it’s not really a conclusion in the classical sense. It doesn’t necessarily flow from their data; it’s just good practice. A thoughtful designer chooses the right tools for the job. And there are some experiences that are best handled by a computer, like manipulating data, and displaying dynamic responses to changing inputs.Some experiences are best handled as mechanical interactives. And many only work as staff-mediated experiences. It’s the developer’s job to pick the right one.
And finally…
Lest you think I’m a complete ingrate, sneering at the huge amount of work Reach Advisors has done and shared for free, be assured that is not the case. I think this report should be a great conversation starter. I’ve already had some great talks with colleagues at other museums about the report itself, about what else we could/should ask visitors, and more. I just wouldn’t use the report as anything more than suggestive data.
What did you think of the report?
Tell your friends:
Like this:
8 Comments
Posted in Design process, New Media, Professional Development, Reviews, Thinking tools
Tagged comment cards, computers, design, museum visitors, professional evaluator, Reach Advisors