These are a few of my favourite (audience research) things

On Friday I popped into London to give a talk at the Art of Digital meetup at the Photographer’s Gallery. It’s a great series of events organised by Caroline Heron and Jo Healy, so go along sometime if you can. I talked about different ways of doing audience research. (And when I wrote the line ‘getting to know you’ it gave me an earworm and a ‘lessons from musicals’ theme). It was a talk of two halves. In the first, I outlined different ways of thinking about audience research, then went into a little more detail about a few of my favourite (audience research) things.

There are lots of different ways to understand the contexts and needs different audiences bring to your offerings. You probably also want to test to see if what you’re making works for them and to get a sense of what they’re currently doing with your websites, apps or venues. It can help to think of research methods along scales of time, distance, numbers, ‘density’ and intimacy. (Or you could think of it as a journey from ‘somewhere out there’ to ‘dancing cheek to cheek’…)

‘Time’ refers to both how much time a method asks from the audience and how much time it takes to analyse the results. There’s no getting around the fact that nearly all methods require time to plan, prepare and pilot, sorry! You can run 5 second tests that ask remote visitors a single question, or spend months embedded in a workplace shadowing people (and more time afterwards analysing the results). On the distance scale, you can work with remote testers located anywhere across the world, ask people visiting your museum to look at a few prototype screens, or physically locate yourself in someone’s office for an interview or observation.

Numbers and ‘density’ (or the richness of communication and the resulting data) tend to be inversely linked. Analytics or log files let you gather data from millions of website or app users, one-question surveys can garner thousands of responses, you can interview dozens of people or test prototypes with 5-8 users each time. However, the conversations you’ll have in a semi-structured interview are much richer than the responses you’ll get to a multiple-choice questionnaire. This is partly because it’s a two-way dialogue, and partly because in-person interviews convey more information, including tone of voice, physical gestures, impressions of a location and possibly even physical artefacts or demonstrations. Generally, methods that can reach millions of remote people produce lots of point data, while more intimate methods that involve spending lots of time with just a few people produce small datasets of really rich data.

So here are few of my favourite things: analytics, one-question surveys, 5 second tests, lightweight usability tests, semi-structured interviews, and on-site observations. Ultimately, the methods you use are a balance of time and distance, the richness of the data required, and whether you want to understand the requirements for, or measure the performance of a site or tool.

Analytics are great for understanding how people found you, what they’re doing on your site, and how this changes over time. Analytics can help you work out which bits of a website need tweaking, and for measuring to see the impact of changes. But that only gets you so far – how do you know which trends are meaningful and which are just noise? To understand why people are doing what they do, you need other forms of research to flesh them out. 
One question surveys are a great way of finding out why people are on your site, and whether they’ve succeeded in achieving their goals for being there. We linked survey answers to analytics for the last Let’s Get Real project so we could see how people who were there for different reasons behaved on the site, but you don’t need to go that far – any information about why people are on your site is better than none! 
5 second tests and lightweight usability tests are both ways to find out how well a design works for its intended audiences. 5 second tests show people an interface for 5 seconds, then ask them what they remember about it, or where they’d click to do a particular task. They’re a good way to make sure your text and design are clear. Usability tests take from a few minutes to an hour, and are usually done in person. One of my favourite lightweight tests involves grabbing a sketch, an iPad or laptop and asking people in a café or other space if they’d help by testing a site for a few minutes. You can gather lots of feedback really quickly, and report back with a prioritised list of fixes by the end of the day. 
Semi-structured interviews use the same set of questions each time to ensure some consistency between interviews, but they’re flexible enough to let you delve into detail and follow any interesting diversions that arise during the conversation. Interviews and observations can be even more informative if they’re done in the space where the activities you’re interested in take place. ‘Contextual inquiry’ goes a step further by including observations of the tasks you’re interested in being performed. If you can ‘apprentice’ yourself to someone, it’s a great way to have them explain to you why things are done the way they are. However, it’s obviously a lot more difficult to find someone willing and able to let you observe them in this way, it’s not appropriate for every task or research question, and the data that results can be so rich and dense with information that it takes a long time to review and analyse. 
And one final titbit of wisdom from a musical – always look on the bright side of life! Any knowledge is better than none, so if you manage to get any audience research or usability testing done then you’re already better off than you were before.

[Update: a comment on twitter reminded me of another favourite research thing: if you don’t yet have a site/app/campaign/whatever, test a competitor’s!]

Who loves your stuff? How to collect links to your site

If you’ve ever wondered who’s using content from your site or what people find interesting, here are some ways to find out, using the Design Museum’s URL as an example.

‘Links to your site’ via Google Webmaster Tools https://support.google.com/webmasters/answer/55281

Reddit – plug your URL in after /domain/
http://www.reddit.com/domain/designmuseum.org

Wikipedia – plug your URL in after target=
http://en.wikipedia.org/w/index.php?title=Special%3ALinkSearch&target=*.designmuseum.org
Depending on your topic coverage you may want to look at other language Wikipedias.

Pinterest – plug your URL in after /source/
http://www.pinterest.com/source/designmuseum.org/

Twitter – search for the URL with quotes around it e.g. “designmuseum.org”

If you can see one particular page shooting up in your web stats, you could try a reverse image search on TinEye to see where it’s being referenced.

What am I missing? I’d love to hear about similar links and methods for other sites – tell me in the comments or on twitter @mia_out.

Update: in a similar vein, Tim Sherratt @wragge launched a new experiment called Trove Traces the same day, to ‘explore how Trove newspapers are used’ by listing pages that link to articles: http://trovespace.webfactional.com/traces/

Update 2: Desi Gonzalez @desigonz tried out some of these techniques and put together a great post on ‘Thoughts on what museums can learn from Reddit, Yelp, and what @briandroitcour calls vernacular criticism
You might also be interested in: Can you capture visitors with a steampunk arm?

Has Christmas changed how your audience sees your site?

Hands up if someone you know gave or received a mobile phone or tablet over the holidays? And how long was it before they snuck away to quietly checked their favourite social networks or the sales with their new device? Some people will end up on a cultural heritage site. Sometimes it’s because they now have a device to hand to look up random questions that arise while they’re watching Downton Abbey or they’re looking for entertainment for future commutes; others might try booking tickets for a show from their kitchen or keeping the kids quiet with a few games.

What will they see when they hit your site? Will the games and interactives have disappeared for Apple devices without Flash, will they struggle to fill in forms on your non-responsive site – or will they be welcomed to a site optimised for their device?

Of course the short answer is, yes, Christmas (and the past few years) have changed how audiences see your website. This post is a guide to using Google Analytics to put numbers against that statement and working out where you need to improve the experience for visitors on mobiles and tablets, but if you don’t have access to Google Analytics then just assume lots of visitors are on mobile and make sure your site will work for them. There’s no substitute for trying to perform typical visitor tasks with real devices, but emulators for iPadstablets, TVs, mobilesresponsive design, etc can help you get started. And if you’re promoting content on social media and don’t have a mobile/tablet ready site, then you’re effectively inviting people over then slamming the door in their face, so just fix your site already.

If you do have access to Google Analytics, here are some tips for assessing the impact of all that gift-giving and working out the velocity of change in mobile and tablet visits on your site to understand what you’re facing in the coming year (and getting to grips with Analytics while you’re at it). Understanding how quickly your audience is changing and what people are doing on your site will a) help you decide which key tasks and sections to test with actual devices, PageSpeed Insights etc, and b) prioritise technical changes in the immediate future.

This assumes you know the basics of using Google Analytics – if you need a refresher, try the guides to Google Analytics Healthcheck and Google Analytics Segments I wrote for the Audience Agency’s Audience Finder (though some of the screens have changed since then).

View mobile and tablet visits with built-in Google Analytics Segments and custom date ranges

The simplest way to assess how much of your website traffic is from mobile or tablet devices is to navigate to the Audiences/Mobile/Overview report, then click on the dates on the top right-hand corner and set the left-hand date to a year or two in the past and the right-hand date to now. You’ll probably see a gradual increase in total visits over time, and some information underneath that about the total numbers of visits from ‘desktop’, ‘mobile’ and ‘tablet’ devices.

For a more useful breakdown of the number of desktop, mobile and tablet devices over time on other reports – whether Content reports like popular, Landing and Exit pages, location (‘geo’) or how people got to your site (aka ‘acquisition’) – you need to apply some Segments. To open the Segments option box, click the small down arrow next to ‘All visits’, as in the screenshot.

This will open a screen (below) showing a range of built-in segments. For now, click ‘Mobile Traffic’ and ‘Tablet Traffic’ to add them to the selected Segments list. Click ‘Apply’ and view the breakdown of visits by device over time. Hover over the lines for more detail. You can change reports and the segments will stay selected until you unselect them. (For later: explore other built-in segments, and learn how to make your own to answer questions that matter to your organisation.)

Tips: viewing stats by week rather than by day can help any patterns stand out more clearly. If you get a lot of traffic overall, you’ll be able to see the difference in mobile/tablet visits more clearly if you take out the ‘All Visits’ segment.

Using the built-in date range fields to compare change over time

One of the key pieces of work I did for the Culture24 Let’s Get Real project involved calculating the ‘velocity’ of change in mobile visits for a range of museums and arts organisations (see Section 8, ‘Understanding mobile behaviours’ in the project report). If you want to get a sense of how quickly your audience is changing (e.g. to make a case for resources), you can compare two date ranges. 
First, click the date range on the top right-hand corner to open the custom date options. Enter a date range that covers a period before and after the holidays, then tick ‘Compare to: Previous Period’ and add the same dates for an earlier year there.  Tip: copying and pasting then changing the dates is quicker than navigating to them via the left-hand side calendar.
The screenshot below shows the results on the Behaviour Overview report, including key indicators like the increase in time-on-site on tablets and the overall increase in visits from mobiles and tablets. As you can see, both mobile and tablet traffic is higher in the past week. This may even out as people head back to work, but the only way you’ll know is by looking at your own stats.

Tips: if there are odd spikes or gaps in your stats, you might want to pick your dates around them (or add an explanatory annotation).  If you have the data, try comparing the same time of year over e.g. 2011/12 to see the difference a couple of years have made. If you want to dive into the numbers to understand the devices more, the ‘Browser & OS’ report is useful, or explore the ‘Primary Dimensions’ on the ‘Devices’ report.

Learn Analytics by answering questions specific to your site

Working with participants in the Let’s Get Real project reminded me that having a specific question to answer is a good way to find your way through the mass of options in Google Analytics, so for bonus points, pick one or two of these to answer:

  • What traffic was there to your website on Christmas Day? 
  • How did they get there and what were they looking at? 
  • What kinds of transactions or interactions have people attempted on your site in the last six weeks? 
  • Where do you lose people? Does it vary by device or is another factor more important?
  • On which pages or site sections do tablet visitors spend the most time? What about mobile visitors? 
  • Was the increase in device usage larger this holiday or in previous years?
  • And following a comment from the Guardian’s Tom Grinsted, how do visit demographics differ on weekends/weekdays? What about morning/daytime/evening visits?
Some museums have been blogging about their own investigations into mobile and tablet visits to their sites: for example, Graham Davies, National Museum Wales, wrote The steady march of the mobile device; the V&A’s Andrew Lewis wrote Making visitor information easier for mobile phone users.