Has Christmas changed how your audience sees your site?

Hands up if someone you know gave or received a mobile phone or tablet over the holidays? And how long was it before they snuck away to quietly checked their favourite social networks or the sales with their new device? Some people will end up on a cultural heritage site. Sometimes it's because they now have a device to hand to look up random questions that arise while they're watching Downton Abbey or they're looking for entertainment for future commutes; others might try booking tickets for a show from their kitchen or keeping the kids quiet with a few games.

What will they see when they hit your site? Will the games and interactives have disappeared for Apple devices without Flash, will they struggle to fill in forms on your non-responsive site – or will they be welcomed to a site optimised for their device?

Of course the short answer is, yes, Christmas (and the past few years) have changed how audiences see your website. This post is a guide to using Google Analytics to put numbers against that statement and working out where you need to improve the experience for visitors on mobiles and tablets, but if you don't have access to Google Analytics then just assume lots of visitors are on mobile and make sure your site will work for them. There's no substitute for trying to perform typical visitor tasks with real devices, but emulators for iPadstablets, TVs, mobilesresponsive design, etc can help you get started. And if you're promoting content on social media and don't have a mobile/tablet ready site, then you're effectively inviting people over then slamming the door in their face, so just fix your site already.

If you do have access to Google Analytics, here are some tips for assessing the impact of all that gift-giving and working out the velocity of change in mobile and tablet visits on your site to understand what you're facing in the coming year (and getting to grips with Analytics while you're at it). Understanding how quickly your audience is changing and what people are doing on your site will a) help you decide which key tasks and sections to test with actual devices, PageSpeed Insights etc, and b) prioritise technical changes in the immediate future.

This assumes you know the basics of using Google Analytics – if you need a refresher, try the guides to Google Analytics Healthcheck and Google Analytics Segments I wrote for the Audience Agency's Audience Finder (though some of the screens have changed since then).

View mobile and tablet visits with built-in Google Analytics Segments and custom date ranges

The simplest way to assess how much of your website traffic is from mobile or tablet devices is to navigate to the Audiences/Mobile/Overview report, then click on the dates on the top right-hand corner and set the left-hand date to a year or two in the past and the right-hand date to now. You'll probably see a gradual increase in total visits over time, and some information underneath that about the total numbers of visits from 'desktop', 'mobile' and 'tablet' devices.

For a more useful breakdown of the number of desktop, mobile and tablet devices over time on other reports – whether Content reports like popular, Landing and Exit pages, location ('geo') or how people got to your site (aka 'acquisition') – you need to apply some Segments. To open the Segments option box, click the small down arrow next to 'All visits', as in the screenshot.

This will open a screen (below) showing a range of built-in segments. For now, click 'Mobile Traffic' and 'Tablet Traffic' to add them to the selected Segments list. Click 'Apply' and view the breakdown of visits by device over time. Hover over the lines for more detail. You can change reports and the segments will stay selected until you unselect them. (For later: explore other built-in segments, and learn how to make your own to answer questions that matter to your organisation.)

Tips: viewing stats by week rather than by day can help any patterns stand out more clearly. If you get a lot of traffic overall, you'll be able to see the difference in mobile/tablet visits more clearly if you take out the 'All Visits' segment.

Using the built-in date range fields to compare change over time

One of the key pieces of work I did for the Culture24 Let's Get Real project involved calculating the 'velocity' of change in mobile visits for a range of museums and arts organisations (see Section 8, 'Understanding mobile behaviours' in the project report). If you want to get a sense of how quickly your audience is changing (e.g. to make a case for resources), you can compare two date ranges. 
First, click the date range on the top right-hand corner to open the custom date options. Enter a date range that covers a period before and after the holidays, then tick 'Compare to: Previous Period' and add the same dates for an earlier year there.  Tip: copying and pasting then changing the dates is quicker than navigating to them via the left-hand side calendar.
The screenshot below shows the results on the Behaviour Overview report, including key indicators like the increase in time-on-site on tablets and the overall increase in visits from mobiles and tablets. As you can see, both mobile and tablet traffic is higher in the past week. This may even out as people head back to work, but the only way you'll know is by looking at your own stats.

Tips: if there are odd spikes or gaps in your stats, you might want to pick your dates around them (or add an explanatory annotation).  If you have the data, try comparing the same time of year over e.g. 2011/12 to see the difference a couple of years have made. If you want to dive into the numbers to understand the devices more, the 'Browser & OS' report is useful, or explore the 'Primary Dimensions' on the 'Devices' report.

Learn Analytics by answering questions specific to your site

Working with participants in the Let's Get Real project reminded me that having a specific question to answer is a good way to find your way through the mass of options in Google Analytics, so for bonus points, pick one or two of these to answer:

  • What traffic was there to your website on Christmas Day? 
  • How did they get there and what were they looking at? 
  • What kinds of transactions or interactions have people attempted on your site in the last six weeks? 
  • Where do you lose people? Does it vary by device or is another factor more important?
  • On which pages or site sections do tablet visitors spend the most time? What about mobile visitors? 
  • Was the increase in device usage larger this holiday or in previous years?
  • And following a comment from the Guardian's Tom Grinsted, how do visit demographics differ on weekends/weekdays? What about morning/daytime/evening visits?
Some museums have been blogging about their own investigations into mobile and tablet visits to their sites: for example, Graham Davies, National Museum Wales, wrote The steady march of the mobile device; the V&A's Andrew Lewis wrote Making visitor information easier for mobile phone users.

If an app is the answer…

…what was the question?*  And seriously, what questions should museums ask before investing in a mobile or tablet app?

First, some background. In We’re not ‘appy. Not ‘appy at all., Tom Loosemore of the Government Digital Service (GDS) gives examples of the increases in mobile traffic to UK government sites, including up to 60% mobile visits to site with complex transactions like booking driving tests.

For a cultural heritage perspective: as part of the Culture24 Let's Get Real project, I looked at the percentage of mobile visits (and visitors) to 22 cultural websites for Jan 1, 2012 – September 2, 2012 (an extended time to try and reduce the effect of the Olympics) and found that on average, museums and arts organisations were already seeing an average of 20% mobile visits.  I also reported the percentage change from the same period last year so that people could get a sense of the velocity of change: on average there was a 170% increase in mobile visits to cultural websites compared to the same period in 2011.

We'll re-run the stats when writing the final report in July, but in the meantime, there's a recent significant increase in web traffic from tablet devices to take into account.  In BBC iPlayer: tablet viewing requests nearly double in two months the Guardian reported BBC figures:

"Tablets' share of total iPlayer requests grew from just 6% (TV only: 7%) in January 2012 to 10% (12%) in November and 15% (18%) last month. Smartphone requests have seen similar growth from 6% (TV only: 6%) of the total a year ago to 16% (18%) in January. [… ]A spokesman said it is thought the rise of the "phablet" – smartphones that are almost as big as a tablet, such as the Samsung Note – that have driven the surge."

Some museums are reporting seeing 40-50% increases in tablet traffic in the past few months. So, given all that, are apps the answer?  Over to Loosemore:

"Our position is that native apps are rarely justified. […] Apps may be transforming gaming and social media, but for utility public services, the ‘making your website adapt really effectively to a range of devices’ approach is currently the better strategy. It allows you to iterate your services much more quickly, minimises any market impact and is far cheaper to support."

Obviously there are exceptions for apps that meet particular needs or genres, but this stance is part of their Government Digital Strategy:

"Stand-alone mobile apps will only be considered once the core web service works well on mobile devices, and if specifically agreed with the Cabinet Office." 

So if your cultural organisation is considering an app, perhaps you should consider the questions the GDS poses before asking for an exemption to the requirement to just build a responsive website:

  1. Is our web service already designed to be responsive to different screen sizes? If not, why not?
  2. What is the user need that only a native/hybrid app can meet?
  3. Are there existing native/hybrid apps which already meet this user need?
  4. Is our service available to 3rd parties via an API or open data? If not, why not?
  5. Does meeting this need justify the lifetime cost of a native or hybrid app?
What questions should we add for cultural heritage, arts and educational organisations?  (My pet hate: are you creating amazing content that's only accessible to people with the right device?) And since I know I'm being deliberately provocative – what exceptions should be allowed? What apps have you seen that could only work as an app with current technology?

* I can't claim credit for the challenge 'if an app is the answer, what was the question', it's been floating around for a while now and possibly originated at a Let's Get Real workshop or conference.

A sort of private joy? User-generated content and museums

I came across this lovely perspective on the content visitors create with museums:

We didn't start out asking people to leave their work, but it always happened. Now, we build it into the consideration of the activities that will be offered in the space. It isn't really like the formal artist-displaying-work model that is in evidence throughout the museum…the work is typically anonymous and individual pieces aren't highlighted.

When you walk into the space during the last month or so of an exhibition you experience the visitor-created artwork as a single, room-sized installation first, and only later do you focus on individual pieces. I think it is closer in some ways to the urge behind street art…the sort of private joy to be had from making something great and then leaving it behind for others to discover. I sometimes see visitors coming back to find something that they left behind a month or two before, not to reclaim it, just to see where it is now.

From the post 'Show your work' at the Indianapolis Museum of Art Blog.

It's talking about work created during physical visits to a museum rather than virtual visits but I think I noticed it because there's been a spurt of discussion about user-generated content and visitor participation on museum websites on the MCG mailing list following a workshop at the London Museums Hub on 'Understanding collections use and online access across the London Hub'.

A study presented at the event found an apparent lack of interest from museum website visitors in user-generated content, but in discussion at the event it appeared that the findings might have been different if the questions had been asked differently (with examples of some possible outcomes from UGC, perhaps 'would you like museum collections to be more discoverable because other visitors had tagged them with everyday words you'd use' rather than 'would you like opportunities to comment or upload content'), or if the focus groups hadn't been recruited from people who were physically visiting a museum and who were therefore fairly traditional museum-goers.

This lead to some interesting discussion about the differing reactions to the idea of opinions from other visitors versus real-life stories from other visitors; and of the idea that sometimes the value in user-created content lives with the person who contributes rather than those who read their contributions lately. This last idea was also raised at the User-generated content session at Museums and the Web in Montreal, my notes are here. I think the role of authority and trust and the influence of the context (type of museum or collection, user goal) need to be teased out into a more sophisticated model for analysing user-generated content in the cultural heritage sector.

There's a lot of research into user-generated content, participation and social software going on in the UK at the moment, it'd be great if there was somewhere that results, and ideally the raw data too, could be shared. Perhaps the MCG site?

Metrics and ROI for social software

A useful post about Social Media Metrics/Return on Investment with some thoughts on "how to provide useful metrics and measurements on the effects of social media for a nonprofit organization" and lots of useful links. It suggests "audience, engagement, loyalty, influence, and action" can put metrics in the "more holistic" context of outcomes, measures, strategy.

New models for measuring the effectiveness of websites

From an article called Evidence-based website management, some thoughts from Gerry McGovern on measuring the effectiveness of websites:

We can, with increasing precision, know what content gets people to act and what content doesn't. The length of time people spend on the page is just a basic measure. Here are some others:

  • How many links have there been to the content. This is the ultimate measure as the link is the gold standard of the Web.
  • Where did the customer go once they read the content? Did they have a positive or negative reaction?
  • How has the content been rated by customers?
  • Has the content been blogged about? Did it get a conversation going?

New culture secretary says era of "targets" is over

I wonder what effect this will have on our online metrics and evaluation.

James Purnell, 37, the newly-appointed secretary of State for Culture, made his first substantive speech calling for a change in statistical "targets" in the arts: "Targets were probably necessary in 1997 [when Labour came to power], to force a change of direction in some parts of the arts world. But now, we risk idolising them." He has appointed Sir Brian McMaster, a member of Arts Council England, to advise on "how we can remove crude targets".

Article from The Art Newspaper

I'm sure it'll be eons before it trickles down into the museum sector, but it's an interesting change:

Nielsen/NetRatings to use total time spent by users of a site as its primary measurement metric

In a nod to the success of emerging Web 2.0 technologies like AJAX and streaming media, one of the country's largest Internet benchmarking companies will no longer use page views as its primary metric for comparing sites.

Nielsen/NetRatings will announce Tuesday that it will immediately begin using total time spent by users of a site as its primary measurement metric.

Nielsen/NetRatings will still report page views as a secondary metric, and it will continue to reevaluate its primary metric as technology continues to evolve, Ross added. "For the foreseeable future, we will champion minutes if you are comparing two sites. Going forward, we'll see what that equates to in terms of true advertising opportunity," he said.