Looking for (crowdsourcing) love in all the right places

One of the most important exercises in the crowdsourcing workshops I run is the 'speed dating' session. The idea is to spend some time looking at a bunch of crowdsourcing projects until you find a project you love. Finding a project you enjoy gives you a deeper insight into why other people participate in crowdsourcing, and will see you through the work required to get a crowdsourcing project going. I think making a personal connection like this helps reduce some of the cynicism I occasionally encounter about why people would volunteer their time to help cultural heritage collections. Trying lots of projects also gives you a much better sense of the types of barriers projects can accidentally put in the way of participation. It's also a good reminder that everyone is a nerd about something, and that there's a community of passion for every topic you can think of.

If you want to learn more about designing history or cultural heritage crowdsourcing projects, trying out lots of project is a great place to start. The more time you can spend on this the better – an hour is ideal – but trying just one or two projects is better than nothing. In a workshop I get people to note how a project made them feel – what they liked most and least about a project, and who they'd recommend it to. You can also note the input and output types to help build your mental database of relevant crowdsourcing projects.

The list of projects I suggest varies according to the background of workshop participants, and I'll often throw in suggestions tailored to specific interests, but here's a generic list to get you started.

10 Most Wanted http://10most.org.uk/ Research object histories
Ancient Lives http://ancientlives.org/ Humanities, language, text transcription
British Library Georeferencer http://www.bl.uk/maps/ Locating and georeferencing maps (warning: if it's running, only hard maps may be left!)
Children of the Lodz Ghetto http://online.ushmm.org/lodzchildren/ Citizen history, research
Describe Me http://describeme.museumvictoria.com.au/ Describe objects
DIY History http://diyhistory.lib.uiowa.edu/ Transcribe historical letters, recipes, diaries
Family History Transcription Project http://www.flickr.com/photos/statelibrarync/collections/ Document transcription (Flickr/Yahoo login required to comment)
Herbaria@home http://herbariaunited.org/atHome/ (for bonus points, compare it with Notes from Nature https://www.zooniverse.org/project/notes_from_nature) Transcribing specimen sheets (or biographical research)
HistoryPin Year of the Bay 'Mysteries' https://www.historypin.org/attach/project/22-yearofthebay/mysteries/index/ Help find dates, locations, titles for historic photographs; overlay images on StreetView
iSpot http://www.ispotnature.org/ Help 'identify wildlife and share nature'
Letters of 1916 http://dh.tcd.ie/letters1916/ Transcribe letters and/or contribute letters
London Street Views 1840 http://crowd.museumoflondon.org.uk/lsv1840/ Help transcribe London business directories
Micropasts http://crowdsourced.micropasts.org/app/photomasking/newtask Photo-masking to help produce 3D objects; also structured transcription
Museum Metadata Games: Dora http://museumgam.es/dora/ Tagging game with cultural heritage objects (my prototype from 2010)
NYPL Building Inspector http://buildinginspector.nypl.org/ A range of tasks, including checking building footprints, entering addresses
Operation War Diary http://operationwardiary.org/ Structured transcription of WWI unit diaries
Papers of the War Department http://wardepartmentpapers.org/ Document transcription
Planet Hunters http://planethunters.org/ Citizen science; review visualised data
Powerhouse Museum Collection Search http://www.powerhousemuseum.com/collection/database/menu.php Tagging objects
Reading Experience Database http://www.open.ac.uk/Arts/RED/ Text selection, transcription, description.
Smithsonian Digital Volunteers: Transcription Center https://transcription.si.edu/ Text transcription
Tiltfactor Metadata Games http://www.metadatagames.org/ Games with cultural heritage images
Transcribe Bentham http://www.transcribe-bentham.da.ulcc.ac.uk/ History; text transcription
Trove http://trove.nla.gov.au/newspaper?q= Correct OCR errors, transcribe text, tag or describe documents
US National Archives http://www.amara.org/en/teams/national-archives/ Transcribing videos
What's the Score at the Bodleian http://www.whats-the-score.org/ Music and text transcription, description
What's on the menu http://menus.nypl.org/ Structured transcription of restaurant menus
What's on the menu? Geotagger http://menusgeo.herokuapp.com/ Geolocating historic restaurant menus
Wikisource – random item link http://en.wikisource.org/wiki/Special:Random/Index Transcribing texts
Worm Watch http://www.wormwatchlab.org Citizen science; video
Your Paintings Tagger http://tagger.thepcf.org.uk/ Paintings; free-text or structured tagging

NB: crowdsourcing is a dynamic field, some sites may be temporarily out of content or have otherwise settled in transit. Some sites require registration, so you may need to find another site to explore while you're waiting for your registration email.

It's here! Crowdsourcing our Cultural Heritage is now available

My edited volume, Crowdsourcing our Cultural Heritage, is now available! My introduction (Crowdsourcing our cultural heritage: Introduction), which provides an overview of the field and outlines the contribution of the 12 chapters, is online at Ashgate's site, along with the table of contents and index. There's a 10% discount if you order online.

If you're in London on the evening of Thursday 20th November, we're celebrating with a book launch party at the UCL Centre for Digital Humanities. Register at http://crowdsourcingculturalheritage.eventbrite.co.uk.

Here's the back page blurb: "Crowdsourcing, or asking the general public to help contribute to shared goals, is increasingly popular in memory institutions as a tool for digitising or computing vast amounts of data. This book brings together for the first time the collected wisdom of international leaders in the theory and practice of crowdsourcing in cultural heritage. It features eight accessible case studies of groundbreaking projects from leading cultural heritage and academic institutions, and four thought-provoking essays that reflect on the wider implications of this engagement for participants and on the institutions themselves.

Crowdsourcing in cultural heritage is more than a framework for creating content: as a form of mutually beneficial engagement with the collections and research of museums, libraries, archives and academia, it benefits both audiences and institutions. However, successful crowdsourcing projects reflect a commitment to developing effective interface and technical designs. This book will help practitioners who wish to create their own crowdsourcing projects understand how other institutions devised the right combination of source material and the tasks for their ‘crowd’. The authors provide theoretically informed, actionable insights on crowdsourcing in cultural heritage, outlining the context in which their projects were created, the challenges and opportunities that informed decisions during implementation, and reflecting on the results.

This book will be essential reading for information and cultural management professionals, students and researchers in universities, corporate, public or academic libraries, museums and archives."

Massive thanks to the following authors of chapters for their intellectual generosity and their patience with up to five rounds of edits, plus proofing, indexing and more…

  1. Crowdsourcing in Brooklyn, Shelley Bernstein;
  2. Old Weather: approaching collections from a different angle, Lucinda Blaser;
  3. ‘Many hands make light work. Many hands together make merry work’: Transcribe Bentham and crowdsourcing manuscript collections, Tim Causer and Melissa Terras;
  4. Build, analyse and generalise: community transcription of the Papers of the War Department and the development of Scripto, Sharon M. Leon;
  5. What's on the menu?: crowdsourcing at the New York Public Library, Michael Lascarides and Ben Vershbow;
  6. What’s Welsh for ‘crowdsourcing’? Citizen science and community engagement at the National Library of Wales, Lyn Lewis Dafis, Lorna M. Hughes and Rhian James;
  7. Waisda?: making videos findable through crowdsourced annotations, Johan Oomen, Riste Gligorov and Michiel Hildebrand;
  8. Your Paintings Tagger: crowdsourcing descriptive metadata for a national virtual collection, Kathryn Eccles and Andrew Greg.
  9. Crowdsourcing: Crowding out the archivist? Locating crowdsourcing within the broader landscape of participatory archives, Alexandra Eveleigh;
  10.  How the crowd can surprise us: humanities crowdsourcing and the creation of knowledge, Stuart Dunn and Mark Hedges;
  11. The role of open authority in a collaborative web, Lori Byrd Phillips;
  12. Making crowdsourcing compatible with the missions and values of cultural heritage organisations, Trevor Owens.

We're all looking at the stars: citizen science projects at ZooCon13

Last Saturday I escaped my desk to head to the Physics department at the University of Oxford and be awed by what we're learning about space (and more terrestrial subjects) through citizen science projects run by Zooniverse at ZooCon13. All the usual caveats about notes from events apply – in particular, assume any errors are mine and that everyone was much more intelligent and articulate than my notes make them sound. These notes are partly written for people in cultural heritage and the humanities who are interested in the design of crowdsourcing projects, and while I enjoyed the scientific presentations I am not even going to attempt to represent them!  Chris Lintott live-blogged some of the talks on the day, so check out 'Live from ZooCon' for more. If you're familiar with citizen science you may well know a lot of these examples already – and if you're not, you can't really go wrong by looking at Zooniverse projects.

Aprajita Verma kicked off with SpaceWarps and 'Crowd-sourcing the Discovery of Gravitational Lenses with Citizen Scientists'. She explained the different ways gravitational lenses show up in astronomical images, and that 'strong gravitational lensing research is traditionally very labour-intensive' – computer algorithms generate lots of false positives, so you need people to help. SpaceWarps includes some simulated lenses (i.e. images of the sky with lenses added), mostly as a teaching tool (to provide more examples and increase familiarity with what lenses can look like) but also to make it more interesting for participants. The SpaceWarps interface lets you know when you've missed a (simulated, presumably) lens as well as noting lenses you've marked. They had 2 million image classifications in the first week, and 8500 citizen scientists have participated so far, 40% of whom have participated in 'Talk', the discussion feature. As discussed in their post 'What happens to your markers? A look inside the Space Warps Analysis Pipeline', they've analysed the results so far on ranges between astute/obtuse and pessimistic/optimistic markers – it turns out most people are astute. Each image is reviewed by ten people, so they've got confidence in the results.

Karen Masters talked about 'Cosmic Evolution in the Galaxy Zoo', taking us back to the first Galaxy Zoo project's hopes to have 30,000 volunteers and contrasting that with subsequent peer-reviewed papers that thanked 85,000, or 160,000 or 200,000 volunteers. The project launched in 2007 (before the Zooniverse itself) to look at spiral vs elliptical galaxies and it's all grown from there. The project has found rare objects, most famously the pea galaxies, and as further proof that the Zooniverse is doing 'real science online', the team have produced 36 peer reviewed paper, some with 100+ citations. At least 50 more papers have been produced by others using their data.

Phil Brohan discussed 'New Users for Old Weather'. The Old Weather project is using data from historic ships logs to help answer the question 'is this climate change or just weather?'. Some data was already known but there's a 'metaphorical fog' from missing observations from the past. Since the BBC won't let him put a satellite in a Tardis, they've been creative about finding other sources to help lift 'the fog of ignorance'. This project has long fascinated me because it started off all about science: in Phil's words, 'when we started all this, I was only thinking about the weather', but ended up being about history as well: 'these documents are intrinsically interesting'– he learnt what else was interesting about the logs from project participants who discovered the stories of people, disasters and strange events that lay within them. The third thing the project has generated (after weather and history) is 'a lot of experts'. One example he gave was evidence of the 1918-19 Spanish flu epidemic on board ship, which was investigated after forum posts about it. There's still a lot to do – more logs, including possibly French and Dutch – to come, and things would ideally speed up 'by a factor of ten'.

In Brooke Simmons' talk on 'Future plans for Galaxy Zoo', she raised the eternal issue of what to call participants in crowdsourcing: 'just call everyone collaborators'. 'Citizen scientists' makes a distinction between paid and unpaid scientists, as does 'volunteers'. She wants to help people do their own science, and they're working on making it easier than downloading and learning how to use more complicated tools. As an example, she talked about people collecting 'galaxies with small bulges' and analysing the differences in bulges (like a souped-up Galaxy Zoo Navigator?). She also talked about Zoo Teach, with resources for learning at all ages.

After the break we learnt about 'The Planet 4 Invasion', the climate and seasons of Mars from Meg Schwamb and about Solar Stormwatch in 'Only you can save planet Earth!' from Chris Davis, who was also presenting research from his student Kim Tucker-Wood (sp?). Who knew that solar winds could take the tail off a comet?!

Next up was Chris Lintott on 'Planet Hunting with and without Kepler'. Science communication advice says 'don't show people graphs', and since Planet Hunters is looking at graphs for fun, he thought no-one would want to do Planet Hunters. However, the response has surprised him. And 'it turns out that stars are actually quite interesting as well'. In another example of participants going above and beyond the original scope of the project, project participants watched a talk streamed online on 'heartbeat binaries', and went and found 30 of them from archives, their own records and posted them on the forum.  Now a bunch of Planet Hunters are working with Kepler team to follow them up.  (As an aside, he showed a screenshot of a future journal paper – the journal couldn't accept the idea that you could be a Planet Hunter and not be part of an academic team so they're listed as the Department of Astronomy at Yale.)

The final speaker was Rob Simpson on 'The Future of the Zooniverse'.  To put things in context, he said the human race spends 16 years cumulatively playing the game Angry Birds every day; people spend 2 months every day on the Zooniverse. In the past year, the human race spent 52 years on the Zooniverse's 15 live projects (they've had 23 projects in total). The Andromeda project went through all their data in 22 days – other projects take longer, but still attract dedicated people.  In the Zooniverse's immediate future are 'tools for (citizen) scientists' – adding the ability to do analysis in the browser, 'because people have a habit of finding things, just by being given access to the data'. They're also working on 'Letters' – public versions of what might otherwise be detailed forum posts that can be cited, and as a form of publication, it puts them 'in the domain'.  They're helping people communicate with each other and embracing their 'machine overlords', using Galaxy Zoo as a training tool for machine learning.  As computers get more powerful, the division of work between machines and people will change, perhaps leaving the beautiful, tricky, or complex bits for humans. [Update, June 29, 2013: Rob's posted about his talk on the Zooniverse blog, '52 Years of Human Effort', and corrected his original figure of 35 years to 52 years of human effort.]

At one point a speaker asked who in the room was a moderator on a Zooniverse project, and nearly everyone put their hand up. I felt a bit like giving them a round of applause because their hard work is behind the success of many projects. They're also a lovely, friendly bunch, as I discovered in the pub afterwards.

Conversations in the pub also reminded me of the flipside of people learning so much through these projects – sometimes people lose interest in the original task as their skills and knowledge grow, and it can be tricky to find time to contribute outside of moderating.  After a comment by Chris at another event I've been thinking about how you might match people to crowdsourcing projects or tasks – sometimes it might be about finding something that suits their love of the topic, or that matches the complexity or type of task they've previously enjoyed, or finding another unusual skill to learn, or perhaps building really solid stepping stones from their current tasks to more complex ones. But it's tricky to know what someone likes – I quite like transcribing text on sites like Trove or Notes from Nature, but I didn't like it much on Old Weather. And my own preferences change – I didn't think much of Ancient Lives the first time I saw it, but on another occasion I ended up getting completely absorbed in the task. Helping people find the right task and project is also a design issue for projects that have built an 'ecosystem' of parts that contribute to a larger programme, as discussed in 'Using crowdsourcing to manage crowdsourcing' in Frequently Asked Questions about crowdsourcing in cultural heritage and 'A suite of museum metadata games?' in Playing with Difficult Objects – Game Designs to Improve Museum Collections.

An event like ZooCon showed how much citizen science is leading the way – there are lots of useful lessons for humanities and cultural heritage crowdsourcing. If you've read this thinking 'I'd love to try it for my data, but x is a problem', try talking to someone about it – often there are computational techniques for solving similar problems, and if it's not already solved it might be interesting enough that people want to get involved and work with you on it.

Notes from 'The Shape of Things: New and emerging technology-enabled models of participation through VGC'

I've just spent two days in Leicester for the 'The Shape of Things: New and emerging technology-enabled models of participation through VGC' conference at the school of Museum Studies, part of the AHRC-funded iSay project focusing on Visitor-Generated Content (VGC) in heritage institutions. There will be lots of posts on the conference blog, so these are just some things that struck me or I've found useful concepts for thinking about my own museum practice.

I tweeted about the event as I headed to Leicester, and that started a conversation about the suitability of the term 'visitor-generated content' that continued through the event itself. I think it was Giasemi who said that one problem with 'visitor-generated content' is that the term puts the emphasis on content and that's not what it's about. Jeremy Ottevanger suggested 'inbound communications' as a possible replacement for VGC.

The first keynote was Angelina Russo, who reminded us of the importance of curiosity and of finding ways to make museum collections central to visitor engagement work. She questioned the value of some comments left on museum collections other than the engagement in the process of leaving the comment. Having spent too much time reviewing visitor comments, I have to agree that not all comments (particularly repetitive ones) have inherently valuable content or help enhance another visitor's experience – a subject that was debated during the conference. A conversation over twitter during the conference with Claire Ross helped me realise that designing interfaces that respect and value the experience of both the commenter and reading is one of the interesting challenges in digital participation.

She then used Bourdieu's ideas around 'restricted cultural production' to characterise the work of curators as producers who create cultural goods for other producers, governed by specific norms and sanctions, within relatively self-contained communities where their self-esteem depends on peers. However, this creates a tension between what curators think their role is and what museums need it to be in an age when museums are sites of large-scale cultural production for 'the public at large', driven by a quest for market share and profits. Visitor-generated content and the related issues of trust, authority, or digitisation highlight the tensions between these models of restricted or large-scale cultural production – we need to find 'a pathway through the sand'. Angelina suggested that a version of Bourdieu's 'gift economies', where products are created and given away in return for recognition might provide a solution, then asked what's required to make that shift within the museum. How can we link the drive for participation with the core work of museums and curatorial scholarship? She presented a model (which I haven't gone into here) for thinking about 'cultural communication', or communication which is collection-led; curiosity-driven; is scholarly; experiential; and offers multi-platform opportunities for active cultural participation, engagement and co-creation.

Carl Hogsden from the Museum of Archaeology and Anthropology and University of Cambridge talked about the Reciprocal Research Network and moving beyond digital feedback to digital reciprocation. This project has been doing innovative work for a long time, so it was good to see it presented again.

Jenny Kidd from Cardiff University posed some useful questions in 'VGC and ethics – what we might learn from the media and journalism' – it's questionable how much VGC (or user-generated content, UGC) has actually changed journalism, despite the promise of increased civic engagement, diversity, more relevant news and a re-framing of the audience as active citizens rather than consumers. One interesting point was the impact of the 'Arab Spring' on UGC – content that couldn't be verified couldn't be shown by traditional media so protesters started including establishing shots and improving the quality of their recordings. This was also the first of several papers that referenced 'Whose cake is it anyway', a key text for conversations about visitor participation and museums and Jenny suggested that sometimes being seen to engage in participatory activity is currently possibly end goal in itself for a museum. She presented questions for further research and debate including: is the museum interested in quality of process or product of VGC and do creators feel the same? How does VGC fit in workflow models of museums?

Giasemi Vavoula's paper on 'The role of VGC in digital transformations in Museum Learning' (slides) was fascinating, particularly as it presented frameworks for audience engagement taken from learning theory that closely matched those I'd found from studies of citizen science and engagement in heritage and sport (e.g. cognitive engagement model – highest is theorising, then applying, relating, explaining, describing, note-taking, memorising… Good visitor experiences get most visitors to use the higher engagement level processes that the more focused visitors use spontaneously). I love learning from Learning people – in museum learning/visitor studies, social interaction facilitates learning; visitors negotiate the meanings of exhibits through conversation with their companions. Giasemi called for museums to weave VGC into the fabric of visitors social contexts; to scaffold and embed it into visiting experience; and to align with visitors and organisations' social agendas.

In 'A Tale of Two Workhouses' Peter Rogers and Juliet Sprake spoke of 'filling in the gaps rather than being recipients of one-way information flow', which tied in nicely with discussion around the role of curiosity in audience participation.

In the afternoon there was a Q&A session with Nina Simon (via skype). A number of the questions were about sustainability, designing for mixed contexts, and the final question was 'where next from here?'. Nina advised designing participatory experiences so that people can observe the activity and decide to take part when they're comfortable with it – this also works for designing things that work as spectator experiences for people who don't want to join in. Nina's response to a question about 'designing better questions' – 'find questions where you have genuine interest in what the visitor has to say about it' – resonated with wider discussion about meaningful visitor participation. Nina talked about the cumulative effect of participatory work on the museum itself, changing not only how the museum sees itself but how others see it – I wonder how many museums in the UK are engaging with visitor participation to the extent that it changes the museum itself? Nina also made the point that you tend to have either highly participatory process to make conventional product, or conventional process to make highly participatory product, and that not everything has to be wholly participatory from start to finish, which is useful for thinking how co-creative projects.

On Friday morning I gave a keynote on 'crowdsourcing as productive engagement with cultural heritage'. My slides for 'Crowdsourcing as productive engagement with cultural heritage' are now online. I partly wanted to problematise the power relationships in participatory projects – whose voice can affect change? – and to tease out different ways of thinking about crowdsourcing in cultural heritage as productive both in terms of the process (engaging in cultural heritage) and the product (the sheer number of items transcribed, corrected, etc). I've been going back to research on motivations for volunteering in cultural heritage, working on open source projects and reviewing discussions with participants in crowdsourcing projects, and I hope it'll help people design projects that meet those altruistic, extrinsic and intrinsic motivations. Thinking about my paper in the context of the other presentations also got me thinking about the role of curiosity in audience engagement and encouraging people to start researching a subject (whether a ship's history, an individual or a general topic) more deeply. On a personal note, this paper was a good chance to reflect on the different types of audience engagement with museum collections or historic sources and on the inherent value of participation in cultural heritage projects that underpin my MSc and PhD research and my work in museums generally.

Areti Galani presented research she'd done with Rachel Clarke (Newcastle University), and asked 'how can accessible technology lead to inaccessible participation paradigms?'. I was really interested in the difference between quality of the visitor contributions in-gallery vs online (though of course 'quality' is a highly subjective term), a question that surfaced through the day. Areti's research might suggest that building in some delay in the process of contributing in-gallery could lead to better quality (i.e. more considered) contributions. The novelty of the technology used might also have an effect – 'pen-happy visitors' who used the technology for the sake of interacting with it but didn't know what to do after picked up the pen.

The paper from Jeremy Ottevanger (Imperial War Museums) on "Social Interpretation" as a catalyst for organisational change generated more discussion on possible reasons why online comments on museum sites tend to be more thoughtful than in-gallery comments, with one possible reason being that online commenters have deliberately sought out the content, so already have a deeper engagement with those specific items, rather than just coming across them while moving through the physical gallery. Jeremy talked about the need for the museum to find an internal workflow that was appropriately responsive to online comments – in my experience, this is one of the most difficult issues in planning for digital projects. Jeremy presented a useful categorisation of online contributions as personal (emotional, opinion, personal information, anecdotes, family history), requests and queries (object info, valuation, family history, digitisation and licencing, offering material, access, history, general/website), and informational (new information, corrections) and looked at which types of contribution were responded to by different departments. He finished with a vision of the IWM harnessing the enthusiasm and knowledge of their audiences to help serve the need of other audiences, of connecting people with expertise with people who have questions.

Jack Ashby talked about finding the right questions for the QRator project at the Grant Museum of Zoology – a turtle is a turtle, and there's not a lot of value in finding out what visitors might want to call it, but asking wider questions could be more useful. Like the wider Social Interpretation project, QRator always raises questions for me about whether museums should actively 'garden' visitor interactives, pruning out less relevant questions to create a better experience for other visitors.

Rolf Steier and Palmyre Pierroux discussed their findings on the role of the affordances of social media and visitor contributions in museums. Rosie Cardiff talked about the Tate's motivations for participatory projects with audiences, and audience motivations for participating in Tate's projects. She presented some considerations for organisations considering participatory projects: who is the audience? What motivations for visitor and for organisation? What platform will you use? How will the content be moderated? (Who will do it?) Where will it sit in relation to organisational space online or in-gallery? How long will it run for? What plans for archiving and maintaining content beyond lifetime of project? How will you measure success? How will you manage audience expectations about what's going to happen to their work? This last point was also picked up in discussions about audience expectations about how long museums will keep their contributions.

The final presentation was Ross Parry's keynote on 'The end of the beginning: Normativity in the postdigital museum. Based on new research into how six UK national (i.e. centrally funded, big, prestigious museums) have started to naturalise 'digital' into their overall museum vision, this paper gave me hope for the future. There's still a long way to go, but Ross articulated a vision of how some museums are integrating digital in the immediate future, and how it will integrated once the necessary stage of highlighting 'digital' in strategies, organisational structures and projects has given way to a more cohesive incorporation of 'digital' into the fabric of museums. It also makes sense in the context of discussions about digital strategies in museums over the past year (e.g. at the Museums Assocation and UK Museums on the Web (themes, my report) conferences).

I had to leave before the final session, so my report ends here, but I expect there'll be more reports on the project blog and I've saved an archive of isayevent_tweets_2013_02_01 (CSV).

I think the organisers, Giasemi Vavoula and Jenny Kidd, did a great job on the conference programme. The papers and audience were a well-balanced combination of academics and practioners – the academic papers gave me interesting frameworks to think with, and the case studies provided material to think about.

'War, Plague and Fire' and 'Bootstrapping Innovation in Museums' at 'Museum Ideas 2012 – Museums in the Era of Participatory Culture'

I've finally had a moment to catch up and post the first part of my notes from Museum/iD's conference, Museum Ideas 2012 – Museums in the Era of Participatory Culture. Overall it was a great conference that left me with a lot of things to think about for how museums can adapt and thrive in the current international context, and reminded me why museums should survive: they matter. I've posted my thoughts from the later sessions at Why museums matter: 'Museum Ideas 2012 – Museums in the Era of Participatory Culture' with a short summary of the whole event at the start.

Sharon Ament's keynote at Museum of London Docklands

The day was chaired by Ben Gammon who began by pointing out that innovation is no longer a luxury, it's now critical for survival.

The keynote speaker was the new Director of the Museum of London, Sharon Ament, who spoke on War, Plague and Fire: museums and libraries in the era of participatory culture. Previously Ament was director of public engagement at the Natural History Museum, and she drew on that background in her talk while also relating it to the collections of the Museum of London and the docklands location of the conference. She called for museums to look at what participatory culture means to the people they serve, especially when the individual has the capacity to be heard more loudly than ever before. The international context in which we're living – with civil unrest, economic crises and global warming – is a time of change and fear means that adaptation to the external environment is an important concept for museums today. Her talk, and some of the discussion afterwards, focused on the role of museums and libraries as venues for independent discovery; accessible to many because entry was free. She suggested that creative responses – small things that can happen spontaneously, like the 'pop-up' concept – might be useful for reaching people.

One final quote to close, from the Salzburg Global Seminar and the Institute of Museum and Library Services report on 'Libraries and Museums in an Era of Participatory Culture': 'technology is a tool, not an objective, and that the creation of increased public value is the end goal. Identifying stakeholders’ needs means addressing human relationships, a sense of organization, and an intellectual construct to shape information and access'.

The next session was a 'fireside chat' with Rob Stein (Dallas Museum of Art) and Seb Chan (Cooper-Hewitt Museum) reflecting on 'Bootstrapping Innovation in Museums' and their experiences in changing museums. They discussed collaboration (Stein noted that everything he's built that's had a modicum of success has been a collaboration with lots of people), the pace of change in different museums (including the need to build a risk-tolerant culture), and the risk of assuming that technology is an inherent part of innovation (Stein observed that the change that needs to happen at DAM is cultural, about shifting ambition). How do you create a culture of innovation? Chan mentioned Elaine Heumann Gurian's Wanting to be the Third on your Block and said that the first thing he did when he started at the Cooper-Hewitt was create a space that gave people permission to change. He set up 'labs' as a space for people to talk about stuff, which also gave his immediate team a public voice for the first time. He pushed fast to get quick results on some straightforward things to start to set an expectation of speed and accelerate culture: 'right now, doing things fast matters more than doing things well'. He talked about cultivating rogues and tricksters in the museum to accelerate change and get a paradigm shift and suggested tackling root problems rather than symptoms for issues like copyright. They also discussed how to play up the fun of museum jobs to make them more attractive in a competitive tech jobs market, and the importance of putting some money into innovation where possible. Stein suggested that it's possible to support innovation without a budget, e.g. museums can hold 'research forums' where people share what they're working on.

Chan also said museums have turned themselves into 'exhibition farms', letting them suck huge amounts of resource; together with the obsession with 'finish' this slows innovation that could come from re-thinking how exhibitions and public programmes work together. Stein observed 'museums seem to like gargantuan problems, things that take five years to get out the door [like] exhibitions, publications, buildings.'

They discussed the mismatch between museum exhibition launch models and software models: 'people want to feel that something's finished when it launches, they want the party and the holiday'. But in software development, no-one takes a holiday straight after launch because they're watching what people do with the new software. [I was really interested in this section as it's something I've thought about a lot (e.g. does a museum's obsession with polish hinder innovation?) – I suspect museum technologists have two clashing mental models about how to work: one is the web agency model, based around cycles of 'launch, observe, iterate, update'; the other is the 'long slog to an unmovable launch date then onto the next project' of museums. When the rest of the world moves on the agile, iterative model, it's frustrating being tied to the museum model, particularly when it seems to have more flaws than benefits for modern audiences.] In closing they talked about the effectiveness of various models of innovation, whether attempts at top-down innovation, departments of innovation or more integrated models of innovation.

This post is already quite long, so I might hit publish now and come back to the other talks later.

Disclosure: my ticket was provided by Museum/iD.

Frequently Asked Questions about crowdsourcing in cultural heritage

Over time I've noticed the repetition of various misconceptions and apprehensions about crowdsourcing for cultural heritage and digital history, so since this is a large part of my PhD topic I thought I'd collect various resources together as I work to answer some FAQs. I'll update this post over time in response to changes in the field, my research and comments from readers. While this is partly based on some writing for my PhD, I've tried not to be too academic and where possible I've gone for publicly accessible sources like blog posts rather than send you to a journal paywall.

If you'd rather watch a video than read, check out the Crowdsourcing Consortium for Libraries and Archives (CCLA)'s 'Crowdsourcing 101: Fundamentals and Case Studies' online seminar.

[Last updated: February 2016, to address 'crowdsourcing steals jobs'. Previous updates added a link to CCLA events, crowdsourcing projects to explore and a post on machine learning+crowdsourcing.]

What is crowdsourcing?

Definitions are tricky. Even Jeff Howe, the author of 'Crowdsourcing' has two definitions:

The White Paper Version: Crowdsourcing is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.

The Soundbyte Version: The application of Open Source principles to fields outside of software.

For many reasons, the term 'crowdsourcing' isn't appropriate for many cultural heritage projects but the term is such neat shorthand that it'll stick until something better comes along. Trevor Owens (@tjowens) has neatly problematised this in The Crowd and The Library:

'Many of the projects that end up falling under the heading of crowdsourcing in libraries, archives and museums have not involved large and massive crowds and they have very little to do with outsourcing labor. … They are about inviting participation from interested and engaged members of the public [and] continue a long standing tradition of volunteerism and involvement of citizens in the creation and continued development of public goods'

Defining crowdsourcing in cultural heritage

To summarise my own thinking and the related literature, I'd define crowdsourcing in cultural heritage as an emerging form of engagement with cultural heritage that contributes towards a shared, significant goal or research area by asking the public to undertake tasks that cannot be done automatically, in an environment where the tasks, goals (or both) provide inherent rewards for participation.

Screenshot from 'Letters of 1916' project.

Who is 'the crowd'?

Good question!  One tension underlying the 'openness' of the call to participate in cultural heritage is the fact that there's often a difference between the theoretical reach of a project (i.e. everybody) and the practical reach, the subset of 'everybody' with access to the materials needed (like a computer and an internet connection), the skills, experience and time…  While 'the crowd' may carry connotations of 'the mob', in 'Digital Curiosities: Resource Creation Via Amateur Digitisation', Melissa Terras (@melissaterras) points out that many 'amateur' content creators are 'extremely self motivated, enthusiastic, and dedicated' and test the boundaries between 'between definitions of amateur and professional, work and hobby, independent and institutional' and quotes Leadbeater and Miller's 'The Pro-Am Revolution' on people who pursue an activity 'as an amateur, mainly for the love of it, but sets a professional standard'.

There's more and more talk of 'community-sourcing' in cultural heritage, and it's a useful distinction but it also masks the fact that nearly all crowdsourcing projects in cultural heritage involve a community rather than a crowd, whether they're the traditional 'enthusiasts' or 'volunteers', citizen historians, engaged audiences, whatever.  That said, Amy Sample Ward has a diagram that's quite useful for planning how to work with different groups. It puts the 'crowd' (people you don't know), 'network' (the community of your community) and 'community' (people with a relationship to your organisation) in different rings based on their closeness to you.

'The crowd' is differentiated not just by their relationship to your organisation, or by their skills and abilities, but their motivation for participating is also important – some people participate in crowdsourcing projects for altruistic reasons, others because doing so furthers their own goals.

I'm worried about about crowdsourcing because…

…isn't letting the public in like that just asking for trouble?

@lottebelice said she'd heard people worry that 'people are highly likely to troll and put in bad data/content/etc on purpose' – but this rarely happens. People worried about this with user-generated content, too, and while kids in galleries delight in leaving rude messages about each other, it's rare online.

It's much more likely that people will mistakenly add bad data, but a good crowdsourcing project should build any necessary data validation into the project. Besides, there are generally much more interesting places to troll than a cultural heritage site.

And as Matt Popke pointed out in a comment, 'When you have thousands of people contributing to an entry you have that many more pairs of eyes watching it. It's like having several hundred editors and fact-checkers. Not all of them are experts, but not all of them have to be. The crowd is effectively self-policing because when someone trolls an entry, somebody else is sure to notice it, and they're just as likely to fix it or report the issue'.  If you're really worried about this, an earlier post on Designing for participatory projects: emergent best practice' has some other tips.

 …doesn't crowdsourcing take advantage of people?

XKCD on the ethics of commercial crowdsourcing

Sadly, yes, some of the activities that are labelled 'crowdsourcing' do. Design competitions that expect lots of people to produce full designs and pay a pittance (if anything) to the winner are rightly hated. (See antispec.com for more and a good list of links).

But in cultural heritage, no. Museums, galleries, libraries, archives and academic projects are in the fortunate position of having interesting work that involves an element of social good, and they also have hugely varied work, from microtasks to co-curated research projects. Crowdsourcing is part of a long tradition of volunteering and altruistic participation, and to quote Owens again, 'Crowdsourcing is a concept that was invented and defined in the business world and it is important that we recast it and think through what changes when we bring it into cultural heritage.'

[Update, May 2013: it turns out museums aren't immune from the dangers of design competitions and spec work: I've written On the trickiness of crowdsourcing competitions to draw some lessons from the Sydney Design competition kerfuffle.]

Anyway, crowdsourcing won't usually work if it's not done right. From A Crowd Without Community – Be Wary of the Mob:

"when you treat a crowd as disposable and anonymous, you prevent them from achieving their maximum ability. Disposable crowds create disposable output. Simply put: crowds need a sense of identity and community to achieve their potential."

…crowdsourcing can't be used for academic work

Reasons given include 'humanists don't like to share their knowledge' with just anyone. And it's possible that they don't, but as projects like Transcribe Bentham and Trove show, academics and other researchers will share the work that helps produce that knowledge. (This is also something I'm examining in my PhD. I'll post some early findings after the Digital Humanities 2012 conference in July).

Looking beyond transcription and other forms of digitisation, it's worth checking out Prism, 'a digital tool for generating crowd-sourced interpretations of texts'.

…it steals jobs

Once upon a time, people starting a career in academia or cultural heritage could get jobs as digitisation assistants, or they could work on a scholarly edition. Sadly, that's not the case now, but that's probably more to do with year upon year of funding cuts. Blame the bankers, not the crowdsourcers.

The good news? Crowdsourcing projects can create jobs – participatory projects need someone to act as community liaison, to write the updates that demonstrate the impact of crowdsourced contributions, to explain the research value of the project, to help people integrate it into teaching, to organise challenges and editathons and more.

What isn't crowdsourcing?

…'the wisdom of the crowds'?

Which is not just another way of saying 'crowd psychology', either (another common furphy). As Wikipedia puts it, 'the wisdom of the crowds' is based on 'diverse collections of independently-deciding individuals'. Handily, Trevor Owens has just written a post addressing the topic: Human Computation and Wisdom of Crowds in Cultural Heritage.

…user-generated content

So what's the difference between crowdsourcing and user-generated content? The lines are blurry, but crowdsourcing is inherently productive – the point is to get a job done, whether that's identifying people or things, creating content or digitising material.

Conversely, the value of user-generated content lies in the act of creating it rather than in the content itself – for example, museums might value the engagement in a visitor thinking about a subject or object and forming a response to it in order to comment on it. Once posted it might be displayed as a comment or counted as a statistic somewhere but usually that's as far as it goes.

And @sherah1918 pointed out, there's a difference between asking for assistance with tasks and asking for feedback or comments: 'A comment book or a blog w/comments isn't crowdsourcing to me … nor is asking ppl to share a story on a web form. That is a diff appr to collecting & saving personal histories, oral histories'.

…other things that aren't crowdsourcing:

[Heading inspired by Sheila Brennan @sherah1918]

  • Crowdfunding (it's often just asking for micro-donations, though it seems that successful crowdfunding projects have a significant public engagement component, which brings them closer to the concerns of cultural heritage organisations. It's also not that new. See Seventeenth-century crowd funding for one example.)
  • Data-mining social media and other content (though I've heard this called 'passive' or 'implict' crowdsourcing)
  • Human computation (though it might be combined with crowdsourcing)
  • Collective intelligence (though it might also be combined with crowdsourcing)
  • General calls for content, help or participation (see 'user-generated content') or vaguely asking people what they think about an idea. Asking for feedback is not crowdsourcing. Asking for help with your homework isn't crowdsourcing, as it only benefits you.
  • Buzzwords applied to marketing online. And as @emmclean said, "I think many (esp mkting) see "crowdsourcing" as they do "viral" – just happens if you throw money at it. NO!!! Must be great idea" – it must make sense as a crowdsourced task.

Ok, so what's different about crowdsourcing in cultural heritage?

For a start, the process is as valuable as the result. Owens has a great post on this, Crowdsourcing Cultural Heritage: The Objectives Are Upside Down, where he says:

'The process of crowdsourcing projects fulfills the mission of digital collections better than the resulting searches… Far better than being an instrument for generating data that we can use to get our collections more used it is actually the single greatest advancement in getting people using and interacting with our collections. … At its best, crowdsourcing is not about getting someone to do work for you, it is about offering your users the opportunity to participate in public memory … it is about providing meaningful ways for the public to enhance collections while more deeply engaging and exploring them'.

And as I've said elsewhere, ' playing [crowdsourcing] games with museum objects can create deeper engagement with collections while providing fun experiences for a range of audiences'. (For definitions of 'engagement' see The Culture and Sport Evidence (CASE) programme. (2011). Evidence of what works: evaluated projects to drive up engagement (PDF).)

What about cultural heritage and citizen science?

[This was written in 2012. I've kept it for historical reasons but think differently now.]

First, another definition. As Fiona Romeo writes, 'Citizen science projects use the time, abilities and energies of a distributed community of amateurs to analyse scientific data. In doing so, such projects further both science itself and the public understanding of science'. As Romeo points out in a different post, 'All citizen science projects start with well-defined tasks that answer a real research question', while citizen history projects rarely if ever seem to be based around specific research questions but are aimed more generally at providing data for exploration. Process vs product?

I'm still thinking through the differences between citizen science and citizen history, particularly where they meet in historical projects like Old Weather. Both citizen science and citizen history achieve some sort of engagement with the mindset and work of the equivalent professional occupations, but are the traditional differences between scientific and humanistic enquiry apparent in crowdsourcing projects? Are tools developed for citizen science suitable for citizen history? Does it make a difference that it's easier to take a new interest in history further without a big investment in learning and access to equipment?

I have a feeling that 'citizen science' projects are often more focused on the production of data as accurately and efficiently as possible, and 'citizen history' projects end up being as much about engaging people with the content as it is about content production. But I'm very open to challenges on this…

What kind of cultural heritage stuff can be crowdsourced?

I wrote this list of 'Activity types and data generated' over a year ago for my Masters dissertation on crowdsourcing games for museums and a subsequent paper for Museums and the Web 2011, Playing with Difficult Objects – Game Designs to Improve Museum Collections (which also lists validation types and requirements).  This version should be read in the light of discussion about the difference between crowdsourcing and user-generated content and in the context of things people can do with museums and with games, but it'll do for now:

Activity Data generated
Tagging (e.g. steve.museum, Brooklyn Museum Tag! You're It; variations include two-player 'tag agreement' games like Waisda?, extensions such as guessing games e.g. GWAP ESP Game, Verbosity, Tiltfactor Guess What?; structured tagging/categorisation e.g. GWAP Verbosity, Tiltfactor Cattegory) Tags; folksonomies; multilingual term equivalents; structured tags (e.g. 'looks like', 'is used for', 'is a type of').
Debunking (e.g. flagging content for review and/or researching and providing corrections). Flagged dubious content; corrected data.
Recording a personal story Oral histories; contextualising detail; eyewitness accounts.
Linking (e.g. linking objects with other objects, objects to subject authorities, objects to related media or websites; e.g. MMG Donald). Relationship data; contextualising detail; information on history, workings and use of objects; illustrative examples.
Stating preferences (e.g. choosing between two objects e.g. GWAP Matchin; voting on or 'liking' content). Preference data; subsets of 'highlight' objects; 'interestingness' values for content or objects for different audiences. May also provide information on reason for choice.
Categorising (e.g. applying structured labels to a group of objects, collecting sets of objects or guessing the label for or relationship between presented set of objects). Relationship data; preference data; insight into audience mental models; group labels.
Creative responses (e.g. write an interesting fake history for a known object or purpose of a mystery object.) Relevance; interestingness; ability to act as social object; insight into common misconceptions.

You can also divide crowdsourcing projects into 'macro' and 'micro' tasks – giving people a goal and letting them solve it as they prefer, vs small, well-defined pieces of work, as in the 'Umbrella of Crowdsourcing' at The Daily Crowdsource and there's a fair bit of academic literature on other ways of categorising and describing crowdsourcing.

Using crowdsourcing to manage crowdsourcing

There's also a growing body of literature on ecosystems of crowdsourcing activities, where different tasks and platforms target different stages of the process.  A great example is Brooklyn Museum’s ‘Freeze Tag!’, a game that cleans up data added in their tagging game. An ecosystem of linked activities (or games) can maximise the benefits of a diverse audience by providing a range of activities designed for different types of participant skills, knowledge, experience and motivations; and can encompass different levels of participation from liking, to tagging, finding facts and links.

A participatory ecosystem can also resolve some of the difficulties around validating specialist tags or long-form, more subjective content by circulating content between activities for validation and ranking for correctness, 'interestingness' (etc) by other players (see for example the 'Contributed data lifecycle' diagram on my MW2011 paper or the 'Digital Content Life Cycle' for crowdsourcing in Oomen and Aroyo's paper below). As Nina Simon said in The Participatory Museum, 'By making it easy to create content but impossible to sort or prioritize it, many cultural institutions end up with what they fear most: a jumbled mass of low-quality content'.  Crowdsourcing the improvement of cultural heritage data would also make possible non-crowdsourcing engagement projects that need better content to be viable.

See also Raddick, MJ, and Georgia Bracey. 2009. “Citizen Science: Status and Research Directions for the Coming Decade” on bridging between old and new citizen science projects to aid volunteer retention, and Nov, Oded, Ofer Arazy, and David Anderson. 2011. “Dusting for Science: Motivation and Participation of Digital Citizen Science Volunteers” on creating 'dynamic contribution environments that allow volunteers to start contributing at lower-level granularity tasks, and gradually progress to more demanding tasks and responsibilities'.

What does the future of crowdsourcing hold?

Platforms aimed at bootstrapping projects – that is, getting new projects up and running as quickly and as painlessly as possible – seem to be the next big thing. Designing tasks and interfaces suitable for mobile and tablets will allow even more of us to help out while killing time. There's also a lot of work on the integration of machine learning and human computation; my post 'Helping us fly? Machine learning and crowdsourcing' has more on this.

Find out how crowdsourcing in cultural heritage works by exploring projects

Spend a few minutes with some of the projects listed in Looking for (crowdsourcing) love in all the right places to really understand how and why people participate in cultural heritage crowdsourcing.

Where can I find out more? (AKA, a reading list in disguise)

There's a lot of academic literature on all kinds of aspects of crowdsourcing, but I've gone for sources that are accessible both intellectually and in terms of licensing. If a key reference isn't there, it might be because I can't find a pre-print or whatever outside a paywall – let me know if you know of one!

9781472410221Liked this post? Buy the book! 'Crowdsourcing Our Cultural Heritage' is available through Ashgate or your favourite bookseller…

Thanks, and over to you!

Thanks to everyone who responded to my call for their favourite 'misconceptions and apprehensions about crowdsourcing (esp in history and cultural heritage)', and to those who inspired this post in the first place by asking questions in various places about the negative side of crowdsourcing.  I'll update the post as I hear of more, so let me know your favourites.  I'll also keep adding links and resources as I hear of them.

You might also be interested in: Notes from 'Crowdsourcing in the Arts and Humanities' and various crowdsourcing classes and workshops I've run over the past few years.

Museums and the audience comments paradox

I was at the Imperial War Museum for an advisory board meeting for the Social Interpretation project recently, and had a chance to reflect on my experiences with previous audience participation projects.  As Claire Ross summarised it, the Social Interpretation project is asking: does applying social media models to collections successfully increase engagement and reach?  And what forms of moderation work in that environment – can the audience be trusted to behave appropriately?

One topic for discussion yesterday was whether the museum should do some 'gardening' on the comments.  Participation rates are relatively high but some of the comments are nonsense ('asdf'), repetitive (thousands of variants of 'Cool' or 'sad') or off-topic ('I like the museum') – a pattern probably common to many museum 'have your say' kiosks.  Gardening could involve 'pruning' out comments that were not directly relevant to the question asked in the interactive, or finding ways to surface the interesting comments.  While there are models available in other sectors (e.g. newspapers), I'm excited by the possibility that the Social Interpretation project might have a chance to address this issue for museums.

A big design challenge for high-traffic 'have your say' interactives is providing a quality experience for the audience who is reading comments – they shouldn't have to wade through screens of repeated, vacuous or rude comments to find the gems – while appropriately respecting the contribution and personal engagement of the person who left the comment.

In the spirit of 'have your say', what do you think the solution might be?  What have you tried (successfully or not) in your own projects, or seen working well elsewhere?

Update: the Social Interpretation have posted I iz in ur xhibition trolling ur comments:

"One of the most discussed issues was about what we have termed ‘gardening comments’ but to put it bluntly it’s more a case of should we be ‘curating the visitor voice’ in order to improve the visitor experience? It’s a difficult question to deal with… 

We are at the stage where we really do want to respect the commenter, but also want to give other readers a high value experience. It’s a question of how we do that, and will it significantly change the project?"

If you found this post, you might also be interested in Notes from 'The Shape of Things: New and emerging technology-enabled models of participation through VGC'.

Update, March 2014: I've just been reading a journal article on 'Normative Influences on Thoughtful Online Participation'. The authors set out to test this hypothesis:

'Individuals exposed to highly thoughtful behavior from others will be more thoughtful in their own online comment contributions than individuals exposed to behavior exhibiting a low degree of thoughtfulness.' 

Thoughtful comments were defined by the number of words, how many seconds it took to write them, and how much of the content was relevant to the issue discussed in the original post. And the results? 'We found significant effects of social norm on all three measures related to participants’ commenting behavior. Relative to the low thoughtfulness condition, participants in the high thoughtfulness condition contributed longer comments, spent more time writing them, and presented more issue-relevant thoughts.' To me, this suggests that it's worth finding ways to highlight the more thoughtful comments (and keeping pulling out those 'asdf' weeds) in an interactive as this may encourage other thoughtful comments in turn.

Reference: Sukumaran, Abhay, Stephanie Vezich, Melanie McHugh, and Clifford Nass. “Normative Influences on Thoughtful Online Participation.” In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, 3401–10. Vancouver, BC, Canada: ACM, 2011. http://dl.acm.org/citation.cfm?id=1979450.

Designing for participatory projects: emergent best practice, getting discussion started

I was invited over to New Zealand (from Australia) recently to talk at Te Papa in Wellington and the Auckland Museum.  After the talks I was asked if I could share some of my notes on design for participatory projects and for planning for the impact of participatory projects on museums.  Each museum has a copy of my slides, but I thought I'd share the final points here rather than by email, and take the opportunity to share some possible workshop activities to help museums plan audience participation around its core goals.

Both talks started by problematising the definition of a 'museum website' – it doesn't work to think of your 'museum website' as purely stuff that lives under your domain name when it's now it's also the social media accounts under your brand, your games and mobile apps, and maybe also your objects and content on Google Art Project or even your content in a student’s Tumblr.  The talks were written to respond to the particular context of each museum so they varied from there, but each ended up with these points.  The sharp-eyed among you might notice that they're a continuation of ideas I first shared in my Europeana Tech keynote: Open for engagement: GLAM audiences and digital participation.  The second set are particularly aimed at helping museums think about how to market participatory projects and sustain them over the longer term by making them more visible in the museum as a whole.

Best practice in participatory project design

  • Have an answer to 'Why would someone spend precious time on your project?'
  • Be inspired by things people love
  • Design for the audience you want
  • Make it a joy to participate
  • Don't add unnecessary friction, barriers (e.g. don't add sign-up forms if you don’t really need them, or try using lazy registration if you really must make users create accounts)
  • Show how much you value contributions (don't just tell people you value their work)
  • Validate procrastination – offer the opportunity to make a difference by providing meaningful work
  • Provide an easy start and scaffolded tasks (see e.g. Nina Simon's Self-Expression is Overrated: Better Constraints Make Better Participatory Experiences)
  • Let audiences help manage problems – let them know which behaviours are acceptable and empower them to keep the place tidy
  • Test with users; iterate; polish

Best practice within your museum

  • Fish where the fish are – find the spaces where people are already engaging with similar content and see how you can slot in, don't expect people to find their way to you unless you have something they can’t find anywhere else
  • Allow for community management resources – you’ll need some outreach to existing online and offline communities to encourage participation, some moderation and just a general sense that the site hasn’t been abandoned. If you can’t provide this for the life of the project, you might need to question why you’re doing it.
  • Decide where it's ok to lose control. Try letting go… you may find audiences you didn't expect, or people may make use of your content in ways you never imagined. Watch and learn and tweak in response – this is a good reason to design in iterations, and to go into public or invited-beta earlier rather than later. 
  • Realistically assess fears, decide acceptable levels of risk. Usually fears can be turned into design requirements, they’re rarely show-stoppers.
  • Have a clear objective, ideally tied to your museum’s mission. Make sure the point of the project is also clear to your audience.
  • Put the audience needs first. You’re asking people to give up their time and life experience, so make sure the experience respects this. Think carefully before sacrificing engagement to gain efficiency.
  • Know how to measure success
  • Plan to make the online activity visible in the organisation and in the museum. Displaying online content in the museum is a great way to show how much you value it, as well as marketing the project to potential contributors.  Working out how you can share the results with the rest of the organization helps everyone understand how much potential there is, and helps make online visitors ‘real’.
  • Have an exit strategy – staff leave, services fold or change their T&Cs

I'd love to know what you think – what have I missed?  [Update: for some useful background on the organisational challenges many museums face when engaging with technology, check out Collections Access and the use of Digital Technology (pdf).]

More on designing museum projects for audience participation

I prepared this activity for one of the museums, but on the day the discussion after my talk went on so long that we didn't need to use a formal structure to get people talking. In the spirit of openness, I thought I'd share it. If you try it in your organisation, let me know how it goes!

The structure – exploratory idea generation followed by convergence and verification – was loosely based on the 'creativity workshops' developed by City University's Centre for Creativity (e.g. the RESCUE creativity workshops discussed in Use and Influence of Creative Ideas and Requirements for a Work-Integrated Learning System).  It's designed to be a hackday-like creative activity for non-programmers.

In small groups…

  • Pick two strategic priorities or organisational goals…
  • In 5 minutes: generate as many ideas as possible
  • In 2 minutes: pick one idea to develop further

Ideas can include in-gallery and in-person activity; they must include at least two departments and some digital component.

Developing your idea…
Ideas can include in-gallery and in-person activity; they must include at least two departments

  • You have x minutes to develop your idea
  • You have 2 minutes each to report back. Include: which previous museum projects provide relevant lessons? How will you market it? How will it change the lives of its target audience? How will it change the museum?
  • How will you alleviate potential risks?  How will you maximise potential benefits?
  • You have x minutes for general discussion. How can you build on the ideas you've heard?

For bonus points…

These discussion points were written for another museum, but they might be useful for other organisations thinking about audience participation and online collections:

What are the museum’s goals in engaging audiences with collections online?

  • What does success look like?
  • How will it change the museum?
  • Which past projects provide useful lessons?

How can the whole organisation be involved in supporting online conversations?

  • What are the barriers?
  • What small, sustainable steps can be taken?
  • Where are online contributions visible in the museum?

My Europeana Tech keynote: Open for engagement: GLAM audiences and digital participation

This is a slightly abridged version of my notes for my keynote, 'Open for engagement: GLAM audiences and digital participation' at EuropeanaTech (#etech11) in Vienna in October 2011.

Introduction 
I'm really excited about being here to talk about some of my favourite things with you. I think helping people appreciate cultural heritage is one of the best jobs in the world so I feel lucky to be here with people working toward the same goal.

This is a chance to remind ourselves why we should get audiences participating digitally – how does it benefit both GLAM (galleries, libraries, archives, museums) and their audiences? I'm going to take you through some examples of digital participation and explain why I think they're useful case studies. I'll finish by summarising what we can learn from those case studies, looking for tips you can take back to your organisations. Hopefully we'll have time for a few questions or some discussion.

Why enable participation?
Isn't it easier to just keep doing what we're already doing? Maybe not – here are some problems your GLAM organisation might be facing…

You need to think digitally to enable participation at scale – to reach not tens or hundreds, but thousands or hundreds of thousands of people. As cultural heritage organisations, we have lots of experience with access and participation at reference desks and in galleries. We are good at creating experiences to engage, delight, and educate in person, but these are limited by the number of staff required, the materiality of the objects or documents, the size of a venue, its location and opening hours. We're still learning how to translate those brilliant participative experiences into the digital domain…

Collections are big, resources are small. In most cases we're still digitising catalogue records, let alone taking images and writing beautiful contextualised interpretative material for our collections. We'll be at it for centuries if we try to do it alone…

What's more, it's not enough for content to be online – it has to be findable. Our digitised content is still not very discoverable in search engines – which means it's effectively invisible to most potential audiences. We need better content to help search engines find the stuff we've put so much work into putting online. For example, I wanted to use Europeana images to illustrate my slides, but I had trouble finding images to match my ideas – but if other people had tagged them with words like 'happiness', 'excitement', 'crowds', I might have been able to find what I needed.

User-contributed content can help bridge the 'semantic gap' between the language used in catalogues and the language that most people would use to look for content.

Even when our content is found by our audiences, it's not always very accessible without information about the significance, and cultural and historical context of the item. Further, in Europeana's case, there's a gap between the many languages of the user community and the catalogue metadata; as well as gaps between historical and contemporary language. Sadly, at the moment, many records lack enough context for a non-expert to have a meaningful experience with them.

Why support participation?
So, those are some of the problems we're looking for solve… what are the benefits of digital participation?
Firstly, the benefits to organisations

Engagement and participation is often part of your core mission.

inspire, passion, educate, enhance, promote preserve, record, access, learn, discover, use, memory, culture, conservation, innovation

I had a look at some mission statements from various museums, libraries, and archives, and these are the words that frequently occurred. The benefits of audience participation are both tangible and intangible, and exactly how they relate to your mission (and can be measured in relation to it) depends on the organisation. And don't forget that access may not be enough if your content isn't also discoverable and engaging.

Participation can increase traffic. It's pretty simple – if content is more discoverable, more people will discover it. If audiences can actively participate, they'll engage with your collections for longer, and return more often. They may even turn into physical visitors or buy something online…

Turn audiences into advocates – there are many people who forget that GLAMs even exist once they've left school – but these are often the people we can reach with digital projects. When people directly benefit from your resources, they know why your organisation is important. You're no longer dusty old stuff in boxes, you're their history, part of the story of how their lives came to be and how their future is formed.  When people have a great experience with you, they become fans. When you encourage people to participate in meaningful work, they gain a sense of ownership and pride. These intangible outcomes can be as important as the content created through audience participation.  It's a chance to let people see the full complexity of what you do, how much work goes into providing access and interpretation; understand that what they see on the shelves or in the galleries is the tip of the iceberg..

There are more experts outside your GLAM than within. Participatory projects let you access external knowledge.  This knowledge can include the experience of using, repairing or building an object; memories of the events or places you've recorded; or it may be specialist knowledge they've built through their own research. Let them share their knowledge with you, and through you, with your audiences.

Finally, the rest of the world is moving from broadcast to dialogue and interaction. If you spend time around kids, you may have seen them interact with old-fashioned screens – for them, an interface you can only look at is broken.

Benefits to audience

It's all very well saying participation creates deeper engagement, but rather than tell you again, I'd rather show you with a quick thought experiment.

First I want you to imagine taking a photo of an object in a museum. Ok – so, how many times do you really go back and look at that photo? How much do you remember about that object? Do you find yourself thinking about it later? Do you ever have a conversation with friends about it?

Now I want you to imagine sketching the object, perhaps at this handy sketching station in the Musée des Beaux-Arts de Dijon.

As you draw, you'll find yourself engaging with the particular materiality of the object – the details of its construction, the way time has affected it. You may start wondering about the intention of the creators, what it was like to use it or encounter it in everyday life. In having an active relationship with that object, you've engaged more deeply, perhaps even changed a little as a result. New questions have been raised that you may find yourself pondering, and may even decide to find out more, and start your own research, or share your feelings with others.

Perhaps surprisingly, even the act of tagging an object has a similar effect, because you have to pay it some attention to say something about it…

A big benefit for audiences is that participation is rewarding. There are many reasons why, but these are some I think are relevant to participation. Games researcher Jane McGonigal (Gaming the future of museumssays people crave:

1. satisfying work to do
2. the experience of being good at something
3. time spent with people we like
4. the chance to be a part of something bigger

Participation in digital cultural heritage projects can meet all those needs.

Types of participation

The Center for Advancement of Informal Science Education came up with these forms of public participation in science research. Nina Simon of the Museum 2.0 blog mapped them to museums and added 'co-option'; I've included 'platform'.

  • Contributory – Most GLAM user-generated content projects. Designed by the organisation, the public contributes data.
  • Collaborative – the public may be active partners in some decisions, but the project is lead by the organisation
  • Co-creative – all partners define goals and make decisions together
  • Platform – organisation as venue or host for other activity.

It's also important to remember that there are some types of participation where the value lies mostly in the effect of the act of creation for the individual – for example, most commenting doesn't add much to my experience of the thing commented on. However, sometimes there's also value more widely – for example, when someone comments and includes a new fact or interesting personal story. Taking this further, participatory projects can be designed so that each contribution helps meet a defined goal. Crowdsourcing involves designing carefully scaffolded tasks so that the general public can contribute to a shared goal. Crowdsourcing in cultural heritage is probably most often contributory rather than collaborative or co-creative.

Case studies
I've chosen two established examples and two experimental ones to demonstrate how established digital participation is, and also where it's going…

Flickr Commons – I'm sure you've all probably heard of this, but it's a great reminder of how effective simply sharing content in places where people hang out can be. The first tip: go fishing where the fish are biting. Find the digital spaces where people are already engaging with similar content.

Example page: [Sylvia Sweets Tea Room, corner of School and Main streets, Brockton, Mass.].  You can see from the number of views, comments, tags, favourites and notes that organisations are still finding much higher levels of discoverability, traffic and user contributions on the Commons than they'd ever get on their own, individual sites. It's also a nice example of the public identifying a location, and there are wonderful personal recollections and family histories in the comments below.

Trove – crowdsourcing OCR correction.  Tasks like OCR correction that require judgement or complicated visual processing are perfect for crowdsourcing.

Crowdsourcing can solve real problems – helping scientists identify galaxies and proteins that could save lives, or providing data about climate change through history. In this example, crowdsourcing is helping correct optical character recognition (OCR) errors. In the example here, the correction is subtle, but as someone from the location described, I can tell you that the transcription now makes a lot more sense… And making that correction felt good.

According to the National Library of Australia, by February 2011 they had '20,000+ people helping out and 30 million lines of text had been corrected during the last 2 years'. This is a well-designed interface. Their clear 'call to action' – 'fix this text' – is simple and located right where it needs to be.  Another tip: you don't need to register, but you can if you want to track your progress. Registration isn't a barrier, and it's presented as a benefit to the audience, not the organisation. They've also got a forum as a platform for conversation between participants.

So, crowdsourcing is great. But as crowdsourcing gets more popular, you will be competing for 'participation bandwidth' with other participatory and crowdsourcing projects – people will be deciding whether to work with your site or something else that meets their needs… What to do?

Well, it turns out that crowdsourcing games can act as 'participation engines'…

[I then talked about 'a small tagging game I researched, designed and made in my evenings and weekends, so that you can see the potential for crowdsourcing games even for GLAMs that don't have a lot of resources' – if you're curious, it's probably easiest to check out the slides at http://www.slideshare.net/miaridge/everyone-wins-crowdsourcing-games-and-museums alongside the video at http://vimeo.com/26858316].

Because crowdsourcing games can be more accessible to the general public, they can also increase the number of overall contributors, as well as encouraging each contributor to stay for longer, do more work, engage more deeply. Crowdsourcing games can be much more productive than a non-game interface by encouraging people to spend more time and play with more content. If games not suitable for your audience, you can adopt some of the characteristics of games – clear initial tasks to start with and a sense of the rules of the game, good feedback on the results of player actions towards a goal, mastering new skills and providing interesting problems to solve…

Continuing the [Europeana Tech] theme of openness, this project was only possible because the Science Museum (UK) and the Powerhouse Museum had APIs into their object records – I was able to create a game that united their astronomy objects without ever having to negotiate a partnership or licensing agreement.

Oramics – co-creation (and GLAM as platform).  My final example is something I worked on just before I left the Science Museum but I make the caveat that I can't claim any credit for all the work done since, and I haven't seen any internal evaluation on the project.

The Oramics project was a conscious experiment in co-curation and public history, part of a wider programme of research. This is the Oramics machine. It's a difficult object to interpret – it's a hand-built synthesiser, and not much to look at – it's all about how it sounded, but it's too fragile to restore to working order. So the museum needed help interpreting the object, in understanding how to explain its significance and market it to new audiences. They tried a few different things in this project… They worked with young people from the National Youth Theatre who met museum staff to learn about the people who invented and built the machine, and they visited the object store to see the machine. They worked with developers to make an app to recreate the sounds of the synthesiser so that people could make new music with it. They also worked with a group of co-curators recruited online to help make it interesting to general visitors as well as music fans – the original call to action was something like 'we have an amazing object we need to bring to life, and six empty cases – help us fill them!'.

While the main outputs of all this activity are pretty traditional – a performance event, an exhibition – it's also been the catalyst for the creation of an ad hoc online community and conversations on Facebook and blogs.

As Clay Shirky told the Smithsonian 2.0 workshop in 2009, it's possible that "the artefact itself has created the surface to which the people adhere. … Every artefact is a latent community". It's nice to think we're finally getting to that point.

Best practice tips
So what do you need to think about to design a participatory project?

  • Have an answer to 'Why would someone spend precious time on your project?'
  • Be inspired by things people love
  • Design for the audience you want
  • Make participating pleasurable
  • Don't add unnecessary friction, barriers
  • Show how much you value contributions
  • Validate procrastination – offer the opportunity to make a difference, and show, don't tell, how it's making a difference
  • Make it easy to start participating, design scaffolded tasks to keep people going
  • Let audiences help manage problems
  • Test with users; iterate; polish
  • Empower audience to keep the place tidy – let them know what's acceptable and what's discouraged and how they can help.

Best practice within your GLAM
How can your organisation make the most of the opportunities digital participation provides?

  • Have a clear objective
  • Know how to measure success
  • Allow for community management resources
  • Realistically assess fears, decide acceptable risk
  • Put the audience's needs first. You need a balance between the task want to achieve, the skills and knowledge of audience and the content you have to work with.
  • Fish where the fish are – find the spaces where people are already engaging with similar content and see how you can slot in, don't expect people to find their way to you.
  • Decide where it's ok to lose control – let go… you may find audiences you didn't expect, or people may make use your content in ways you never imagined. Watch and learn – another reason to iterate and go into public beta earlier rather than later.
  • Open data – let people make new things with your content. Bad people will do it anyway, but by not having open data, you're preventing exactly the people you want to work with from doing anything with your data. Unclear or closed licenses are the biggest barrier that friendly hackers and developers raise with me when I ask about cultural heritage data…

In a 2008 post about museum-as-platform, Nina Simon says it's about moving from controlling everything to providing expertise; learning to change from content provider to platform. [More recently, Rob Stein posted about participatory culture and the subtle differences between authoritarian and authoritative approaches.]

Conclusion
Perhaps most important of all – enjoy experiencing your collections through new eyes!

Notes on 'User Generated Content' session, Open Culture Conference 2010

My notes from the 'user generated content' parallel track on first day of the Open Culture 2010 conference. The session started with brief presentations by panellists, then group discussions at various tables on questions suggested by the organisers. These notes are quite rough, and of course any mistakes are mine. I haven't had a chance to look for the speakers' slides yet so inevitably some bits are missing, and I can only report the discussion at the table I was at in the break-out session. I've also blogged my notes from the plenary session of the Open Culture 2010 conference.

User-generated content session, Open Culture, Europeana – the benefits and challenges of UGC.
Kevin Sumption, User-generated content, a MUST DO for cultural institutions
His background – originally a curator of computer sciences. One of first projects he worked on at Powerhouse was D*Hub which presented design collections from V&A, Brooklyn Museum and Powerhouse Museum – it was for curators but also for general public with an interest in design. Been the source of innovation. Editorial crowd-sourcing approach and social tagging, about 8 years ago.

Two years ago he moved to National Maritime Museum, Royal Observatory, Greenwich. One of the first things they did was get involved with Flickr Commons – get historic photographs into public domain, get people involved in tagging. c1000 records in there. General public have been able to identify some images as Adam Villiers images – specialists help provide attribution for the photographer. Only for tens of records of the 000s but was a good introduction to power of UGC.

Building hybrid exhibition experiences – astronomy photographer of the year – competition on Flickr with real world exhibition for the winners of the competition. 'Blog' with 2000 amateur astronomers, 50 posts a day. Through power of Flickr has become a significant competition and brand in two years.

Joined citizen science consortia. Galaxy Zoo. Brainchild of Oxford – getting public engaged with real science online. Solar Stormwatch c 3000 people analysing and using the data. Many people who get involved gave up science in high school… but people are getting re-engaged with science *and* making meaningful contributions.

Old Weather – helping solve real-world problems with crowdsourcing. Launched two months ago.
Passion for UGC is based around where projects can join very carefully considered consortia, bringing historical datasets with real scientific problems. Can bring large interested public to the project. Many of the public are reconnecting with historical subject matter or sciences.

Judith Bensa-Moortgat, Nationaal Archief, Netherlands, Images for the Future project
Photo collection of more than 1 million photos. Images for the future project aims to save audio-visual heritage through digitisation and conservation of 1.2 million photos.

Once digitised, they optimise by adding metadata and context. Have own documentalists who can add metadata, but it would take years to go through it all. So decided to try using online community to help enrich photo collections. Using existing platforms like Wikipedia, Flickr, Open Street map, they aim to retrieve contextual info generated by the communities.  They donated political portraits to Wikimedia Commons and within three weeks more than half had been linked to relevant articles.

Their experiences with Flickr Commons – they joined in 2008. Main goal was to see if community would enrich their photos with comments and tags. In two weeks, they had 400,000 page views for 400 photos, including peaks when on Dutch TV news. In six months, they had 800 photos with over 1 million views. In Oct 2010, they are averaging 100,000 page views a month; 3 million overall.

But what about comments etc? Divided them into categories of comments [with percentage of overall contributions]:

  • factual info about location, period, people 5%; 
  • link to other sources eg Wikipedia 5%; 
  • personal stories/memories (e.g. someone in image was recognised); 
  • moral discussions; 
  • aesthetical discussions; 
  • translations.

The first two are most important for them.
13,000 tags in many languages (unique tags or total?).
10% of the contributed UGC was useful for contextualisation; tags ensure accessibility [discoverability?] on the web; increased (international) visibility. [Obviously the figures will vary for different projects, depending on what the original intent of the project was]

The issues she'd like to discuss are – copyright, moderation, platforms, community.

Mette Bom, 1001 Stories about Denmark
Story of the day is one of the 1001 stories. It's a website about the history and culture of Denmark. The stories have themes, are connected to a timeline.  Started with 50 themes, 180 expert writers writing the 1001 stories, now it's up to the public to comment and write their own stories. Broad definition of what heritage is – from oldest settlement to the 'porn street' – they wanted to expand the definition of heritage.

Target audiences – tourists going to those places; local dedicated experts who have knowledge to contribute. Wanted to take Danish heritage out of museums.

They've created the main website, mobile apps, widget for other sites, web service.  Launched in May 2010.  20,000 monthly users. 147 new places added, 1500 pictures added.

Main challenges – how to keep users coming back? 85% new, 15% repeat visitors (ok as aimed at tourists but would like more comments). How to keep press interested and get media coverage? Had a good buzz at the start cos of the celebrities. How to define participation? Is it enough to just be a visitor?

Johan Oomen, Netherlands Institute for Sound and Vision, Vrij Uni Amsterdam. Participatory Heritage: the case of the Waisda? video labelling game.
They're using game mechanisms to get people to help them catalogue content. [sounds familiar!]
'In the end, the crowd still rules'.
. Tagging is a good way to facilitate time-based annotation [i.e. tag what's on the screen at different times]

Goal of game is consensus between players. Best example in heritage is steve.museum; much of the thinking about using tagging as a game came from Games with a Purpose (gwap.com).  Basic rule – players score points when their tag exactly matches the tag entered by another within 10 seconds. Other scoring mechanisms.  Lots of channels with images continuously playing.

Linking it to twitter – shout out to friends to come join them playing.  Generating traffic – one of the main challenges. Altruistic message 'help the archive' 'improve access to collections' came out of research with users on messages that worked. Worked with existing communities.

Results, first six months – 44,362 pageviews. 340,000 tags to 604 items, 42,068 unique tags.
Matches – 42% of tags entered more than 2 times. Also looked at vocab (GTAA, Cornetto), 1/3 words were valid Dutch words, but only a few part of thesauruses.  Tags evaluated by documentalists. Documentary film 85% – tags were useful; for reality series (with less semantic density) tags less useful.

Now looking at how to present tags on the catalogue Powerhouse Museum style.  Experimenting with visualising terms, tag clouds when terms represented, also makes it easy to navigate within the video – would have been difficult to do with professional metadata.  Looking at 'tag gardening' – invite people to go back to their tags and click to confirm – e.g. show images with particular tags, get more points for doing it.

Future work – tag matching – synonyms and more specific terms – will get more points for more specific terms.

Panel overview by Costis Dallas, research fellow at Athena, assistant professor at Panteion University, Athens.
He wants to add a different dimension – user-generated content as it becomes an object for memory organisations. New body of resources emerging through these communication practices.
Also, we don't have a historiography anymore; memory resides in personal information devices.  Mashups, changes in information forms, complex composed information on social networks – these raise new problems for collecting – structural, legal, preservation in context, layered composition.  What do we need to do now in order to be able to make use of digital technologies in appropriate, meaningful ways in the future? New kinds of content, participatory curation are challenges for preservation.

Group discussion (breakout tables)
Discussion about how to attract users. [It wasn't defined whether it was how to attract specifically users who'll contribute content or just generally grow the audience and therefore grow the number of content creators within the usual proportions of levels of participation e.g. Nielsen, Forrester; I would also have liked to discussed how to encourage particular kinds of contributions, or to build architectures of participation that provided positive feedback to encourage deeper levels of participation.]

Discussion and conclusions included – go with the strengths of your collections e.g. if one particular audience or content-attracting theme emerges, go with it.  Norway has a national portal where people can add content. They held lots of workshops for possible content creators; made contact with specialist organisations [from which you can take the lesson that UGC doesn't happen in a vacuum, and that it helps to invest time and resources into enabling participants and soliciting content].  Recording living history.  Physical presence in gallery, at events, is important.  Go where audiences already are; use existing platforms.

Discussion about moderation included – once you have comments, how are they integrated back into collections and digital asset management systems?  What do you do about incorrect UGC displayed on a page?  Not an issue if you separate UGC from museum/authoritative content in the interface design.  In the discussion it turned out that Europeana doesn't have a definition of 'moderation'.  IMO, it should include community management, including acknowledging and thanking people for contributions (or rather, moderation is a subset of community management).  It also includes approving or reviewing and publishing content, dealing with corrections suggested by contributors, dealing with incorrect or offensive UGC, adding improved metadata back to collections repositories.

User-generated content and trust – British Library apparently has 'trusted communities' on their audio content – academic communities (by domain name?) and 'everyone else'.  Let other people report content to help weed out bad content.

Then we got onto a really interesting discussion of which country or culture's version of 'offensive' would be used in moderating content.  Having worked in the UK and the Netherlands, I know that what's considered a really rude swear word and what's common vocabulary is quite different in each country… but would there be any content left if you considered the lowest common standards for each country?  [Though thinking about it later, people manage to watch films and TV and popular music from other countries so I guess they can deal with different standards when it's in context.]  To take an extreme content example, a Nazi uniform as memorabilia is illegal in Germany (IIRC) but in the UK it's a fancy dress outfit for a member of the royal family.

Panel reporting back from various table discussions
Kevin's report – discussion varied but similar themes across the two tables. One – focus on the call to action, why should people participate, what's the motivation? How to encourage people to participate? Competitions suggested as one solution, media interest (especially sustained). Notion of core group who'll energise others. Small groups of highly motivated individuals and groups who can act as catalysts [how to recruit, reward, retain]. Use social media to help launch project.

1001 Danish Stories promotional video effectively showed how easy the process of contributing content was,  and that it doesn't have to to be perfect (the video includes celebrities working the camera [and also being a bit daggy, which I later realised was quite powerful – they weren't cool and aloof]).
Giving users something back – it's not a one-way process. Recognition is important. Immediacy too – if participating in a project, people want to see their contributions acknowledged quickly. Long approval processes lose people.
Removal of content – when different social, political backgrounds with different notions of censorship.

Mette's report – how to get users to contribute – answers mostly to take away the boundaries, give the users more credit than we otherwise tend to. We always think users will mess things up and experts will be embarrassed by user content but not the case. In 1001 they had experts correcting other experts. Trust users more, involve experts, ask users what they want. Show you appreciate users, have a dialouge, create community. Make it a part of life and environment of users. Find out who your users are.

Second group – how Europeana can use the content provided in all its forms. Could build web services to present content from different places, linking between different applications.
How to set up goals for user activity – didn't get a lot of answers but one possibility is to start and see how users contribute as you go along. [I also think you shouldn't be experimenting with UGC without some goal in mind – how else will you know if your experiment succeeded?  It also focusses your interaction and interface design and gives the user some parameters (much more useful than an intimidating blank page)].

Judith's report (including our table) – motivation and moderation in relation to Europeana – challenging as Europeana are not the owners of the material; also dealing with multilingual collections. Culturally-specific offensive comments. Definition and expectations of Europeana moderation. Resources need if Europeana does the moderation.
Incentives for moderation – improving data, idealism, helping with translations – people like to help translate.

Johan's report – rewards are important – place users in social charts or give them a feeling of contributing to larger thing; tap into existing community; translate physical world into digital analogue.
Institutional policy – need a clear strategy for e.g. how to integrate the knowledge into the catalogue. Provide training for staff on working with users and online tools. There's value in employing community managers to give people feedback when they leave content.
Using Amazon's Mechanical Turk for annotations…
Doing the projects isn't only of benefit in enriching metadata but also for giving insight into users – discover audiences with particular interests.

Costis commenting – if Europeana only has thumbnails and metadata, is it a missed opportunity to get UGC on more detailed content?

Is Europeana highbrow compared to other platforms like Flickr, FB, so would people be afraid to contribute? [probably – there must be design patterns for encouraging participation from audiences on museum sites, but we're still figuring out what they are]
Business model for crowdsourcing – producing multilingual resources is perfect case for Europeana.

Open to the floor for questions… Importance of local communities, getting out there, using libraries to train people. Local newspapers, connecting to existing communities.