Year: 2009

Mashups made of messages – tech talk at Open Hack London

More (very) rough notes from the weekend's Open Hack London event – please let me know of clarifications, questions, links or comments. You can also check out other posts here tagged openhacklondon.

Mashups made of messages, Matt Biddulph (Dopplr)

Systems architecture on Doppler lets them combine 3rd party systems with their stuff without tying their servers up in knots.

At a rough count, Dopplr uses about 25 third party web APIs.

If you're going to make a web service, site, concentrate on the stuff you're good at. [Use what other people are good at to make yours ace.]

But this also means you're outsourcing and part of your reliability to other people. For each bit of service you add, network latency [is?] putting another bit of risk into your web architecture. Use messaging systems to make server side stuff asynchronous.

'&' is his favourite thing about Linux. Fundamental in Unix that work is divided into packets; each doing the thing it does well. Not even very tightly coupled. Anything that can be run on the command line, stick & on the end, do it in the background. Can forget about things running in the background – don't have to manage the processes, it's not tightly coupled.

Nothing in web apps is simple these days – lots of interconnected bits.

In the physical world, big machines use gearing – having different bits of system run at different speeds. Also things can freewheel then lock in to system again when done.

When building big systems, there's a worry that one machine, one bit it depends on can bring down everything else.

[Slide of a] Diagram of all the bits of the system that don't run because someone has sent an HTTP request – [i.e. background processes]

Flickr is doing less database work up front to make pages load as quickly as possible. They queue other things in the background. e.g. photos load, tags added slightly later. (See post 'Flickr engineers do it offline'.)

Enterprise Integration Patterns (Hohpe et al) is a really good book. Banks have been using messaging for years to manage the problems. Atomic packets of data can be sent on a channel – 'Email for applications'.

Designing – think about what needs to be done now, what can be done in the background? Think of it as part of product design – what has instant effect, what has slower effect? Where can you perform the 'sleight of hand' without people noticing/impacting their user experience?

Example using web services 1: Dopplr and AMEE. What happens when someone asks to see their carbon impact? A request for carbon data goes to Ruby on Rails (memory hungry, not the fastest thing in the world, try to take things off that and process elsewhere). Refresh user screen 'check back soon', send request to message broker (in JSON). Worker process connected to message broker sends request to AMEE. Update database.

Example using web services 2: Flickr pictures on Dopplr page. When you request a trip page, the page loads with all usual stuff and empty div in page with a piece of Javascript on a timer that polls Flickr.

Keeps open connection, a way to push messages to the client while it's waiting to do something.

When processing lots of stuff, worker processes write to memcache as a form of progress bar, but the process is actually disconnected from the webserver so load/risk is outsourced.

'Sites built with glue and string don't automatically scale for free.' You can have many webservers, but the bottleneck might be in the database. Splitting work into message queues is a way of building so things can scale in parallel.

Slide of services, companies that offer messaging stuff. [Did anyone get a photo of that?]

Because of abstraction and with things happening in the background, it's a different flow of control than you might be used to – monitoring is different. You can't just sit there with a single debugger.

[Slide] "If you can't see your changes take effect in a system your understanding of cause and effect breaks down" – not just about it being hard to debug, it's also about user expectations.

I really liked this presentation – it's always good to learn from people who are not only innovating, but are also really solid on performance and reliability as well as the user experience.

[Update: a version of this talk is on the Dopplr blog with slides and notes.]

Rasmus Lerdorf on Hacking with PHP – tech talk at Open Hack London

Same deal as my first post from today's Open Hack London event – these are (very) rough notes, please let me know of clarifications, questions or comments.

Hacking with PHP, Rasmus Lerdorf

Goal of talk: copy and pastable snippets that just work so you don't have to fight to get things that work [there's not enough of this to help beginners get over that initial hump]. The slides are available at http://talks.php.net/show/openhack and these notes are probably best read as commentary alongside the code examples.

[Since it's a hack day, some] Hack ideas: fix something you use every day; build your own targeted search engine; improve the look of search results; play with semantic web tools to make the web more semantic; tell the world what kind of data you have – if a resume, use hResume or other appropriate microformats/markup; go local – tools for helping your local community; hack for good – make the world a better place.

SearchMonkey and BOSS are blending together a little bit.

What we need to learn
With PHP – enough to handle simple requests; talk to backend datastore; how to parse XML with PHP, how to generate JSON, some basic javasccript, a JavaScript utility library like YUI or jquery.

parsing XML: simpleXML_load_file() – can load entire URL or local file.

Attributes on node show up as array. Namespace attributes call children of node, name namespace as argument.

Now know how to parse XML, can get lots of other stuff.
Context extraction service, Yahoo – doesn't get enough attention. Post all text, gives you back four or five key terms – can then do an image search off them. Or match ads to webpages.

Can use get or post (curl) – usually too much for get.

PHP to JavaScript on initial page load: JSON_encode -> javascript.

Javascript to PHP (and back)
If you can figure out these six lines of code, you can write anything in the world. How every modern web application works.
Server-side php, client-side javascript.

'There's nothing to building web applications, you just have to break everything down into small enough chunks that it all becomes trivial'.

AJAX in 30 seconds.
Inline comments in code would help for people reading it without hearing the talk at the same time.

JavaScript libraries to the rescue
load maps API, create container (div) for the map, then fill it.

Form – on submit call return updateMap(); with new location.

YGeoRSS – if have GeoRSS file… can point to it.

GeoPlanet – assigns a WOE ID to a place. Locations are more than just a lat long – carry way more information. Basically gives you a foreign key. YQL is starting to make the web a giant database. Can make joins across APIs – woeid works as fk.

YQL – 'combines all the APIs on the web into a single API'.

Add a cache – nice to YQL, and also good for demos etc. Copy and paste cache function from his slides – does a local cache on URL. Hashed with md5. Using PHP streams – #defn. Adding a cache speeds up developing when hacking (esp as won't be waiting for the wifi). [This is a pretty damn good tip cos it's really useful and not immediately obvious.]

XPath on URL using PHP's OAuth extension

SearchMonkey – social engineering people into caring about semantic data on the web. For non-geeks, search plug-in mechanism that will spruce up search results page. Encourages people to add semantic data so their search result is as sexy as their competitors – so goal is that people will start adding semantic data.

'If you're doing web stuff, and don't know about microformats, and your resume doesn't have hResume, you're not getting a job with Yahoo.'

Question: how are microformats different to RDFa?
Answer: there are different types of microformats – some very specific ones, eg hResume, hCal. RDFa – adding arbitrary tags to page. even if no specific way to describe your data. But there's a standard set of mark-ups for a resume so can use that. if your data doesn't match anything at microfomats.org then use RDFa or erdf (?).

RDFa, SearchMonkey – tech talks at Open Hack London

While today's Open Hack London event is mostly about the 24-hour hackathon, I signed up just for the Tech Talks because I couldn't afford to miss a whole weekend's study in the fortnight before my exams (stupid exams). I went to the sessions on 'Guardian Data Store and APIs', 'RDFa SearchMonkey', Arduino, 'Hacking with PHP', 'BBC Backstage', Dopplr's 'mashups made of messages' and lightning talks including 'SPARQL and semantic web' stuff you can do now.

I'm putting my rough and ready notes online so that those who couldn't make it can still get some of the benefits. Apologies for any mishearings or mistakes in transcription – leave me a comment with any questions or clarifications.

One of the reasons I was going was to push my thinking about the best ways to provide API-like access to museum information and collections, so my notes will reflect that but I try to generalise where I can. And if you have thoughts on what you'd like cultural heritage institutions to do for developers, let us know! (For background, here's a lightning talk I did at another hack event on happy museums + happy developers = happy punters).

RDFa – now everyone can have an API.
Mark Birkbeck

Going to cover some basic mark-up, and talk about why RDFa is a good thing. [The slides would be useful for the syntax examples, I'll update if they go online.]

RDFa is a new syntax from W3C – a way of embedding metadata (RDF) in HTML documents using attributes.

e.g. <span property="dc:title"> – value of property is the text inside the span.

Because it's inline you don't need to point to another document to provide source of metadata and presentation HTML.

One big advance is that can provide metadata for other items e.g. images, so you can e.g. attach licence info to the image rather than page it's in – e.g. <img src="" rel="licence" resource="[creative commons licence]">

Putting RDFa into web pages means you've now got a feed (the web page is the RSS feed), and a simple static web page can become an API that can be consumed in the same way as stuff from a big expensive system. 'Growing adoption'.

Government department Central Office of Information [?] is quite big on RDFa, have a number of projects with it. [I'd come across the UK Civil Service Job Service API while looking for examples for work presentations on APIs.]

RDFa allows for flexible publishing options. If you're already publishing HTML, you can add RDFa mark-up then get flexible publishing models – different departments can keep publishing data in their own way, a central website can go and request from each of them and create its own database of e.g. jobs. Decentralised way of approaching data distribution.

Can be consumed by: smarter browsers; client-side AJAX, other servers such as SearchMonkey.

He's interested where browsers can do something with it – either enhanced browsers that could e.g. store contact info in a page into your address book; or develop JavaScript libraries that can parse page and do something with it. [screen shot of jobs data in search monkey with enhanced search results]

RDFa might be going into Drupal core.

Example of putting isbn in RDFa in page, then a parser can go through the page, pull out the triples [some explanation of them as mini db?], pull back more info about the book from other APIs e.g. Amazon – full title, thumbnail of cover. e.g. pipes.

Example of FOAF – twitter account marked up in page, can pull in tweets. Could presumably pull in newer services as more things were added, without having to re-mark-up all the pages.

Example of chemist writing a blog who mentions a chemical compound in blog post, a processor can go off and retrieve more info – e.g. add icon for mouseover info – image of molecule, or link to more info.

Next plan is to link with BOSS. Can get back RDFa from search results – augment search results with RDFa from the original page.

Search Monkey (what it is and what you can do with it)
Neil Crosby (European frontend architect for search at Yahoo).

SearchMonkey is (one of) Yahoo's open search platforms (along with BOSS). Uses structured data to enhance search results. You get to change stuff on Yahoo search results page.

SearchMonkey lets you: style results for certain URL patterns; brand those results; make the results more useful for users.

[examples of sites that have done it to see how their results look in Yahoo? I thought he mentioned IMDb but it doesn't look any different – a film search that returns a wikipedia result, OTOH, does.]

Make life better for users – not just what Yahoo thinks results should be, you can say 'actually this is the important info on the page'

Three ways to do it [to change the SERP [search engine results page]: mark up data in a way that Yahoo knows about – 'just structure your data nicely'. e.g. video mark-up; enhance a result directly; make an infobar.

Infobar – doesn't change result see immediately on the page, but it opens on the page. e.g. of auto-enhanced result- playcrafter. Link to developer start page – how to mark it up, with examples, and what it all means.

User-enhanced result – Facebook profile pages are marked up with microformats – can add as friend, poke, send message, view friends, etc from the search results page. Can change the title and abstract, add image, favicon, quicklinks, key/value pairs. Create at [link I can't see but is on slides] Displayed in screen, you fill it out on a template.

Infobar – dropdown in grey bar under results. Can do a lot more, as it's hidden in the infobar and doesn't have to worry people.

Data from: microformats, RDF, XSLT, Yahoo's index, and soon, top tags from delicious.

If no machine data, can write an XSLT. 'isn't that hard'. Lots of documentation on the web.

Examples of things that have been made – a tool that exposes all the metadata known for a page. URL on slide. can install on Yahoo search page, add it in. Use location data to make a map – any page on web with metadata about locations on it – map monkey. Get qype results for anything you search for.

There's a mailing list (people willing and wanting to answer questions) and a tutorial.

Questions

Question: do you need to use a special doctype [for RDFa]?
Answer: added to spec that 'you should use this doctype' but the spec allows for RDFa to be used in situations when can't change doctype e.g. RDFa embedded in blogger blogpost. Most parsers walk the DOM rather than relying on the doctype.

Jim O'D – excited that SearchMonkey supports XSLT – if have website with correctly marked up tables, could expose those as key/value pairs?
Answer: yes. XSLT fantastic tool for when don't have data marked up – can still get to it.

Frankie – question I couldn't hear. About info out to users?
Answer: if you've built a monkey, up to you to tell people about it for the moment. Some monkeys are auto-on e.g. Facebook, wikipedia… possibly in future, if developed a monkey for a site you own, might be able to turn it auto-on in the results for all users… not sure yet if they'll do it or not.
Frankie: plan that people get monkeys they want, or go through gallery?
Answer: would be fantastic if could work out what people are using them for and suggest ones appropriate to people doing particular kinds of searches, rather than having to go to a gallery.

'Organisational change' session at MW2009

I was chairing the session so my notes are a bit sketchy. It's worth reading the full papers and following the slides online.

Intro notes: it's an interesting moment for the sector, maturity of approaches to the web. Turning the analytical gaze inwards, working towards a more effective, integrated and considered use of technology. This brings new challenges in managing expectations and demand. Wider consultation means adapting our language and understanding, but the benefits of collaboration are worth it.

Organisational Change for the On-line World – Steering the Good Ship Museum Victoria
David Methven, Head, ICT, and Timothy Hart, Director, Information Multimedia and Technology, Museum Victoria, Australia.  Slides: http://www.slideshare.net/museumsandtheweb/tim-hart-david-methven-organisational-change-for-the-online-world-steering-the-good-ship-museum-victoria

Tim Hart started, talking about their in-sourcing model; build capability, drive money otherwise spent outsourced inside the org. Interruption by David! Trying to change org culture, 'blah blah blah'. They used an audience volunteer for dramatisation!
Therapy for Tim. Circle. Telling people what we should be doing, not how. Changing work practices. Not consulting us, asking us what we want to do and how we should change what we're doing.
Process. Once strategy was done, job not done. Didn't understand how much ownership the org wanted of the strategy. People who weren't involved in the process kicked up.
Established exhibition production processes.
Interesting conceptual model. Relationship.
Internal experience of applications, IT systems.

Down To Earth: Social Media and Institutional Change
Patricia Deiser, Museum voor Communicatie; and Vincent de Keijzer, Gemeentemuseum, The Netherlands. Slides: http://www.slideshare.net/museumsandtheweb/vincent-de-keijzer-patricia-deiser-down-to-earth-social-media-and-institutional-change

Vincent and Patricia: addressing people who are not willing or able to come to the museum. New roles for the public. Make use of knowledge, time, enthusiasm of the public.
Brave new world, head spinning. But had to get down to earth. Colleagues were being polite, but no one was actually doing anything. Realised approaching it in the wrong way – presenting it as something everyone is doing, we should be doing it. But should try to convince them about what would benefit them in these web 2.0 things. Had to seduce them. Much harder to do. Asked experts from outside the museum to help develop online strategy. Stop talking with people outside the museum, start talking with people inside the museum about this. Let people discuss it among themselves. Let them go online, learn about it for themselves. Low profile platform for staff to experiment. Start with your own, internal community, build a community from there.
Continuous access to cultural heritage with university of Amsterdam. Built a platform for museum staff, for ideas, proposals, projects. Asked Patricia, as a student, to research, interview colleagues. Outsider perspective.
Machiavelli quote.
Patricia's research: How do people interact with public, how much do they know about web technology, do they use it themselves; what are they enthusiastic about?
Talked about research process. Showed colleagues examples of other things. Asked colleagues to research their presence online on e.g. Flickr, see what people had already put out there.
Models of staff members from the research: Lecturer – likes to prepare thoroughly, then make a publication/presentation of it. One-way focus. they send their knowledge out to the public, not interested in feedback from people who aren't also scholars.
Fear of losing expertise if everything goes online.
Educators – not people in education dept, label for group. More into interaction, want some feedback . Teacher – pupil relationship. Afraid of examples where people could load UGC onto website.
Presenters – same attitude to communication as educators, but more advanced in web technology.
Interactors – already working with the public, do want to have interaction with public, but not advanced with technology. Old school education departments
Connectors – same attitudes to public, but advanced in using web technology.
Mapped staff into the categories. Difficult diagram to show internally! Scale of communication style (one way, two way focused) and use of technology. Difficult to get everyone into connectors corner, but at least get people to move up scale on use of technology.
Communities of practice.
Everything that goes onto desk goes onto website.
Still needs a lot of social skills, persuasion.

After the Heroism, Collaboration: Organizational Learning and the Mobile Space
Peter Samis and Stephanie Pau, San Francisco Museum of Modern Art, USA. Slides: http://www.slideshare.net/psamis/after-the-heroism-collaboration-organizational-learning-and-the-mobile-space

Stephanie and Peter: digital and analogue resources. Benefiting from experience of other institutes either as staff change or working with other orgs.
Interpretive goal process. Cross departmental dialogue and interpretive brainstorm process. Workshop – answers to basic questions to help formulate a strategy.
Key questions – what's the rationale for this project? Why here, why now?
List 1 – 3 main visitor take aways.
Who's the intended audience, and why?
What didactic elements are planned? What other modes of interp inc multimedia should we consider?
Case studies. Showing how the process worked in exhibs with really different requirements.
Peter – evaluation studies. Different modes of use – wall texts vs multimedia guides.
"What a visual interface brings to the party…" Break picture into components, not a slave to a minute and a half overview.
What people want – pre-loaded vs call in.
Sharing usage figures – ace.
What information did on-site visitors not get? if they didn't have the cell phones. Breakdown of what content was available by which methods.

Christian Heilmann on Yahoo!'s YQL, open data tables, APIs

My notes from Christian Heilmann's talk on 'Reaching those web folk' with Yahoo!'s new-ish YQL, open data tables and APIs at the National Maritime Museum [his slides]. My notes are a bit random, but might be useful for people, especially the idea of using YQL as an easy way to prototype APIs (or implement APIs without too much work on your part).

For him it's about data on the web, not just technology.

Number of users is a crap metric, [should consider the user experience].

Stats should be what you use to discover areas where are the problems, not to pat yourself on the back.

People with blackberries have no Javascript, no CSS. Don't have front-loading navigation they have to scroll through – cos they won't.

If you think of your site as content, then visitors can become 'broadcasting stations' and relay your message. Information flows between readers and content. They're passing it on through distribution channels you're not even aware of.

Content on the web is validated with links and quotes from other sources e.g. Wikipedia. People mix your information with other sources to prove a point or validate it. eg. photos on maps.

How can you be part of it?
Make it easy to access. Structure your websites in (plain old semantic HTML) a semantic manner. Title is important, etc. Add more semantic richness with RDF and microformats. Provide data feeds or RSS. Consider the Rolls Royce of distribution – an API. Help other machines make sense of your content – search engines will love you too.

Yahoo index via BOSS API – Yahoo do it because they know 'search engines are dying'. Catch-all search engines are stupid. Apples are not the same apples for everyone. Build a cleverer web search.

http://ask-boss.appspot.com/ – nlp analysis of search results. Try 'who is batman in the dark knight' – amazing.

BOSS provides mainstream channel for semantic web and microformats. Microformats are chicken and egg problem. Using searchmonkey technology, BOSS lists this information in the results. BOSS can return all known information about a page, structured.

Key terms parameter in BOSS – what did people enter to find a site/page? http://keywordfinder.org/ – what successful websites have for a given keyword.

Clean HTML is the most important thing, semantic and microformats are good.

If your data is interesting enough, people will try to get to it and remix it.

[Curl has grown up since I last used it! Can be any browser, do cookies, etc.]

Now the web looks like an RSS reader.

Include RSS in your stats.

Guardian – any of their content websites put out RSS through CMS. They then provided an API so end users can filter down to the data they need.

Programmable Web – excellent resource but can be overwhelming.

The more data sources you use, the more time you spend reading API documentation, sos every API is different. Terms, formats, etc. The more sources you connect to, the more chances of error. The more stuff you pull in, the slower the performance of your website.

So you need systems to aggregate sources painlessly. Yahoo Pipes. A visual interface, changes have to be made by hand.

You can't quickly use a pipe in your code and change it on the fly. e.g. change a parameter for one implementation. No version control.

So that's one of the reasons for YQL: Yahoo Query Language. SQL style interface to all yahoo data (all Yahoo APIs) and the web. Yahoo build things with APIs cos it's the only way to scale. Book: 'scalable websites', all about APIs.

Build queries to Yahoo APIs, try them out in YQL console. Provides diagnostics – which URLs, how long it took, any problems encountered. Allows nesting of API calls.

Outputs XML or JSON, consistent format so you know how to use that information.

YQL also helped internally because of varying APIs between departments.

Gives access to all Yahoo services, any data sources on the web, including html and microformats, and can scrape any website.

Open tables
Easy way to add own information to YQL. Tell Yahoo end point where can get the info.

Jim wanted to allow people to access data without building an API. All it needed was a simple XML file.

[Though you do need RSS results from a search engine to point to – I'm going to see what we can output from our Google Mini and will share any code – or would appreciate some time-saving pointers if anyone has any. Yes, hello, lazyweb, that's my coat, thanks.]

Basically it's a way of providing an API without having to develop one.

Concluding: you can piggyback on people's social connections with other people by making data shareable. [Then your data is shared, yay. Assuming your institution is down with that, and no copyrights or puppies were hurt in the process.]

APIs are a commitment – have to be available all the time, lot of traffic, but hard to measure traffic and benefits. Making APIs scale is a pain and have to be clever to do it. Pointing YQL open data table pointing to search engine on your site also works.

Saves documenting API? [??]

YQL handles the interface, caching and data conversion for you. Also limits the access to sensible levels – 10,000 hits/hour.

Jim – 'images from collection' displayed on page as badge thing with YQL as RSS browser. Can just create RSS feed for exhibition than can new badge for new exhibition.

Using YQL protects against injection attacks.

Comment from audience – YQL as meta-API.

Registering is basically making the XML file. You need a Yahoo ID to use the console. [The console is cool, basically like a SQL 'enterprise' system console, with errors and transaction processing costs.]

We had questions about adding in metrics, stats, to use both for reporting and keeping funders/bosses happy and for diagnostics – to e.g. find out which areas of the collection are being queried, what people are finding interesting.

github repository as place to register open tables to make them discoverable.

There's a YQL blog.

[So, that's it – it's probably worth a play, and while your organisation might not want to use it in production without checking out how long the service is likely to be around, etc, it seems like an easy way of playing with API-able data. It'd be really interesting to see what happened if a few museums with some overlap in their collections coverage all made their data available as an open table.]

Running notes, day 3 (Saturday) of MW2009

These are my running notes from day 3 of the Museums and the Web conference – as the perfect is the enemy of the good I'm getting these up 'as is'. I did a demo [abstract] in the morning but haven't written up my notes yet – shame on me!

The session 'Building and using online collections' included three papers, I've got notes from all three but my laptop battery died halfway through the session so only some of them are already typed – I'll update this entry when I can sneak some time.

Paul Rowe presented on NZMuseums: Showcasing the collections of all New Zealand museums (the linked abstract includes the full paper and slides).

National Services Te Paerangi (NSTP).

4 million NZers, 400 museums.  NZMuseums website – focal point for all NZ museums. NSTP administers the site, Vernon Systems is solution provider.

Each museum has a profile page including highlights of their collections. Web-based collection management system.

What needs to be in place for small museums to contribute? How can a portal be built with limited resources? What features of the website would encourage re-use of the data?

Some museums had good web presences, but what about the small museums? Facing same issues that small or local govt museums in the UK face.

Museums are treasures of the country, they show who we are. Website needs to reflect that.

Focus groups – volunteers are important – keep it simple; keep costs low; some places had limited internet connectivity; reservations about content being on the internet were common.

Promoting involvement to the sector – used existing national monthly newsletters to advertise workshops and content deadlines. Minimum of 20 items for placement on site to avoid 'box ticking' [some real commitment required]. Used online forum for FAQs.

Lack of skills – NSTP were trained so could then train staff and volunteers in museums. Digitising, photography for the web.

Had to explain benefits to small museums. It gave them an easy start to getting an online presence.

They overcame resistance by allowing watermarking and clear copyright statements; they showed existing museums sites that allowed tagging; promoted that would help them reach a diverse dispersed audience.

First tag on site – 'shiny nose'. First comment was someone admitting they'd touched the nose on a bronze sculpture.

eHive.

Could also import Excel spreadsheets as content management system didn't exist at early stage of project. Also provided a workaround for people with lack of internet – the spreadsheet could be posted on CD.

API provides glue to connect eHive (Collections Management System) and NZMuseums site together.  

Tips for success
Use OS software where possible; use existing online forums and communication networks to save answering questions over again.

90% of these collection items not previously available on the internet. 99% of collection items have images.

[Kiwis are heroes!  Everyone was incredibly modest about their achievements, but I think they're amazing.]

Next was Eero Hyvönen on CultureSampo – Finnish Culture on the Semantic Web 2.0: Thematic Perspectives for the End-user (the linked abstract includes the full paper and slides).

Helsinki semantic web thingies
Part of national ontology project, Finland
Vision – international semantic web of cultural heritage. Marriage between semweb and web 2.0

Challenges – content heterogeneity, complexity 

Other challenge relates to the way cultural content is produced – Freebase, Wikipedia, open street maps, etc, 

Semweb for data integration; web.2 0 approach for content production

Automatically enriched by each piece of knowledge.

In Finnish the sampo is a magic drum that makes everything possible.  

Portal intended for human users and machines. Trying to establish a national way of producing content so can be published automatically.  

Infrastructure – 37,000 class concepts in ontology. MAO, TAO – museum ontologies, collaboratively built ontologies, then mapped to national system. End user sees one unified ontology. [A little pause while I pick my jaw up from the ground.]  66 vocabularies, taxonomies and ontologies available online as services, can be used as AJAX widgets. Some vocabularies are proprietary so can't be published online in the service.

28 content providers, 22 libraries and museums and some international associates like Getty places, Wikipedia.

16 different metadata schemas. [Including some for poetry!]

134,000 cultural collection items (artefacts, books, videos, etc)

285,000 other resources (places, people etc)

Annotation channel for content items – web 2.0 type interface.

Semantic web 2.0 portal

Portal users – for humans, Google-like but semantic search. Nine perspectives into cultural heritage. Three languages. Recently view items, recently commented items.  

Map view.

With one line of JavaScript on own website, can incorporate CultureSampo on own website.

[Sadly my laptop died here and the rest of my notes are handwritten.  You can probably get the gist from the published paper and the slide, but the coolness of their project was summed up by this tweet: Musebrarian: What can you do with a semantic knowledgebase? Search for "beard fashion in Finland" across time and place. #mw2009

It might not sound like much, but the breadth of content, and the number of interfaces onto it was awe-inspiring.]

Sadly my notes from Brian Dawson's paper, Collection effects: examining the actual use of on-line archival images are also still on notepaper.  The paper was a really useful examination of analytical approaches to understanding the motivations of people using cultural heritage collections.

A quick summary of my MW2009

I'm posting this now to get it out of the way (and done in April) though I still haven't caught up on the Museums and the Web 2009 'backchannel', tidied my notes or read all the papers I wanted to read. I may update this later as I remember things I wanted to say.

Some strong themes (memes?) emerged during the conference. In general, while lots of great sites and projects were presented, including some lovely examples of projects breaking new ground in best practice, some of the most important ideas weren't about presenting new, flashy things but rather reflected a maturity in approach, and a consolidation of the role of the web in museums.

Breaking out of the bubble
From the informal conversations and unconference sessions proposed it seems to be an issue lots of people are struggling with – how do we communicate with managers, curators, educators to get them excited about the possibilities of the web; Nina's question about how we bring the levels of participation we're seeing on museum websites into the physical museum; how does (or how should) an integrated web program change an organisation; how do web teams go from mavericks to maturity?

And leading on from that: the post-conference challenge – do one thing in April

Conferences are great, especially one as social as Museums and the Web. Those inspiring late night conversations, the unexpected connections, putting faces to names… but I sometimes come away from conferences as cynical as I am enthused because before you know it, you're back at the same conference next year and nothing has changed.
The 'do one thing different when you get back' idea that suffused the crowd-sourced closing plenary really inspired me. Using the post-conference high to make one small change or proactively share with colleagues rather than letting it dissipate seemed to appeal to lots of people – I wonder if there's a way of finding out who's taken up the challenge. I hope I'm going to keep the inspiration to do the Right Thing, to keep pushing for quality when resources and energy are limited and projects are many.
I also realised that after all the inspiring conversations of last year some of us came back from MW2008 and ended up with BathCamp, so while the post-conference crash back to reality may be unavoidable, it doesn't mean you can't get something done anyway.
So I've been working away on the museums API wiki (possibly better known as 'museums and re-usable shareable data' but hey ho), tagging links 'mw2009' in delicious, and following up some contacts with email conversations. There's a lot more I should be doing, and if I haven't yet been in contact with you about something we discussed, let me know.

The unconference
I want to write a proper post about how it worked so that other people would feel comfortable running one of their own, but in the meantime, I'll just say that I was thrilled that it seems to have been so useful for people.

Twitter
The impact of Twitter was really evident at this conference. Apart from finding people for food or drinks, I used it most usefully to suggest an informal meetup of people interested in museum APIs during the Friday, and to find a whole bunch of people to go and eat noodles with. You can get a sense of the progress of the conference from my MW2009 tweets (from my 'event' twitter account).

Randomness
On a personal note, I also made up a new description for myself as I needed one in a hurry for moo cards: cultural heritage technologist. I felt like a bit of a dag but then the lovely Ryan from the George Eastman House said it was also a title he'd wanted to use and that made me feel better. And I won a 'backchannel award' for blogging from the conference, woo!

As well as earlier posts on the opening plenary and the unconference session on failure I still have more notes to dump into posts, I'll tag them all so you can find them under MW2009.

Notes from the closing plenary, MW2009

These are my quick and dirty notes from the closing plenary of the 2009 Museums and the Web conference .  If I've quoted you but gotten your name wrong, I'm very sorry – please let me know and I'll correct it.  I haven't put links in for anyone yet so I'll be editing the entry anyway.

'We are the program.'  Awards for blog posts, tweets, Flickr photos then David Bearman invited people to come up and talk about what they've learnt, what they'll take away.

Nina, Museum 2.0 – inspired by Max's keynote address. But she didn't feel that difference in the institution. Didn't see the transparency and openness that you get on the web, on their dashboard. Not saying they have to do that, but wants to bring up idea of participatory ghetto… forming relationships with visitors on the web, who'll show up at museums and wonder why the same relationship isn't reflected in the building. Pushing in institutions to establish parity, not to give up on physical space also being somewhere for openness and transparency. IMA – had experience of extreme cognitive dissonance. How can you start the conversation, taking great stuff from web world into physical environment of institutions. Her first time at MW.

Heather from Balbao – new to conference and museum world, great introduction.

Nate, Walker Art Centre – I always leave inspired, seen it happen every time- a month worth of trying new things, then it trickles off and fades… go to the wiki and take the post-conference challenge to do one thing in April – choose one task that you can achieve by the end of April. Distributed agile development … beyond API, everyone can benefit from going home and immediately doing just one thing. [eek I feel weird taking notes about my ideas]

Frankie, Rattle – be excited about tin mining.

Brian, UKOLN – danger that losing accessibility cos doing innovative things, but there have been some really great examples. Universally accessible – pushing it (the definition) of it forward.

Seb, Powerhouse – need to bring people in, curators, management.

Julie (?) – boundaries between web and physical boundaries – problematising the name of the conference. Is 'web' starting to constrain what we're about?

Nina – comment on that – conference in US called WebWise – lousy content but less funded projects, mostly director level people who go. How do we get these people in a situation that's more blended with the kind of people who are here?

Victoria, Smithsonian? carrying on Nina and Seb's point – spends first month being excited, but directors etc aren't going to come to conferences like this. You may have five minutes to articulate why something is important – and it's not heard when it's someone outside, even if you've been saying it on the inside for years. Having someone who's succeeded from outside, doing snippets of video or whatever – convincing.

David – seeing what can share back. Spend time at conference demanding people write papers, share slides… would really love for the post-conference discussion that takes place online to incorporate thoughts, experience about what doing. Extension into social space of a discourse we've never really had – how do you use that post-conference excitement… how do organisations change, which is becoming the centre of the discourse… take it further, keep talking to each other about how do you make it work.

Jennifer – the thing we can do by the end of April, if you write a report, share it with your colleagues. Let people pinch your ideas, send it out. Share the reports as well as the stuff that happens when we're right here.

Jon Pratty – we need a more social media within the museum.

Peter Samis – can remember this camaraderie in 1991… hearing it just as fresh now with people who are coming to their first conference, loving it… this is going to have legs, it's going to keep running, continue this spirit throughout the year.

Rich (another Rich) – haven't really felt the amount of community before, but have been coming since 1999. Being able to catch up on the things he missed while he was here.

Brian – people in the community can fall out, it's happened in the UK. People have strongly held views, need to depersonalise disputes, constructive criticism.

Scott (?) – we're not the only people talking about these subjects, it's happening in higher education, the commercial sector, not a whole of discussion here about what's happening out there and what impact it has here. Would be neat to do some headlines on what's going on in the world outside museum, add to the implications for this audience.
[This final session probably contributed quite a bit to my summary of MW2009 – I'd written the 'MW2009 challenge' a little while before (after discussions at the ice cream API meet) and it was wonderful to feel so much excitement (tempered with realistic cynicism) in the room about the positive changes we could make when we went back to our home institutions.]

The museum/academic divide, museum labels and mobile-accessible interpretation…

This post from the Smarthistory blog, Writing the Museum Label on a Wiki (and some other ideas), very neatly brings together some of the proposals and conversations from MW2009.

The authors suggest two 'notions' to bring about greater collaboration between museum educators (in the broadest sense) and academics, starting with the premise that:

For some time now, we have been publicly questioning the division that exists between two professional groups tasked with educating the public about art: those in museums (curators and educators) and those in the academy (art historians). These two communities share expertise that is sought by the museum visitor and the student, yet they rarely meet, too often do not attend the same conferences, and almost never collaborate.

I won't pre-empt their notions here – go and read the article, but I just wanted to highlight this quote:

Can the tired modernist fiction that the direct experience of the object must remain unencumbered by the frame of context really still be operative?

I really hope not!

(As an aside, my inner web geek is hugely amused by the irony of the '403' contained within the URL of the article – 403 is the HTTP status code for 'access forbidden'.)

Max Anderson, Indianapolis Museum of Art, 'Moving from virtual to visceral'

[My very rough notes, I've been distracted and missed bits, probably misheard things, etc, but at least it gives people who weren't there an idea of the flavour of the event.]

[Update – the video is online, you may find that easier to digest.]

Max Anderson, director of the Indianapolis Museum of Art – opening plenary at Museums and the Web 2009, 'Moving from virtual to visceral'.

Behind the applications we're producing, for an art museum in particular, we're looking for something beyond exposure to things. Felt, deep, elemental emotions.

Museums tend to define selves in terms of number of objects, acres, visitors… difficult to translate that endeavour online in a way that sticks. If you get too caught up in plumbing of technology, you forget about the outcomes. "In addition to bragging rights, what features of a museum should be online?"

[missed bits]

So much of what goes on behind scenes is critical to experience of front of house, give visitors an experience of that behind the scenes stuff.

Volunteers are important. Enormous swathe of public who could be continually engaged…

Review of what happens on site in museums: flirting, gossiping, looking, shopping, eating. Looking is wedged among all other activities. Is it heresy to admit that looking and learning are only a small part of a museum visit?

George Hein, visitor survival curves… pathways through museums. Opportunities to construct a narrative – but this isn't the normal approach to museum attendance. Choice and opportunity should govern what happens, rather than doing what we think the public demands.

Average time in front of a work of art is 3 seconds, it's 'glancing at picture to confirm that's what you've just read about' on the wall label. There are precious few moments when people linger in front of objects – it's like the Museum of Drag Racing. But look at the percentage of time learning versus that spent living in university. So much of time in college is extrinsic to learning. Give permission for that kind of experience in institutions, make a bond with visitors that's visceral, not just virtual.

Excessive orientation can compromise the experiential/learning environment.

There's too much clutter in experiential terms, we're torn constantly by stimuli. 'Virtual' museum programs – we fall in love with the technical solutions, but it's not obvious that they have the intended effect, and are also good for audiences.

An interactive map is still a map – show what's behind the velvet rope, it's sensual, appealing.

Databases – data cleaning before putting things online vs put everything online now and see how much have to clean up. Publishing databases about collections is just a baseline, just the beginning of something. Having high resolution images is, for lots of visitors, not much better than credits at the end of a movie. It's necessary, but not sufficient. We've only gotten to building the ground floor – we need to take visitors to the whole movie, get beyond showing them the credits.

Eschew the virtual, promote the visceral. What we collect has stories behind it, telling the stories is the key issue. What are the avenues for achieving something visceral in a museum visit?

Suspend commercial intrusion and attempts at mind control. Bruegel King.

Privatisation takes away the possibilities of adventure and play. Encourage voyeurism – show staff in action. Public wants to see how museums operate and function. CSI is a hit because it shows what goes on behind police work. We should take a cue from that appetite and enthusiasm.

[Missed a bit]

People enjoy being near original objects. Revel in the thrill of proximity to the original object.

Foster projection into another time, place or condition.

Disorient visitor, give them permission to move away from comfort zone. Encourage playtime, dream time.

Help visitors savour and retain memories and empathetic response. The empathetic response is the memory maker.

Help visitors apply that memory and empathetic response to daily life. Take the lesson embedded in museum objects, find the link to continued application to life after you've left the museum. Extensions of learning beyond what happens on site.

Teach now to practice connoisseurship at the mall – aesthetic standards around quality, etc.

Preferable scenario to onsite orientation – use pre-visit planning and post-visit environment. Encourage visitors to get ready before visit online beforehand. Kids who come on school visits to the Indianapolis Museum of Art get memberships for families, they can come back and share their learning with their parents.

Pre-visit – choose a path based on visit duration or thematic interest. Wander through the museum behind the scenes online. Show how complicated it is to run museums, "not as simple as switching on the lights and admiring things". Sharing this is useful, particularly given current climate.

On-site: deviate from planned path, improvise. The visitor doesn't need to be tied to our taxonomies. Make room, permission for visitor to experience it that way.

Post-visit – join Facebook, etc. We want to be part of your lives, not just a visit.

Encourage participation by telling the truth, the whole truth and nothing but the truth. We've been good at promoting and marketing and cheering ourselves on but not necessarily good at telling the truth.

It's about shining a light on how far you've got to go, not just what you've got to show off.

IMA had layoffs, their dashboard showing employee count dropped. If a museum has more people in development than in education or interpretation, then is bereft.

One of the truths we need to get out to visitors… permanent collection isn't permanent. Refreshment of new works of art on view. Objects are not permanently on display, they're in storage, lent, borrowed. Rhythm [of display, I think]. Permanent collections are invitational as well.

Attendance figures – "we don't necessarily think that's true but that's the number we quote". Is it a clicker with guards, estimates?

Mapping visitors by postcode, what postcodes aren't represented? Did he just say 'statistical porn'??

Making institutions more a part of your life, talking about collections. Also showing deaccessioning. Policy on deaccessioning on web – 'a no brainer'. Also put up db of works planning to deaccession as well as ones have deaccession. He has the uncomfortable feeling that he shilling for deaccession but they're really just trying to be transparent about process. Next process is to link funds from deaccession to artworks acquired with funds from its sale.

Charity Navigator. Don't trust what you see on the web. Methodology is flawed. We should present those stats ourselves, don't wait for someone else to be arbiter of our fortunes.

Why wait for someone else to publish your financial info, do it yourselves when it's ready.

Steve.museum – more empathetic. Opera in Italy in 19thC was riotous but now is antiseptic – don't make that mistake in museums.

Transcriptions, search terms is the magic of ArtBabble. Search results will take you to that second in the video. [Awesome!]

Click, Brooklyn Museum. Crowd-curated exhibition. Brooklyn Museum API – on screen, [missed stuff]. Te Papa. [Woo, check the love!]

Don't bet the farm on commercial applications that may go away.

[The end! I'll update when slides go online and tidy and correct if I can get more accurate info. Questions to come when I get a moment.]