Try to expose all the metadata being added by users via an API – a “social catalogue”. This enables ‘discovery’, but not ‘delivery’ – this is where Mendeley can make use of external APIs – such as the WorldCat API.

Mendeley invest in APIs because

• It helps them extend their product, but integrating data/functionality from other places
• It enables others to extend their product – they don’t have time to build everything that users are asking for. E.g. Android client built by users, as company didn’t have the resource

Mendeley uses WorldCat registry to find/suggest appropriate OpenURL resolver depending on users location – as most users won’t know what an OpenURL resolver is, or what the detail are.

Mendeley uses OAuth – which means they can integrate with institutional repositories and a users own publications in Mendeley – going to be live soon (working with JISC, Symplectic and University of Cambridge on this – http://jisc-dura.blogspot.com/). Learnt a lot about consuming their own APIs in this project – and uncovered bugs…

“We should have built the API first, and the product second” – the fact they didn’t now creating work. Now they are creating a new application for libraries, and building API first. Ian firmly believes this is a better approach.

Ian’s top 10 tips for API provision:

• first API, then app
• use your own APIs (and he believes Mendeley should do this more)
• make an (API) interface you would use yourself
• provide lots of example docs – coders like to do stuff quickly – if they can get something working from an example quickly, they’ll then invest
• version your API – backwards compatible
• put rate limits in place
• work with a 3rd party to provide keys
• have clear licensing
• engage with your community
• promote, promote, promote, promote, promote

In terms of consuming:

• know what you want to do
• define the value – this may be service delivery, or could be development of skills for developers etc.
• measure the value – otherwise difficult to prioritise future developments
• understand the SLA
• if it’s important – have a backup plan – dependence on 3rd party is a risk which you should manage
• don’t wait on API for page loads – found that Mendeley homepage was waiting for a response from an API was down, and so the page didn’t load…
• get on the mailing list/dev group
• look for good example code
• don’t be afraid to pay – if it’s important, it’s worth paying for
• use Mendeley’s APIs

Now Antonio Tejada and Hans-Siem Schweiger are talking about Citavi which combines Reference Management and Knowledge Organisation. Citavi designed to help with searching, retrieving results, acquiring materials – all of these require interaction with library sources. Citavis supports adding data manually, from file upload, browser extensions and via APIs.

Manual entry is error prone and time consuming

File upload – uses standard formats (RIS/BibTeX); supported by a wide range of catalogues and databases; but still time consuming

Browser Extension – e.g. looks for embedded metadata in the page (e.g. COinS) or find standard identifiers in the page (e.g. ISBN) and import data

APIs – eliminates the browser – you don’t need to go to lots of different sources on the web. Fastest mechanism. Direct. Integrates in the workflow much better. However cost of implementation can vary quite a bit – it all depends on the API – some very fast (e.g. z39.50 can do in minutes now, but custom APIs can be more difficult)

Citavi Features which use APIs:

• Online search – integrated into the Citavi application
• Retrieve by identifier (e.g. DOI, ISBN, PubMed ID)
• Import formatted bibliography – can take a bibliography from a word file and Citavi will run a search for each item in the bibliography
• Find Library Locations
• Find Full Text
• Check availability with OpenURL (seems like this actually just pushes user to their local resolver?)

Universities can get site license for Citavi – allows library to create a special settings file with authentication details for databases (that are not IP authenticated)

Challenges for Citavis using APIs:

• Administrative challenges
• Some libraries don’t want to be accessible (at least via a desktop application)
• Catalogues that charge by the record for metadata
• Inconsistent communication – e.g. change of settings on library system, don’t inform Citavi
• Technical challenges
• Custom catalog software – missing or inconsistent standards support and inconsistent field mapping
• Legacy data – not as well-structured; inconsistent data entry

Going forward:

• Geographic search (WorldCat Search API)
• Enhanced availability search (WorldCat Registry and OpenURL Gateway)
• Acquisitions management (WMS Acquisitions) – Citavi didn’t anticipate this, but some libraries using Citavi to manage acquisitions processes
• Metadata – looking at authority control (WorldCat Identities); Alternate editions (xISBN); ISSN lookup (xISSN)

Terry McAndrew (from JISC TechDis) – want to network experience with resources – identify accessible practice/purposes. Terry says most ‘OER Problems’ are social not technical. Asks – can we find learning registry output via Google?

Walt Grata showing tools that he has built on top of Learning Registry … (on github):

• ‘Landing pages’ for content – that can be indexed via Google (think this is new, and not up at github yet)
• Harvesting tool – to grab stuff from a node and put into another storage mechanism – e.g. couchdb,  postgresql, etc etc

Pat Lockley – slides

• Chrome plugin, code on github
• No-one will search outside Google – so take learning registry to Google. Chrome plugin finds all links on the page, and checks each one on the learning registry – and looks for some common attributes – like ‘title’ or ‘description’ etc. – and can then manipulate browser display to make use of this data.
• WordPress Widget  - code on github
• plugin for WordPress to display content from a learning registry (node) in a wordpress blog

Steven Cook – used Cake (PHP framework) to extract and ‘slice’ data from learning registry node. Also pulling data from other sources – like Topsy. Code on github. Talking about how can’t expect Learning Registry to do the hard work here – have to expect to pull out data, cache it, etc. Notes learning registry API isn’t completely RESTful (? not sure what the issues are) .

A simple use case for the learning registry is:

• Nasa publishes a physics video
• PBS posts a link to the video
• NSDL posts a link to the video
• A school uses the video in a course in their Moodle VLE

Each place/portals where the link is used or published only knows about use of their link or copy of the resource. So Learning Registry aims to support way of sharing this type of activity ‘in the open’ – so that this can be captured and reflected – the ‘social metadata timeline’ – Learning Registry is to provide infrastructure to support this. Learning Registry describes this type of activity/usage data associated with a resource ‘paradata’ – although learning registry learning registry doesn’t care what type of data it stores (as long as it can be expressed as JSON)

The learning registry is “an idea, a research project, an open source community project, a public social metadata distribution network’…

The guiding principles: be enabling, capability not solutions, no barrier entry, no single point of failure – everything distributed…

Not going to try to blog the technical architecture of.. but summary of APIs

• Distribute API - uses http POST. About copying data from one node to another – i.e. achieving the distributed part of the architecture
• Publish API – how you get stuff into a Learning Registry node (that is, you, the producer of information, publish it *to* the learning registry node) – uses http POST. Learning reg also supports SWORD for publishing data into a node
• Obtain API – getting data out of a learning registry node – uses http GET
• Harvest and OAI-PMH APIs – another way of getting stuff out of the node. Harvest returns JSON but supports OAI-PMH type actions. OAI-PMH also supported.

It is stressed that this is really a project at the start of its work – the way to engage and to find out how to do this stuff is to join the community – join the developer list etc. and raise issues, ask questions – this is part of the experiment and will inform the development.

JLern

JLern is the project to setup an experimental node in the UK – being run by Mimas.

2 kinds of nodes in the Learning Registry:

• Common node
• Gateway node

JLern have setup a ‘common node’ – this can support:

• Publish services
• Access services
• Distribution services (JLern now have a 2nd common node up and running to try these)
• ….

Common nodes can be part of ‘networks’. Networks can (only) be connected via ‘gateway nodes’

When networks are connected, this is called a ‘community’. A ‘network community’ is a collection of interconnected resource distribution networks. A resource network can only be a member of one community.

Now have published the JORUM metadata (via OAI-PMH) – so about 15k resources. Open University now looking at similar activity. Now Jorum exploring framework for capturing paradata about resources.

Gathering ideas and use cases now – e.g. see JLern challenge from dev8D http://dev8d.org/challenges/

The JLern ‘Alpha’ node is at alpha.mimas.ac.uk – you can authenticate using details given in this blog post http://jlernexperiment.wordpress.com/2012/02/02/alpha-node/.

As already mentioned they’ve harvested JORUM OAI-PMH data and published on JLern alpha node.

They now have ‘Beta’ node (this doesn’t represent a level of development – just naming convention I think) – this is running on Windows (Alpha is on Linux). Also planning a ‘Gamma’ node running on Amazon EC2.

JLern hackday held in January – write up at http://jlernexperiment.wordpress.com/2012/02/21/the-hackday-report-and-reflections/, and also a Java Library for interacting with Learning Registry nodes at https://github.com/navnorth/LRJavaLib .

A lot of the stuff Ian talks about comes back to games – perhaps because about playing and about building – he went into computing because he wanted to build games. Not just about writing games – but using toolkits to mod characters, game play etc.

Also interest in animation (involved in a BCS group about animation) – and this is about art and technical skill – you need to bring together people with these different skill sets, and each needs to understand what the other has to offer. Things like http://unity3d.com and http://opensimulator.org allow you to write stuff yourself . Tools like http://evolver.com give you easy ways into building characters etc. Also platforms are available http://smartfoxserver.com – used to delivered Club Penguin – and if you have <100 people connecting it is free to run.

Forza – racing game/driving sim – that allows you to mod the car – including paintwork etc. – and then when you race against someone they see your car. Can include things like logos etc…

Ian now demonstrating how he can run OpenSim on his laptop – his customised avatar wears digital version of the leather jacket he is currently wearing – identity and links between virtual and real. Can create virtual objects immediately – and all viewers of shared space see it immediately – you’ve distributed just be creating it. Ian talking about how he finds virtual objects as useful cues for talking – using ’3 dimensional’ cues for what he is going to say (reminds me of ‘palace of memories’ type stuff).

Ian says, a shared ‘space’ when presenting gives different effects and works well for some people – you can share presentations in the space, and also place discussion in the space.

Now moving onto Minecraft – much more game based, but lots of similarity with virtual worlds. You can run a Minecraft server yourself, or on the web. Starting to see some use of this in schools – mentions “Minecraft Teacher” http://minecraftteacher.net/. Ian describing how his children used Minecraft together for the first time – collaboration, exploration, building etc. Minecraft also allows building of mechanical devices – using things like trip switches, trains, etc.

Ian mentions Arduino and 3D printing as things he’s got onto the Cool Stuff program. Ian is especially enthusiastic about 3d printing – highlighting possibilities of moving between environments like Skylander – you could print out your own figures, with RFID chips in them….

Finally Ian closes by talking about how engaging children is about presenting this stuff in fun/interesting ways – but perhaps also about trusting children  will be interested and will learn, if you make it interesting.

Although I knew what SPARQL was, and what it looked like, I really hadn’t got to grips with it, and since I’d just recently purchased “Learning SPARQL” it seemed like a good opportunity to get familiar with the British Museum data and SPARQL syntax. So I had a play (more below). Skip forward a few months, and I noticed some tweets from a JISC meeting about the Pelagios project (which is interested in the creation of linked (geo)data to describe ‘ancient places’), and in particular from Mia Ridge and Alex Dutton which indicated they were experiementing with the British Museum data. My previous experience seemed to gel with the experience they were having, and prompted me to finally get on with a blog post documenting my experience so hopefully others can benefit.

Perhaps one reason I’ve been a bit reluctant to blog this is that I struggled with the data, and I don’t want this post to come across as overly critical of the British Museum. The fact they have their data out there at all is amazing – and I hope other museums (and archives and libraries) follow the lead of the British Museum in releasing data onto the web. So I hope that all comments/criticisms below come across as offering suggestions for improving the Museum data on the web (and offering pointers to others doing similar projects), and of course the opportunity for some dialogue about the issues. There is also no doubt that some of the issues I encountered were down to my own ignorance/stupidity – so feel free to point out obvious errors.

When you arrive at the British Museum SPARQL endpoint the nice thing is there is a pre-populated query that you can run immediately. It just retrieves 10 results, of any type, from the data – but it means you aren’t staring at a blank form, and those ten results give a starting point for exploring the data set. Most URIs in the resulting data are clickable, and give you a nice way of finding what data is in the store, and to start to get a feel for how it is structured.

For example, running the default search now brings back the triple:

Which is intriguing enough to make you want to know more (I am married, and have to admit I don’t remember any special equipment). Clicking on the URI http://collection.britishmuseum.org/id/object/EAF119772 in a browser takes you to an HTML representation of the resource – a list of all the triples that make statements about the item in the British Museum identified by that URI.

While I think it would be an exaggeration to say this is ‘easily readable’, sometimes, as with the triple above, there is enough information to guess the basics of what is being said – for example:

From this it is perhaps easy enough to see that there is some item (identified by the URI http://collection.britishmuseum.org/id/object/EAF119772) which has a note related to it stating that it was acquired (presumably by the museum) in 1994.

So far, so good. I’d got an idea of the kind of information that might be in the database. So the next question I had was “what kind of queries could I throw at the data that might produce some interesting/useful results?” Since I’d recently been playing around with data about composers I thought it might be interesting to see if the British Museum had any objects that were related to a well-known composer – say Mozart.

This is where I started to hit problems…. In my initial explorations, while some information was obvious, I’d also realised that the data was modelled using something called CIDOC CRM, which is intended to model ‘cultural heritage’ data. With some help from Twitter (including staff at the British Museum) I started to read up on CIDOC CRM – and struggled! Even now I’m not sure I’d say I feel completely on top of it, but I now have a bit of a better understanding. Much of the CIDOC model is based around ‘events’ – things that happened at a certain time/in a certain place. This means that often what might seem like a simple piece of information – such as where a item in the museum originates from – become complex.

To give a simple example, the ‘discovery’ of an item is a kind of event. So to find all the items in the British Museum ‘discovered’ in Greenwich you have to first find all the ‘discovery’ events that ‘took place at’ Greenwich, then link these discovery events back to the items they are a related to:

An item -> was discovered by a discovery event -> which took place at Greenwich

This adds extra complexity to what might seem initially (naively?) a simple query. This example was inspired by discussion at the Pelagios event mentioned earlier – the full query is:

SELECT ?greenwichitem WHERE
{
	?s <http://collection.britishmuseum.org/id/crm/P7F.took_place_at> <http://collection.britishmuseum.org/id/thesauri/x34215> .
	?subitem <http://collection.britishmuseum.org/id/crm/bm-extensions/PX.was_discovered_by> ?s .
	?greenwichitem <http://collection.britishmuseum.org/id/crm/P46F.is_composed_of> ?subitem
}

and the results can be seen at http://bit.ly/vojTWq.

To make things even more complex the British Museum data seems to describe all items actually as made up of (what I’m calling) ‘sub-items’. In some cases this makes some sense. If a single item is actually made up of several pieces, all with their own properties and provenance, it clearly makes sense to describe each part separately. Each part of the object will have it’s own properties and provenance, and it makes sense to describe these separately.

However, the British Museum data describes even single items as made up of ‘pieces’ – just that the single item consists of a single piece – and it is then that piece that has many of the properties of the item associated with it. To illustrate. A multi-piece item is like:

Which makes sense to me. But a single piece item is like:

Which I found (and continue to find) this confusing. This isn’t helped in my view by the fact that some properties are attached the the ‘parent’ object, and some to the ‘child’ object, and I can’t really work out the logic associated with this. For example it is the ‘parent’ object that belongs to a department in the British Museum, while it is the ‘child’ object that is made of a specific material. Both the parent and child in this situation are classified as physical objects, and this feels wrong to me.

Thankfully a link from the Pelagios meeting alerted me to some more detailed documentation around the British Museum data (http://www.researchspace.org/Stage-2-Outputs), and this suggests that the British Museum are going to move away from this model:

Firstly, after much debate we have concluded that preserving the existing modelling relationship as described earlier whereby each object always consists of at least one part is largely nonsense and should not be preserved.

While arguments were put forward earlier for retaining this minimum one part per object scheme, it has now been decided that only objects which are genuinely composed of multiple parts will be shown as having parts.

The same document notes that the current modelling “may be slightly counter-intuitive” – I can back up this view!

So – back to finding stuff related to Mozart… apart from struggling with the data model, the other issue I encountered was that it was difficult to approach the dataset through anything except a URI for a entity. That is to say, if you knew the URI for ‘Wolfgang Amadeus Mozart’ in the museum data set, the query would be easy, but if you only know a name, then it is much more difficult. How could I find the URI for Mozart, to then find all related objects?

Just using SPARQL, there are two approaches that might work. If you know the exact (and I mean exact) form of the name in the data, you can query for a ‘literal’ – i.e. do a SPARQL query for a textual string such as “Mozart, Wolfgang Amadeus”. If this is the exact for used in the data, the query will be successful, but if you get this slightly wrong then you’ll fail to get any result. A working example for the British Museum data is:

SELECT * WHERE
{
	?s ?p "Mozart, Wolfgang Amadeus"
}

The second approach you can use is to do a more general query and ‘filter’ the result using a regular expression. Regular expressions are ways of looking for patterns in text strings, and are incredibly powerful (supporting things like wildcards, ignoring case etc. etc.). So you can be a lot less precise than searching for an exact string, and for example, you might try to retrieve all the statements about ‘people’ and filter for those containing the (case insensitive) word ‘mozart’. While this would get you Leopold Mozart as well as Wolfgang Amadeus if both are present in the data, there are probably a small enough number of mozarts that you would be able to pick out WA Mozart by eye, and get the relevant URI which identifies him.

A possible query of this type is:

SELECT * WHERE
{
	?s <http://xmlns.com/foaf/0.1/Name> ?o
	FILTER regex(?o, "mozart", "i")
}

Unfortunately these latter type of ‘filter’ queries are pretty inefficient, and the British Museum SPARQL endpoint has some restrictions which mean that if you try to retrieve more than a relatively small amount of data at one time you just get an error. Since this is essentially how ‘filter’ queries work (retrieve a largish amount of data first, then filter out the stuff you don’t want), I couldn’t get this working. The issue of only being able to retrieve small sets of data was a bit of a frustration overall with the SPARQL endpoint, not helped by the fact that it seemed to be relatively arbitrary in terms of what ‘size’ of result set caused an error – I assume it is something about the overall amount of data retrieved, as it seemed unrelated to the actual number of results retrieved – for example using:

SELECT * WHERE
{
	?s ?p ?o
}

You can retrieve only 123 results before you get an error, while using

SELECT ?s WHERE
{
	?s ?p ?o
}

You can retrieve over 300 results without getting an error.

This limitation is an issue in itself (and the British Museum are by no means alone in having performance issues with an RDF triple store), but it is doubly frustrating that the limit is unclear.

The difficulty of exploring the British Museum data from a simple textual string became a real frustration as I explored the data – it made me realise that while the Linked Data/RDF concept of using URIs and not literals is something I understand and agree with, as people all we know is textual strings that describe things, so to make the data more immediately usable, supporting textual searches (e.g. via a solr index over the literals in the data) might be a good idea.

I got so frustrated that I went looking for ways of compensating. The British Museum data makes extensive use of ‘thesauri’ – lists of terms for describing people, places, times, object types, etc. In theory these thesauri would give the text string entry points into the data, and I found that one of the relevant thesauri (object types) was available on the Collections Link website (http://www.collectionslink.org.uk/assets/thesaurus/Objintro.htm). Each term in this data corresponds to a URI in the British Museum data, and so I wrote a ScraperWiki script which would search for each term in the British Museum data and identify the relevant URI and record both the term and the URI. At the same time a conversation with @portableant on twitter alerted me to the fact that the ‘Portable Antiquities‘ site uses a (possibly modified) version of the same thesaurus for classifying objects, so I added in a lookup of the term on this site to start to form connections between the Portable Antiquities data and the British Museum data. This script is available at https://scraperwiki.com/scrapers/british_museum_object_thesaurus/, but comes with some caveats about the question of how up to date the thesaurus on the Collections Link website is, and the possible imperfections of the matching between the thesaurus and the British Museum data.

Unfortunately it seems that this ‘object type’ thesaurus is the only one made publicly available (or at least the only one I could find), while clearly the people and place thesauri would be really interesting, and provide valuable access points into the data. But really ideally these would be built from the British Museum data directly, rather than being separate lists.

So, finally back to Mozart. I discovered another way into the data – which was via the really excellent British Museum website, which offers the ability to search the collections via a nice web interface. This is a good search interface, and gives access to the collections – to be honest already solving problems such as the one I set myself here (of finding all objects related to Mozart) – but nevermind that now!  If you search this interface and find an object, when the you view the record for the object, you’ll probably be at a URL something like:

http://www.britishmuseum.org/research/search_the_collection_database/search_object_details.aspx?objectid=3378094&partid=1&searchText=mozart&numpages=10&orig=%2fresearch%2fsearch_the_collection_database.aspx&currentPage=1

If you extract the “objectid” (in this case ’3378094′) from this, you can use this to look up the RDF representation of the same object using a query like:

SELECT * WHERE
{
	?s <http://www.w3.org/2002/07/owl#sameAs> <http://collection.britishmuseum.org/id/codex/3378094>
}

This gives you the URI for the object, which you can then use to find other relevant URIs. So in this case I was able to extract the URI for Wolfgang Amadeus Mozart (http://collection.britishmuseum.org/id/person-institution/39629) and so create a query like:

SELECT ?item WHERE
{
	?s ?p <http://collection.britishmuseum.org/id/person-institution/39629> .
	?item <http://collection.britishmuseum.org/id/crm/P46F.is_composed_of> ?s
}

To find the 9 (as of today) items that are in someway related to Mozart (mostly pictures/engravings of Mozart).

The discussion at the Pelgios meeting identified several ‘anti-patterns’ related to the usability of Linked Data – and some of these jumped out at me as being issues when using the British Museum data:

Anti-patterns

• homepages that don’t say where data can be found
• not providing info on licences
• not providing info on RDF syntaxes
• not providing egs of query construction
• not providing easy way to get at term lists
• no html browsing
• complex data models

On the first day of ChrisMash my true love sent to me, some cake and some coffee.

On the second day of ChrisMash my true love sent to me, two APIs and some cake and some coffee.

On the third day of ChrisMash my true love sent to me, three Google Maps, two APIs and some cake and some coffee.

On the fourth day of ChrisMash my true love sent to me, four RPCs, three Google Maps, two APIs and some cake and some coffee.

On the fifth day of ChrisMash my true love sent to me, five Yahoo Pipes. Four RPCs, three Google Maps, two APIs and some cake and some coffee.

On the sixth day of ChrisMash my true love sent to me, six RSS feeds, five Yahoo Pipes. Four RPCs, three Google Maps, two APIs and some cake and some coffee.

On the seventh day of ChrisMash my true love sent to me, seven homemade badgers, six RSS feeds, five Yahoo Pipes. Four RPCs, three Google Maps, two APIs and some cake and some coffee.

On the eighth day of ChrisMash my true love sent to me, eight mashers mashing, seven homemade badgers, six RSS feeds, five Yahoo Pipes. Four RPCs, three Google Maps, two APIs and some cake and some coffee.

On the ninth day of ChrisMash my true love sent to me, nine SPARQL queries, eight mashers mashing, seven homemade badgers, six RSS feeds, five Yahoo Pipes. Four RPCs, three Google Maps, two APIs and some cake and some coffee.

On the tenth day of ChrisMash my true love sent to me, ten QR codes, nine SPARQL queries, eight mashers mashing, seven homemade badgers, six RSS feeds, five Yahoo Pipes. Four RPCs, three Google Maps, two APIs and some cake and some coffee.

On the eleventh day of ChrisMash my true love sent to me, eleven MARC records, ten QR codes, nine SPARQL queries, eight mashers mashing, seven homemade badgers, six RSS feeds, five Yahoo Pipes. Four RPCs, three Google Maps, two APIs and some cake and some coffee.

On the twelfth day of ChrisMash my true love sent to me, twelve fancy cocktails, eleven MARC records, ten QR codes, nine SPARQL queries, eight mashers mashing, seven homemade badgers, six RSS feeds, five Yahoo Pipes. Four RPCs, three Google Maps, two APIs and some cake and some coffee.

I’m working with Evidence Base on the ‘+1’ project which is a support project looking to establish a ‘mobile library community’ and ways of supporting projects/libraries/people working in this area. The first step on this path is the m-libraries support website http://m-libraries.info where @joeyanne has posted some introductory material outlining what we mean by ‘mobile libraries’ in this context (this is about use of mobile devices by and for library services, rather than about ‘collections on wheels’).

Today is an opportunity to hear about some of the other projects and meet the people involved. However, the aim of our support project is not just to support the current projects, but to start a platform for a growing community.

Some quick introductions to the projects:

Phonebooth
Twitter: @jiscphonebooth
Lead: London School of Economics
Partners: Edina
Summary (from http://infteam.jiscinvolve.org/wp/2011/10/11/mobile-infrastructure-for-libraries-new-projects/): PhoneBooth will repurpose the Charles Booth Maps, Descriptive of London Poverty and selected police notebooks, which record eye-witness descriptions of London street-by-street, for delivery to mobile devices. The project will enhance the current online delivery by enabling content to be delivered directly to the location to which it refers.

Introducing this is Ed Fay (@digitalfay). Existing online resource – the Charles Booth maps (http://booth.lse.ac.uk/) which includes maps, classification and notebooks, at a street by street level. They will keep the backend infrastructure, but put a new mobile client interface on it.

Accessing data at a street level is something that already occurs in the teaching of a specific course – but lots of paper based use at the moment. The mobile delivery fits really well into this teaching. Also expect interest from schools, genealogists etc. Also talked to staff at the Museum of London (who hold some of the Booth maps)

Ed stressed that focus is delivery of library content on mobile – not delivering a ‘teaching app’

Going to be an open web app – more sustainable

‘Support of new mobile devices’ is written into new LSE library strategy.

M-Biblio
Hashtag: #mbiblio
Lead: University of Bristol
Summary (from http://infteam.jiscinvolve.org/wp/2011/10/11/mobile-infrastructure-for-libraries-new-projects/): The project will enhance the learning and research activities of the University of Bristol’s academic community by developing a mobile application that can record and organise references to books, journals and other resources. These references can be added actively by scanning barcodes and QR codes, or passively by automatically recording RFID tags in items being used for study and research.

Mike Jones (http://www.bris.ac.uk/ilrt/people/mike-a-jones/overview.html) introducing M-Biblio. Want to develop a mobile application – and with permission collect user activity data

Hope that the library gets useful data – for resources that might not usually be borrowed – like journals, theses, and other “reference only” resources
The staff and students get useful tool
Can trail ‘near field communication’ (NFC) capabilities of newer Android phones – to read RFID tags in books (wonder if this is compatible)
Maybe other technologies …

Will use a web service as a ‘broker’ which will connect between phone clients and bibliographic sources (e.g. their library catalogue – Aleph) and stats collection.

Employing two User Experience and User Interface Design experts to help with those aspects, and engaging users – staff and students – in the process.

MACON
Hashtag: #oumacon
Lead: The Open University
Partners: EBSCO
Summary (from http://infteam.jiscinvolve.org/wp/2011/10/11/mobile-infrastructure-for-libraries-new-projects/): MACON will address challenges involved in delivering quality academic content to mobile devices in a seamless and user-friendly manner. The project will work with EBSCO, a major content and systems provider in order to prototype a mobile friendly resource discovery interface which will discover and expose quality academic content from both third party & local collections.

Introduced by Keren Mills from the OU. Aim to create a mobile interface to the EBSCO discovery tool… – and interested in whether people use it, and how they want to use it. Do they want to read on the mobile device? Do they want to bookmark or save stuff for later? Probably a mix.

Library users are not necessarily (or usually!) expert searchers – so need to look at how can take basic queries and still return useful & relevant results.

Already know that authentication can be an issue on mobile devices – often get bounced around authentication systems via redirects – and after a certain number of redirects the mobile browser can give up (more quickly than on a desktop). Thinking about ways of storing some local user information (via a bookmarklet?) to shortcut some of this and improve user experience.

Outcomes they are looking for:

• Prototype discovery tool for delivery of academic content to mobile devices
• Document detailed user requirements
• Report on user feedback and usability, mapped to type of device used
• Release the code through a suitable code repository

Want to also look at possible delivery of audio-visual content (locally held material)

Want to avoid native apps – users might well be starting from the web (e.g. via the Open University VLE etc.)

Also want to make sure that if the user clicks on a link to a publisher site, they aren’t suddenly in a non-mobile friendly interface – that is, use a publisher’s mobile interface where possible.

Two further projects have been funded that aren’t represented here today, both based at City University:

MoPED
Lead: City University, London
Summary (from http://infteam.jiscinvolve.org/wp/2011/10/11/mobile-infrastructure-for-libraries-new-projects/): The project will develop the MoPED system, which will combine mobile phone interaction with a public display in City University’s Main Library. The aim of the project is to investigate how to encourage the adoption of mobile services through a two-fold strategy: first, a strong, user-centred design process, commencing with an investigation of which mobile services are most likely to be beneficial; second, using an in-situ public display to promote (and assist getting access to) the library’s mobile services and to connect online services to the space of the library itself.

Learnmore
Lead: City University, London
The project will develop the Learnmore Mobile Application using a user-centred design process. Building on the current ‘desktop’ Learnmore content, the interface and content will be tailored to the actual needs of students using mobile devices, with considerations including the preferred media, topic and content size for mobile consumption.

Does size matter?
Some discussions of ‘what is mobile’ – the OU is thinking of devices with smaller than 10” screens – as with iPads and similar devices the desktop interfaces work OK.

The PhoneBooth app is just not going to be usable below a certain size…

Suspect that the question of what aspects of a service or device that make it specifically ‘mobile’ is something that will come up again…

I believe that libraries, museums and archives are not getting the most out of the data about their collections, because they aren’t publishing in ways that enable or encourage others to take the data and use it in new, innovative (or even boring), ways. I think we need to offer data more ‘openly’.

Being open

(Image courtesy of withassociates, CC-BY-SA)

But what does ‘open’ mean in this context? For me, this is not a simple binary open/closed… but rather a continuum. There are a range of factors that affect whether others can easily take, and reuse, your data. But it is easy to focus on a single factor when talking about ‘openess’ – especially to focus on ‘rights’ to reuse data – copyright, database rights, licensing, terms and conditions etc. While these are an important factor, they are not the only factor.

Paul Walk puts it better than me in this slidedeck when he argues we need a ’richer understanding of openness’ which encompasses not just permissive licensing but, more broadly, the ease with which data can be used, taking into consideration aspects such as format and access mechanisms

Friction

I’ve started to think about factors affecting reuse as being causes of friction (an idea I’m pretty sure I got from Tony Hirst). This may not be an exhaustive list, but the things I can see that create friction in the reuse of data are:

• Explicit restrictions on reuse
• Uncertainty about possible restrictions on reuse
• Unusual or unfamiliar interfaces and formats (if you don’t work in the library world, you’ve probably never heard of Z39.50, and yet this is a standard machine to machine interface supported by many library systems)
• Lack of information on data and where the data is available

Sometimes you might deliberately introduce friction – perhaps you don’t want your data to be reused by just anyone, for any purpose. I don’t see friction as bad per se – we just need to be aware of it, and especially avoid introducing friction when we don’t mean to.

Oiling the wheels

There are clear steps that a library, archive or museum can take to ensure there is no unwanted ‘friction’ in the reuse of their data.

1. Apply clear licensing or terms on reuse.

As a signatory of the Discovery Open Metadata Principles, I believe descriptive metadata, such as that in library catalogue records, should be licensed as ‘public domain’ data (using CC0 or ODC-PDDL or equivalent).

However, if reuse is restricted for some reason, be clear about what those restrictions are. Commercial services like Twitter offer clear terms of use on their APIs - these are restrictive, but clear. Similary Wired magazine’s recent decision to offer images under Creative Commons BY-NC, while falling short of ‘open’ offers some level of clarity. In the latter case, the use of the ‘NC’ (Non-commercial) clause can lead to uncertainty about rights for reuse – as noted in this article.

The JISC Guide to Open Bibliographic Data might help inform decisions about licensing metadata, as may the Discovery licensing guide.

2. Adopt widely used (machine) interfaces and formats for data

While any access to machine readable data increases the opportunities for reuse, adopting widely used interfaces and formats – ones for which a wide range of code libraries and tools will be available, and which the development community will be familiar with. Currently this often boils down to offering an interface that delivers data in XML or JSON format over an http interface. Sometimes the term ‘RESTful API’ is used to describe this kind of interface, although it should be noted that in reality providing a RESTful interface is a bit more than just xml/json over http. This article tries to explain more specifically what REST is.

3. Document your APIs and your data

Whatever interfaces/APIs and data formats you support, leaving them undocumented immediately increases friction on reuse. Many of the systems libraries, museums and archives use provide some API, but these are very rarely clearly documented by the organisations using the systems. Without documentation, it’s a huge amount of work for a developer to work out how to interface with the system.

For example, my local public library uses the Aquabrowser interface to their catalogue, which supports a couple of APIs – but in order to use these I had to find out the details of the API from the University of Cambridge documentation, and then apply the details to the public library system. Even just pointing to documentation held elsewhere helps – and sends the message ‘we want you to use this API’ – and without this, the API will be left unused.

The data we deal with in libraries, museums and archives is specialist, and often confusing to those not familiar with the details – therefore not just documenting the APIs available, but also the data available via those APIs (this is also a reason to offer simple representations of data, as well as fuller, more complex, expressions as appropriate).

Finally, data needs to be ‘findable’ – how would a prospective user of your data know what data you have, and where to find an API for it? In Australia the Museum Metadata Exchange is an interesting model for making this information available, but there are also more general tools/sites like like http://thedatahub.org/ and http://getthedata.org/.

4. Use common identifiers

This probably seems less fundamental than the points above,  for me it is absolutely key. The point here is that if anyone wants to combine data together, common identifiers across data sets are what they will be looking for – and I’d argue this is going to be a pretty common use case for your data, or anyone elses, by a third party developer.

While it is possible to write code that tries to match strings like “Austen, Jane” in your data to http://viaf.org/viaf/102333412/, this is much more effort and much less precise than if a shared identifier was used from the start. It’s no surprise that if you look at many mashups created using bibliographic data they rely on the ISBN to match across different data sources (for example, pulling in cover images from Amazon, LibraryThing, Google Books or Open Library).

Supporting Discovery

Much of my thinking in this area has been informed by my work with the ‘Resource Discovery Taskforce‘ and with the Discovery initiativethat followed the work of the taskforce. Discovery is an initiative to improve resource discovery by establishing a clear set of principles and practices for the publication and aggregation of open, reusable, metadata. So far Discovery has published a set of Open Metadata Principles, and a set of draft Technical Principles, as well as running several events and a developer competition.

There will be a lot more coming out of the Discovery initiative over the next few months, and you can follow these via the Discovery Blog (which I occaisionally write for).

Outcomes of Open

Examples

Rufus Pollock, the Director of the Open Knowledge Foundation, said “The coolest thing to do with your data will be thought of by someone else” – but is this true? Perhaps obviously, it isn’t a given that anything will happen when you publish your data for reuse. However, there are now plenty of examples of interesting applications being built on data that has been published with reuse in mind. To just pick a few examples:

This iPhone app to search Cambridge University Library was developed by a postgraduate student – just because they wanted to learn how to develop an app using JSON, and found the API documentation published by the library.

This app allows the user to take a picture of a work of art using their smartphone, and then retrieves information about the item from Europeana – it was built as part of a ‘hackday’ for Europeana.

This novel interface to pictures from the National Archive was built as part of the Discovery Developer competition.

This map brings together information from English Heritage and the British National Bibliography to display location specific information.

… and finally to blow my own trumpet, this bookmarklet I’ve already written about

Supporting developments

Something I don’t feel I really understand yet is how data suppliers can best engage with developers who might build on their data. Emma Mulqueeny (@hubmum) has written eloquently about engaging developers, but I’m still not sure I fully understand the best way that an organisation such as a museum, library or archive can engage with the development community.

Except the Cambridge University Library iPhone app, all the examples above are the results of some explicit stimulus – a competition or hackday. I don’t think any of them can be described as ‘production level’ – they are, in general, proof of concept. If publishing data is going to result in sustainable developments, we need to consider how this is supported – should organisations ‘adopt’ applications or developers? Should they work with relevant organisations to realise some commercial benefit to the developers? Are there other approaches?

I’d say at the least provide somewhere for developers, and potential developers, to talk to you, ask you questions, get permission to try stuff out – that dialogue is at least the first step to something more sustainable.

Take action

After my presentation at ILI 2011, which covered much of the same ground as this blog post, I felt that perhaps I’d missed a key point, and an opportunity while I had an audience – the question of what they should do in light of what I was saying. So, not wanting to make the same mistake again, I would encourage, even exhort, you to take the following actions:

1. Explicitly license your data – whatever it is, put a license on it, be clear about what people can or can’t do with the data, and publish those details on your website
2. Find out about, and document, any APIs you already have to your data – it might be z39.50, it might be SRU/SRW, it might be some RSS feeds – whatever it is, write a short page that says where the API/data can be accessed, some basic instructions on how to use it. Be clear what you expect from people interacting with your data (both in terms of licensing – point 1 – and anything else like “please don’t kill our servers”)
3. Create a place for developers to communicate with you (or hang out somewhere that you can communicate them)

If you can’t do any of these things yourself, find out who can answer the questions, or make this happen – find out if they are interested, and if not, why not and what the barriers are (and then let me know!)

Kevin kicks off: Journalists and librarians dealing with many of the same issues – helping people navigate, interpret and understand information. Going to talk about some of the challenges in this area. First playing Xerox video on ‘information overload’ – http://www.youtube.com/watch?v=CXFEBbPIEOI

Eric Schmidt noted that we are now creating huge amounts of information (5 exabytes every 2 days is the quote, but see disagreement with this figure at http://www.readwriteweb.com/cloud/2011/02/are-we-really-creating-as-much.php)

Amount of time people spend on Facebook massively more than they spend on Newspaper web sites. Evidence that people are having problems moving to conclusions on complex stories – people move to simple narratives instead – Kevins says this equals “car crashes and celebreties”

Social media offers opportunity to re-engage people and help them navigate information.

We are moving from “mass” to “relevance” – e.g. not about how many followers you have on twitter, but about the relevance of what you post. Try to move from information overload (a ‘mass’ problem) and have filtered relevant information (a ‘relevance’ solution)

Social media provides a way of filtering information. But social media has to be ‘social’ – you need people at the heart of this.

Examples of crowdsourcing – Guardian analysis of MP expenses (http://mps-expenses.guardian.co.uk/), Ushahidi crowdsourcing crisis information (http://www.ushahidi.com/).

Kevin also mentions ‘entity extraction’ – uses Calais as an example..
Dewey D. – iphone app to manage ‘reading list’ (not in academic sense) and pulls in stories from the New York Times.

Poligraft – analyses funding of politicial campaigns – you can post URLs (of political stories) to Poligraft – it goes through and identifies politicians and organisations and shows you how politicians get campaign funding etc. Tells you about the major industries funding politicians etc – gives context to political story and help make sense of it.

We (journalists & librarians) have hundreds of years of doing things in a certain way – changing culture is incredibly difficult. If you have more than 5 people in the room, inertia hits …

Now Suw taking the floor… to talk crowdsourcing – breaking large tasks into smaller chunks that individuals can do. Suitable tasks – computational tasks and ‘human’ tasks.

Computational tasks = large datasets of computation that can be split into smaller datasets or computations – e.g. SETI@Home – this is about ‘spare cycles’ from individual’s computers they can contribute to computing power.

Human tasks = tasks that humans find easy but computers find difficult; brain driven; uses participants spare time; individual errors are average away by having the same task completed by many people.

Type of human tasks:

• Recognising and describing things in images
• Reading and transcribing writing
• Applying expertise to identify, sort and catalogue
• Collecting data
• Manipulating models

Examples …

PCF oil paintings tagger – http://tagger.thepcf.org.uk/

• Public catalogue foundation, BBC
• Digitising pictures
• Getting people to tag content with metadata – describe what is in the painting

“You don’t have to be an expert to take part”

Old Weather – http://www.oldweather.org/
Transcribing ships logs – contributes to historic data on climate, as well as other historical background

Ancient Lives – http://ancientlives.org/
Papyrus fragments – transcribe, measure, etc.

Multiple people doing each task gives you confidence when agreement across results

Herbaria@Home – http://herbariaunited.org/atHome/

What’s the Score – http://www.bodleian.ox.ac.uk/bodley/library/specialcollections/projects/whats-the-score
Digitised musical score collection from the Bodleian – will be starting crowdsourcing part of project soon

Why crowdsource?
Provide opportunities for education and knowledge maintenance
Most projects don’t require prior knowledge but people often enjoy learning more about a subject
Improve accessibility through addition of new metadata or improvement of existing metadata – create data for research
Even when digitised, collections are hard to search/comprehend

Galaxy Zoo shows public were as good, or better, than professionals at classifying galaxies
FoldIt found gamers could solve the structure of a protein that causes AIDs in rhesus monkeys in three weeks

Are your projects suitable?

• Can the original material be digitised?
• Can task be broken down into small chunks?
• Can those chunks be done by humans or their computers?

It also helps if…

• There is a benefit for the public – example of Google buying out a image tagging game, which then died
• People feel part of a community
• There are measurable goals and targets

Zooniverse are crowdsourcing gurus..
Citizen Science Alliance – “Science” doesn’t just mean science – looking for projects at the moment…
Events – e.g. Citizen Cyberscience Summit

Q & A:
Failure of crowdsourcing – NASA mapping craters on Mars – mid 80s. But failed to collect data in useful way.
In terms of issues around the data
Wikitorial – not enough community – hurdles to participation not a bad thing