Mendeley and APIs

Now Ian Mulvany talking about Mendeley and how they use APIs – both publishing and consuming.

Try to expose all the metadata being added by users via an API – a “social catalogue”. This enables ‘discovery’, but not ‘delivery’ – this is where Mendeley can make use of external APIs – such as the WorldCat API.

Mendeley invest in APIs because

  • It helps them extend their product, but integrating data/functionality from other places
  • It enables others to extend their product – they don’t have time to build everything that users are asking for. E.g. Android client built by users, as company didn’t have the resource

Mendeley uses WorldCat registry to find/suggest appropriate OpenURL resolver depending on users location – as most users won’t know what an OpenURL resolver is, or what the detail are.

Mendeley uses OAuth – which means they can integrate with institutional repositories and a users own publications in Mendeley – going to be live soon (working with JISC, Symplectic and University of Cambridge on this – http://jisc-dura.blogspot.com/). Learnt a lot about consuming their own APIs in this project – and uncovered bugs…

“We should have built the API first, and the product second” – the fact they didn’t now creating work. Now they are creating a new application for libraries, and building API first. Ian firmly believes this is a better approach.

Ian’s top 10 tips for API provision:

  • first API, then app
  • use your own APIs (and he believes Mendeley should do this more)
  • make an (API) interface you would use yourself
  • provide lots of example docs – coders like to do stuff quickly – if they can get something working from an example quickly, they’ll then invest
  • version your API – backwards compatible
  • put rate limits in place
  • work with a 3rd party to provide keys
  • have clear licensing
  • engage with your community
  • promote, promote, promote, promote, promote

In terms of consuming:

  • know what you want to do
  • define the value – this may be service delivery, or could be development of skills for developers etc.
  • measure the value – otherwise difficult to prioritise future developments
  • understand the SLA
  • if it’s important – have a backup plan – dependence on 3rd party is a risk which you should manage
  • don’t wait on API for page loads – found that Mendeley homepage was waiting for a response from an API was down, and so the page didn’t load…
  • get on the mailing list/dev group
  • look for good example code
  • don’t be afraid to pay – if it’s important, it’s worth paying for
  • use Mendeley’s APIs 😉

 

 

Citavi and APIs

I’m at the OCLC EMEARC meeting today, talking and hearing about APIs. Having done my bit at the start, now trying to relax into the other presentations before questions and general discussion at the end.

Now Antonio Tejada and Hans-Siem Schweiger are talking about Citavi which combines Reference Management and Knowledge Organisation. Citavi designed to help with searching, retrieving results, acquiring materials – all of these require interaction with library sources. Citavis supports adding data manually, from file upload, browser extensions and via APIs.

Manual entry is error prone and time consuming

File upload – uses standard formats (RIS/BibTeX); supported by a wide range of catalogues and databases; but still time consuming

Browser Extension – e.g. looks for embedded metadata in the page (e.g. COinS) or find standard identifiers in the page (e.g. ISBN) and import data

APIs – eliminates the browser – you don’t need to go to lots of different sources on the web. Fastest mechanism. Direct. Integrates in the workflow much better. However cost of implementation can vary quite a bit – it all depends on the API – some very fast (e.g. z39.50 can do in minutes now, but custom APIs can be more difficult)

Citavi Features which use APIs:

  • Online search – integrated into the Citavi application
  • Retrieve by identifier (e.g. DOI, ISBN, PubMed ID)
  • Import formatted bibliography – can take a bibliography from a word file and Citavi will run a search for each item in the bibliography
  • Find Library Locations
  • Find Full Text
  • Check availability with OpenURL (seems like this actually just pushes user to their local resolver?)
Citavi supports a proxy service for some resources, when needed. E.g. for WorldCat API where an API key is required.

Universities can get site license for Citavi – allows library to create a special settings file with authentication details for databases (that are not IP authenticated)

Challenges for Citavis using APIs:

  • Administrative challenges
    • Some libraries don’t want to be accessible (at least via a desktop application)
    • Catalogues that charge by the record for metadata
    • Inconsistent communication – e.g. change of settings on library system, don’t inform Citavi
  • Technical challenges
    • Custom catalog software – missing or inconsistent standards support and inconsistent field mapping
    • Legacy data – not as well-structured; inconsistent data entry

Going forward:

  • Geographic search (WorldCat Search API)
  • Enhanced availability search (WorldCat Registry and OpenURL Gateway)
  • Acquisitions management (WMS Acquisitions) – Citavi didn’t anticipate this, but some libraries using Citavi to manage acquisitions processes
  • Metadata – looking at authority control (WorldCat Identities); Alternate editions (xISBN); ISSN lookup (xISSN)

CETIS Conference: Learning Registry show and tell

First up Scott Wilson (@scottbw) describing potential use of learning registry to bring together ‘paradata’ (activity/usage data) for ‘widgets’ (or apps) across different widget (app) stores – the idea that you could have the same app in different stores, and want to aggregate the reviews or ratings from each store. Put in bid to JISC under rapid innovation call for a project ‘SPAWS’…

Terry McAndrew (from JISC TechDis) – want to network experience with resources – identify accessible practice/purposes. Terry says most ‘OER Problems’ are social not technical. Asks – can we find learning registry output via Google?

Walt Grata showing tools that he has built on top of Learning Registry … (on github):

  • ‘Landing pages’ for content – that can be indexed via Google (think this is new, and not up at github yet)
  • Harvesting tool – to grab stuff from a node and put into another storage mechanism – e.g. couchdb,  postgresql, etc etc

Pat Lockley – slides

  • Chrome plugin, code on github
    • No-one will search outside Google – so take learning registry to Google. Chrome plugin finds all links on the page, and checks each one on the learning registry – and looks for some common attributes – like ‘title’ or ‘description’ etc. – and can then manipulate browser display to make use of this data.
  • WordPress Widget  – code on github
    • plugin for WordPress to display content from a learning registry (node) in a wordpress blog

Steven Cook – used Cake (PHP framework) to extract and ‘slice’ data from learning registry node. Also pulling data from other sources – like Topsy. Code on github. Talking about how can’t expect Learning Registry to do the hard work here – have to expect to pull out data, cache it, etc. Notes learning registry API isn’t completely RESTful (? not sure what the issues are) .

CETIS Conference: Capturing Conversations About Learning Resources

This session is really why I’ve come to the CETIS Conference (apart from the general opportunity to meet and chat to people which is also great) – it’s about “The Learning Registry” (@learningreg and http://learningregistry.org). The Learning Registry is not a destination – it’s about building infrastructure – and in some ways has both parallels and relevance to the work the Discovery programme is undertaking (which I’m involved in).

A simple use case for the learning registry is:

  • Nasa publishes a physics video
    • PBS posts a link to the video
    • NSDL posts a link to the video
    • A school uses the video in a course in their Moodle VLE

Each place/portals where the link is used or published only knows about use of their link or copy of the resource. So Learning Registry aims to support way of sharing this type of activity ‘in the open’ – so that this can be captured and reflected – the ‘social metadata timeline’ – Learning Registry is to provide infrastructure to support this. Learning Registry describes this type of activity/usage data associated with a resource ‘paradata’ – although learning registry learning registry doesn’t care what type of data it stores (as long as it can be expressed as JSON)

The learning registry is “an idea, a research project, an open source community project, a public social metadata distribution network’…

The guiding principles: be enabling, capability not solutions, no barrier entry, no single point of failure – everything distributed…

Not going to try to blog the technical architecture of.. but summary of APIs

  • Distribute API – uses http POST. About copying data from one node to another – i.e. achieving the distributed part of the architecture
  • Publish API – how you get stuff into a Learning Registry node (that is, you, the producer of information, publish it *to* the learning registry node) – uses http POST. Learning reg also supports SWORD for publishing data into a node
  • Obtain API – getting data out of a learning registry node – uses http GET
  • Harvest and OAI-PMH APIs – another way of getting stuff out of the node. Harvest returns JSON but supports OAI-PMH type actions. OAI-PMH also supported.

It is stressed that this is really a project at the start of its work – the way to engage and to find out how to do this stuff is to join the community – join the developer list etc. and raise issues, ask questions – this is part of the experiment and will inform the development.

JLern

JLern is the project to setup an experimental node in the UK – being run by Mimas.

2 kinds of nodes in the Learning Registry:

  • Common node
  • Gateway node

JLern have setup a ‘common node’ – this can support:

  • Publish services
  • Access services
  • Distribution services (JLern now have a 2nd common node up and running to try these)
  • ….

Common nodes can be part of ‘networks’. Networks can (only) be connected via ‘gateway nodes’

When networks are connected, this is called a ‘community’. A ‘network community’ is a collection of interconnected resource distribution networks. A resource network can only be a member of one community.

Now have published the JORUM metadata (via OAI-PMH) – so about 15k resources. Open University now looking at similar activity. Now Jorum exploring framework for capturing paradata about resources.

Gathering ideas and use cases now – e.g. see JLern challenge from dev8D http://dev8d.org/challenges/

The JLern ‘Alpha’ node is at alpha.mimas.ac.uk – you can authenticate using details given in this blog post http://jlernexperiment.wordpress.com/2012/02/02/alpha-node/.

As already mentioned they’ve harvested JORUM OAI-PMH data and published on JLern alpha node.

They now have ‘Beta’ node (this doesn’t represent a level of development – just naming convention I think) – this is running on Windows (Alpha is on Linux). Also planning a ‘Gamma’ node running on Amazon EC2.

JLern hackday held in January – write up at http://jlernexperiment.wordpress.com/2012/02/21/the-hackday-report-and-reflections/, and also a Java Library for interacting with Learning Registry nodes at https://github.com/navnorth/LRJavaLib .

 

CETIS Conference: Bring on the metaverse

Ian Hughes (@epredator) from Feeding Edge ltd. Ian describes himself as a ‘metaverse evangelist’. Ended up presenting a section on ‘Cool Stuff’ on CITV – and finding he was talking about exactly the same stuff to talk to children as he’d been talking to the commercial sector/large corps about. Tries to include mentions of open source, and show that children can get involve and affect stuff … in the way he did when he was young.

A lot of the stuff Ian talks about comes back to games – perhaps because about playing and about building – he went into computing because he wanted to build games. Not just about writing games – but using toolkits to mod characters, game play etc.

Also interest in animation (involved in a BCS group about animation) – and this is about art and technical skill – you need to bring together people with these different skill sets, and each needs to understand what the other has to offer. Things like http://unity3d.com and http://opensimulator.org allow you to write stuff yourself . Tools like http://evolver.com give you easy ways into building characters etc. Also platforms are available http://smartfoxserver.com – used to delivered Club Penguin – and if you have <100 people connecting it is free to run.

Forza – racing game/driving sim – that allows you to mod the car – including paintwork etc. – and then when you race against someone they see your car. Can include things like logos etc…

Ian now demonstrating how he can run OpenSim on his laptop – his customised avatar wears digital version of the leather jacket he is currently wearing – identity and links between virtual and real. Can create virtual objects immediately – and all viewers of shared space see it immediately – you’ve distributed just be creating it. Ian talking about how he finds virtual objects as useful cues for talking – using ‘3 dimensional’ cues for what he is going to say (reminds me of ‘palace of memories’ type stuff).

Ian says, a shared ‘space’ when presenting gives different effects and works well for some people – you can share presentations in the space, and also place discussion in the space.

Now moving onto Minecraft – much more game based, but lots of similarity with virtual worlds. You can run a Minecraft server yourself, or on the web. Starting to see some use of this in schools – mentions “Minecraft Teacher” http://minecraftteacher.net/. Ian describing how his children used Minecraft together for the first time – collaboration, exploration, building etc. Minecraft also allows building of mechanical devices – using things like trip switches, trains, etc.

Ian mentions Arduino and 3D printing as things he’s got onto the Cool Stuff program. Ian is especially enthusiastic about 3d printing – highlighting possibilities of moving between environments like Skylander – you could print out your own figures, with RFID chips in them….

Finally Ian closes by talking about how engaging children is about presenting this stuff in fun/interesting ways – but perhaps also about trusting children  will be interested and will learn, if you make it interesting.