This session by Phil John – Technical Lead for Prism (was Talis, now Capita). Prism is a ‘next generation’ discovery interface – but built on Linked Data through and through.<\/p>\n
Slides available from http:\/\/www.slideshare.net\/philjohn\/linked-library-data-in-the-wild-8593328<\/a><\/p>\n Now moving to next phase of development – not going to be just about library catalogue data – but also journal metadata; archives\/records (e.g. from the CALM archive system); thesis repositories; rare items and special collections (often not done well in traditional OPACs) … and more – e.g. community information systems.<\/p>\n When populating Prism from MARC21 – do initial ‘bulk’ conversion, then periodic ‘delat’ files – to keep in sync with LMS. Borrower and availability data is pulled from LMS “live” – via a suite of RESTful web services.<\/p>\n Prism is also a Linked Data API… just add .rss to collection of .rdf\/.nt\/.ttl\/.json to items. This means simple to publish RSS feeds of preconfigured searches – e.g. new stock, or new stock in specific subjects etc.<\/p>\n Every HTML page in Prism has data behind it you can get as RDF.<\/p>\n One of the biggest challenges – Extracting data from MARC21 – MARC very rich, but not very linked… Phil fills the screen with #marcmustdie tweets \ud83d\ude42<\/p>\n But have to be realistic – 10s of millions of MARC21 records exist – so need to be able to deal with this. Feel they have a robust solution that performs well – even if it only takes 10 seconds to convert a MARC record – then when you use several million records it takes months.<\/p>\n No matter what MARC21 and AACR2 says – you will see variations in real date.<\/p>\n Have a conversion pipeline: First area they tackled was Format (and duration) – good starting point as it allows you to reason more fully about the record – once you know Format you know what kind of data to expect.<\/p>\n In theory should be quite easy – MARC21 has lots of structured info about format – but in practice there are lots of issues:<\/p>\n E.g. places to look for format information: Decided to do the duration at the same time: Now Phil talking about ‘Title’ – v important, but of course quite tricky… Retained a ‘statement of responsibility’ – but mostly for search and display…<\/p>\n Identifiers… Author – Pseudonyms, variations in names, generally no ‘relator terms’ in 100\/700 $$e or $$4 – which would show the nature of the relationship between the person and the work (e.g. ‘author’ ‘illustrator’) – because these are missing have to parse information out of the 245$$c<\/p>\n … and not just dealing with English records – especially in academic libraries.<\/p>\n Have licensed Library of Congress authority files – which helps… – authority matching requirements were: So – store Authorities as RDF but index in SOLR – gives speed and for bulk conversions don’t get http overhead…<\/p>\n Language\/Alternate representation – this is a nice ‘high impact’ feature – allows switching between representations – both forms can be searched for – use RDF content language feature – so useful for people using machine readable RDF<\/p>\n Using and Linking to external data sets… Potential solutions (not there yet):<\/p>\n Want to see Questions?<\/strong> Is there a running instance of Prism to play with:
\nDecided to tackle problem in small chunks. Created a solution that allows you to build a model interatively. Also compartmentalises code for different sections – these can communicate but work separately and can be developed separately. Makes it easy to tweak parts of the model easily.<\/p>\n
\nParser – reads in MARC21- fires events as it encounters different parts of the record – it’s very strict with Syntax – so insists on valid MARC21
\nObserver – listens for MARC21 data structures and hands control over to …
\nHandler – knows how to convert MARC21 structures and fields into Linked data<\/p>\n\n
\n007
\n245$$h
\n300$$a (mixed in with other info)
\n538$$a<\/p>\n
\n306$$a
\n300$$a (but lots of variation in this field)<\/p>\n
\n245 field in MARC may duplicate information from elsewhere
\nGot lots of help from http:\/\/journal.code4lib.org\/articles\/3832<\/a> (with additional work and modification)<\/p>\n
\nLots of non identifier information mixed in with other stuff – e.g. ISBN followed by ‘pbk.’
\nMany variations in abbrevations used – have to parse all this stuff, then validate the identifier
\nOnce you have an identifier, you start being able to link to other stuff – which is great.<\/p>\n
\nHas to be fast – able to parse 2M records in hours not days\/months
\nHas to be accurate<\/p>\n
\npart of the reason for using linked data – but some challenges….<\/p>\n\n
\n
\n<\/strong>More library data as LOD – especially on the peripheries – authority data, author information, etc.
\nLMS vendors adopting LOD
\nLOD replacing MARC21 as standard representation of bibliographic records!<\/p>\n
\nIs process (MARC->RDF) documented?
\nA: Would like to open source at least some of it… but discussions to have internally in Capita – so something to keep and eye on…<\/p>\n
\nA: Yes – e.g. http:\/\/prism.talis.com\/bradford\/<\/a><\/p>\n