{"id":1678,"date":"2014-11-03T17:25:51","date_gmt":"2014-11-03T16:25:51","guid":{"rendered":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/?p=1678"},"modified":"2014-11-03T17:48:06","modified_gmt":"2014-11-03T16:48:06","slug":"tilt-text-to-image-linking-tool","status":"publish","type":"post","link":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/2014\/11\/tilt-text-to-image-linking-tool\/","title":{"rendered":"TILT: Text to Image Linking Tool"},"content":{"rendered":"

This blog post was written during a presentation at the British Library Labs Symposium<\/a> in November 2014. It is likely full of errors and omissions having been written real-time.<\/p>\n

Why this tool?<\/p>\n

Libraries contain thousands of literary & documentary artefacts up to 4000 years old. How to bring these effectively to a modern audience.<\/p>\n

“Images of their own are dull” – browse interfaces tend not to give the user much information. Even at the page image level, it can be difficult to make sense of what you are seeing.<\/p>\n

One approach is to put the text on top of an image:
\n* Correltates words in image\/text
\n* can be searched but…
\n* only works with OCR
\n* if text has errors, hard to fix
\n* text can’t be formattted or annotated<\/p>\n

Another approach is to put the text next to the image:
\n* format text for different devices
\n* can annotate test for stufy
\n* easy to verify and edit
\n* BUT
\n* must keep image and text in sync
\n* increases mental effort to find corresponding words in the text\/image<\/p>\n

If you are going to link text to the image of the text, what level should you do this at?
\n* ilink at page-level – useful but too coarse. Doesn’t reduce mental effort much
\n* link at line level
\n* link at word level<\/p>\n

Word level probably most desirable, but how to achiev it?
\nManual approach:
\n* Manually draw shapes around words
\n* link them to the text by adding markup to the transcription
\n* BUT
\n* tedious & expensive
\n* markup gets complex
\n* end up needing multiple transcriptions<\/p>\n

TILT approach:
\n* find word in an image without reconginsing their content
\n* Use an exsiting transcripto f the page content
\n* Link these two component mostly automatically<\/p>\n

Design:<\/p>\n

TILT Service
\n^ ^ ^
\nImage Text or HTML GeoJSON
\n^ ^ v
\nTILT web-based GUI<\/p>\n

First you have to prepare image in GUI – identify different parts of the text
\nStages:
\n* Colour to greyscale
\n* Greyscale to Black and White
\n* Find lines
\n* Find word shapes
\n* link word-shapes to text<\/p>\n

Recognising words is a challenge:
\n* (in most languages) Words are blocks of connected pixels with small gaps between them
\n* But if there are 300 words on a page are the 299 largest gaps always between words<\/p>\n

How to represent word shapes? Simple polygons do the trick<\/p>\n

Measure width of words in text, and then tries to match against lengths in transcription – so if the word shapes have not been recognised correctly, the matching algorithm just selects more or less text in the transcription.<\/p>\n

Now looking at using of ‘anchor points’ in text that allows the user to identify the start and end of ‘clean’ text in a larger manuscript which might have messy sections that can’t be done automatically. This allows you do what you can automatically, and only deal with the messy bits in a manual way.<\/p>\n

Still working on a GUI to work with<\/p>\n

Code on GitHub<\/p>\n","protected":false},"excerpt":{"rendered":"

This blog post was written during a presentation at the British Library Labs Symposium in November 2014. It is likely full of errors and omissions having been written real-time. Why this tool? Libraries contain thousands of literary & documentary artefacts up to 4000 years old. How to bring these effectively to a modern audience. “Images […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[99],"class_list":["post-1678","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-bl_labs"],"_links":{"self":[{"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/posts\/1678","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/comments?post=1678"}],"version-history":[{"count":3,"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/posts\/1678\/revisions"}],"predecessor-version":[{"id":1698,"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/posts\/1678\/revisions\/1698"}],"wp:attachment":[{"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/media?parent=1678"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/categories?post=1678"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.meanboyfriend.com\/overdue_ideas\/wp-json\/wp\/v2\/tags?post=1678"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}