Image Annotation using common XML namespaces

Posted on January 11, 2008

0


This will outline methods for holding and describing the annotations of regions of images. I place emphasis on the work region as that is what I feel most people mean when they talk about adding metadata to an image; usually, they are more interested with the things in the photo, rather than the whole photo itself.

I’ll give a few examples of what I mean:

A researcher wishes to indicate a particular architectural feature in some photographs of an archaeological site and indicate the same feature in a floorplan diagram of the same site.

A researcher cataloguing photos of insect life in the rainforest needs to be able to note the species of a particular insect in the image, who identified that particular insect and what plant/animal it is seen to be interacting with/eating/etc in this image – all of which pertain to an individual insect in the photo, although there may be one or more animals in the frame of the picture.

A foreign language book has been scanned in, and there needs to be a mechanism to relate each translated sentence and commentary to the relevant section of the image to which it applies.

A collection of ephemera from the 1900’s (bus tickets, etc) have certain items which are compound items – a display case for a set of cigarette cards for example – and it is necessary to relate the location of the individual items to the original – locations on the display case continuing the example.

At first glance, it looks like we need something that can encode some metadata defining a 2-D shape and also encode the shape’s relative positioning on the surface of an image. We will also need to be able to link arbitrary metadata to the individual shapes – arbitrary as we need to be flexible in terms of what information is recorded:

Region metadata –(links to)–> Region –(links to)–> Image

I knew I couldn’t have been the first person to consider this, so I asked around. Benjamin Nowack (bengee) on the irc://irc.freenode.net/swig channel was very helpful and pointed me at the w3photo project which arrived at an ontology for defining regions of images:

Namespace for the ontology
Image Region Vocabulary: RDFS+OWL Documentation

[NB From what I’ve seen, most of the effort and work has gone into defining and producing metadata for the entire image, so I’ll assume that a good image ontology/metadata format has been developed (I’m counting on the exif namespace myself, but a better one may appear.)]

So, to cut to the chase, this is a piece of RDF, hoping giving an example of what it is I am considering:


<rdf:RDF xmlns:foaf="http://xmlns.com/foaf/0.1/"
xmlns:imreg="http://web-semantics.org/ns/image-regions#"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:exif="http://www.w3.org/2003/12/exif/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">

<foaf:Image rdf:about=
"http://ora.ouls.ox.ac.uk/objects/uuid%3Aeb709f3b-a340-48bc-90d8-c8aa54dd7a07/datastreams/IMAGE01">
<exif:orientation>top-left</exif:orientation>
<exif:width>2048px</exif:width>
<exif:height>2048px</exif:height>
<!-- insert misc image metadata here, concerning the entire image. -->
<imreg:hasRegion>
<imreg:Rectangle>
<imreg:coords>1002,920 1100,1035</imreg:coords>
<dc:description>Location of the 1st burial site.</dc:description>
<dc:creator>Ben O'Steen </dc:creator>
<dc:date>circa 200BC </dc:date>
<dc:identifier>uuid:b46d3321-e8fd-41c3-81e4-793a6ab40e68</dc:identifier>
</imreg:Rectangle>
</imreg:hasRegion>
<imreg:hasRegion>
<imreg:Rectangle>
<imreg:coords>1600,1200 1750,1300</imreg:coords>
<dc:description>Location of the 2nd burial site.</dc:description>
<dc:creator>Ben O'Steen </dc:creator>
<dc:date>circa 200BC </dc:date>
<dc:identifier>uuid:7bc22dad-ff1d-45ad-86eb-dd24c1d0964a</dc:identifier>
</imreg:Rectangle>
</imreg:hasRegion>
</foaf:Image>
</rdf:RDF>

Combine that with a decent web interface for making these annotations, and a backend sophisticated to understand these user-made ‘comments’ on the image and you have yourself an exciting system, I feel. (I am using the Fedora RDF relationship of ‘isAnnotationOf’ to mark these connections between comment and original.)

(Oh, and for a good GPL’d DHTML method for drawing rectangles on an image, check out:
SpeedingRhino’s Cropper tool, a working demo of which is here.

Advertisements
Posted in: Uncategorized