Conceptual and content-based annotation of (multimedia) documents

This paper focuses on the techniques used in an NKRL environment (NKRL = Narrative Knowledge Representation Language) to deal with a general problem affecting the so-called “semantic/conceptual annotations” techniques. These last, mainly ontology-based, aim at “annotating” multimedia documents by representing, in some way, the “inner meaning/deep content” of these documents. For documents of sufficient size, the content modeling operations are separately executed on ‘significant fragments’ of the documents, e.g., “sentences” for natural language texts or “segments” (minimal units for story advancement) in a video context. The general problem above concerns then the possibility of collecting all the partial conceptual representations into a global one. This integration operation must, moreover, be carried out in such a way that the meaning of the full document could go beyond the simple addition of the ‘meanings’ conveyed by the single fragments. In this context, NKRL makes use of second order knowledge representation structures, “completive construction” and “binding occurrences”, for collecting within the conceptual annotation of a whole “narrative” the basic building blocks corresponding to the representation of its composing elementary events. These solutions, of a quite general nature, are discussed in some depth in this paper. This last includes also a short “state of the art” in the annotation domain and some comparisons with the different methodologies proposed in the past for solving the above ‘integration’ problem.