Swws

Adding Multimedia to the Semantic Web
- Building an MPEG-7 Ontology
Jane Hunter
DSTC Pty Ltd
University of Qld, Australia
jane@dstc.edu.au

Abstract. For the past two years the Moving Pictures Expert Group (MPEG), a working group of ISO/IEC, have
been developing MPEG-7 [1], the "Multimedia Content Description Interface", a standard for describing
multimedia content. The goal of this standard is to develop a rich set of standardized tools to enable both humans
and machines to generate and understand audiovisual descriptions which can be used to enable fast efficient
retrieval from digital archives (pull applications) as well as filtering of streamed audiovisual broadcasts on the
Internet (push applications). MPEG-7 is intended to describe audiovisual information regardless of storage,
coding, display, transmission, medium, or technology. It will address a wide variety of media types including: still
pictures, graphics, 3D models, audio, speech, video, and combinations of these (e.g., multimedia presentations).
MPEG-7 is due for completion in October 2001. At this stage MPEG-7 definitions (description schemes and
descriptors) are expressed solely in XML Schema [2-4]. XML Schema has been ideal for expressing the syntax,
structural, cardinality and datatyping constraints required by MPEG-7. However it has become increasingly clear
that in order to make MPEG-7 accessible, re-usable and interoperable with other domains then the semantics of
the MPEG-7 metadata terms also need to be expressed in an ontology using a machine-understandable language.
This paper describes the trials and tribulations of building such an ontology represented in RDF Schema [5] and
demonstrates how this ontology can be exploited and reused by other communities on the semantic web (such as
TV-Anytime [6], MPEG-21 [7], NewsML [8], museum, educational and geospatial domains) to enable the
inclusion and exchange of multimedia content through a common understanding of the associated MPEG-7
multimedia content descriptions.

1. Introduction

Audiovisual resources in the form of still pictures, graphics, 3D models, audio, speech, video
will play an increasingly pervasive role in our lives, and there will be a growing need to enable
computational interpretation and processing of such resources. Forms of representation that
will allow some degree of machine interpretation of audiovisual information’s meaning will be
necessary [27]. The goal of MPEG-7 [1] is to support such requirements by providing a rich
set of standardized tools to enable the generation of audiovisual descriptions which can be
understood by machines as well as humans and to enable fast efficient retrieval from digital
archives (pull applications) as well as filtering of streamed audiovisual broadcasts on the
Internet (push applications).
The main elements of the MPEG-7 standard are:
• Descriptors (D), representations of Features, that define the syntax and the semantics of
each feature representation;
• Description Schemes (DS) that specify the structure and semantics of the relationships
between their components. These components may be both Descriptors and Description
Schemes;

• A Description Definition Language (DDL) to allow the creation of new Description
Schemes and, possibly, Descriptors and to allows the extension and modification of
existing Description Schemes;
• System tools, to support multiplexing of descriptions, synchronization of descriptions with
content, transmission mechanisms and coded representations (both textual and binary
formats) for efficient storage and transmission, management and protection of intellectual
property in MPEG-7 descriptions.

XML Schema language has been chosen as the DDL [9] for specifying MPEG-7
descriptors and description schemes because of its ability to express the syntactic, structural,
cardinality and datatyping constraints required by MPEG-7 and because it also provides the
necessary mechanisms for extending and refining existing DSs and Ds. However it has
recently become increasingly clear that there is also a need for a machine-understandable
representation of the semantics associated with MPEG-7 DSs and Ds to enable the
interoperability and integration of MPEG-7 with metadata descriptions from other domains.
New metadata initiatives such as TV-Anytime [6], MPEG-21 [7], NewsML [8], and
communities such as the museum, educational, medical and geospatial communities, want to
combine MPEG-7 multimedia descriptions with new and existing metadata standards for
simple resource discovery (Dublin Core [10]), rights management (INDECS [11]), geospatial
(FGDC [12]), educational (GEM [13], IEEE LOM [14]) and museum (CIDOC CRM [15])
content, to satisfy their domain-specific requirements. In order to do this, there needs to be a
common understanding of the semantic relationships between metadata terms from different
domains. XML Schema provides little support for expressing semantic knowledge. RDF
Schema provides us with a way to do this.
The Resource Description Framework (RDF) [16] is the accepted language of the
semantic web due to its ability to express semantics and semantic relationships through class
and property hierarchies. In this paper, we investigate the feasibility of expressing the
semantics of MPEG-7 Descriptors (Ds) and Description Schemes (DSs) in an RDF Schema [5]
ontology. An earlier paper evaluated RDF Schema for video metadata representation (prior to
the development of MPEG-7) and determined a number of limitations [23]. In this paper we
hope to ascertain whether those limitations still exist when representing the semantics of
MPEG-7 DSs and Ds or whether they can be overcome – either by using the extra constraints
provided by DAML+OIL [17] or through combining RDF Schema semantics with XML
Schema encoding specifications in a complementary manner.
Whilst manually building the RDF Schema for a core subset of MPEG-7, we also hope to
be able to recognize patterns and hence determine automatic mechanisms for generating
compatible RDF Schema definitions corresponding to the complete set of MPEG-7 XML
Schema definitions.
In Section 2 we describe the methodology, problems encountered and results of building
an RDF Schema ontology for MPEG-7. In Section 3 we describe how the RDF Schema
semantic definitions for MPEG-7 can be linked to their corresponding pre-existing XML
Schema definitions (or recommended encodings). In Section 4 we describe how the MPEG-7
RDF Schema can be merged with RDF schemas from other domains to generate a single
"super-ontology" called MetaNet. Expressed in DAML+OIL [17], MetaNet can be used to
provide common semantic understanding between domains. Finally we illustrate how this
super-ontology can be used to enable the co-existence of interoperability, extensibility and
diversity within metadata descriptions generated by integrating metadata terms from different
domains.

2. Building the Ontology
During the early development stages of MPEG-7, Unified Modelling Language (UML) [18]
was used to model the entities, properties and relationships (description schemes and
descriptors) which comprised MPEG-7. However the massive size of the specification (the
Multimedia Description Schemes specification [19] is almost 800 pages and that is only one
part out of 7 parts) combined with the belief that the UML models were a development tool
only, which duplicated information in the XML schemas, led to the decision to drop them
from the final specifications.
Although the lack of an existing data model hampered the development of an RDF
Schema ontology, it also means that the generated RDF Schema will be even more valuable -
providing both a data model as well as definitions of the semantics of the MPEG-7 terms and
the relationships between them. Building the data model and schema should also highlight any
inconsistencies, duplication or ambiguities which exist across the large number of MPEG-7
description schemes and descriptors.
Without a data model to build on, the class and property hierarchies and semantic
definitions had to be derived through reverse-engineering of the existing XML Schema
definitions together with interpretation of the english-text semantic descriptions. To simplify
the process, we used a core subset of the MPEG-7 specification together with a top-down
approach to generate the ontology described here. An additional very helpful mechanism for
determining the data model was to generate the DOM (Document Object Model) for the XML
Schema (using XML Spy). This graphical representation of the structures helped determine the
class and property hierarchies.
The first step was to determine the basic multimedia entities (classes) and their hierarchies
from the Multimedia Description Scheme (MDS) basic entities [19]. This process is described
in Section 2.1. Next the structural hierarchies were determined from the Segment Description
Schemes (Section 2.2). Section 2.3 describes the non-multimedia entities defined within
MPEG-7. Section 2.4 describes the different multimedia and generic properties associated with
the multimedia entities. Sections 2.5 describes the RDF Schema representations of the MPEG-
7 visual and audio descriptors defined in [20] and [21] respectively.
2.1 Top-level MPEG-7 Multimedia Entities
The top-level Multimedia Content entities are described in Section 4.4 of the MDS FCD [19].
The RDF class hierarchy corresponding to these basic entities is illustrated in Figure 1 and the
RDF Schema representation of these entities and relationships is shown in Appendix A.
Within MPEG-7, multimedia content is classified into five types: Image, Video, Audio,
Audiovisual and Multimedia. Each of these types have their own segment subclasses.

Figure 1: Class Hierarchy of MPEG-7 Top-level Multimedia Content Entities

2.2 MPEG-7 Multimedia Segments and Hierarchical Structures
MPEG-7 provides a number of tools for describing the structure of multimedia content in time
and space. The Segment DS (Section 11 of [19]) describes a spatial and/or temporal fragment
of multimedia content. A number of specialized subclasses are derived from the generic
Segment DS. These subclasses describe the specific types of multimedia segments, such as
video segments, moving regions, still regions and mosaics, which result from spatial, temporal
and spatiotemporal segmentation of the different multimedia content types. Table I describes
the different types of MPEG-7 segments and Figure 2 illustrates the corresponding segment
class hierarchy.

Segment Fragment or segment of multimedia content.
StillRegion 2D spatial regions of an image or video frame.
ImageText Spatial regions of an image or video frame corresponding to text or
captions.
Mosaic Mosaics or panoramic view of a video segment.
StillRegion3D 3D spatial regions of a 3D image.
VideoSegment Temporal intervals or segments of video data.
MovingRegion 2D spatio-temporal regions of video data.
VideoText Spatio-temporal regions of video data that correspond to text or captions.
AudioSegment Temporal intervals or segments of audio data.
AudioVisualSegment Temporal intervals or segments of AV data.
AudioVisualRegion Arbitrary spatio-temporal segments of AV data.
MultimediaSegment Composites of segments that form a multimedia presentation.
EditedVideoSegment Video segments that result from an editing work.
Table I: Semantic Definitions of MPEG-7 Segment Types

Figure 2: Class Hierarchy of MPEG-7 Segment Classes
The RDF Schema representation for the segment class hierarchy can be found in Appendix
A. Certain segment entities, such as the VideoSegment, are subclasses of multiple superclasses
i.e., both the Video class and the Segment class. The relationships of these segment types to the
top-level multimedia entities is illustrated in Figure 3. Multimedia resources can be segmented
or decomposed into sub-segments through 4 types of decomposition:
• Spatial Decomposition - e.g., spatial regions within an image;
• Temporal Decomposition - e.g., temporal video segments within a video;
• Spatiotemporal Decomposition - e.g., moving regions within a video;
• MediaSource Decomposition - e.g., the different tracks within an audio file or the different
media objects within a SMIL presentation.

The different types of segment decomposition can be represented via an RDF property
hierarchy. For example:

<rdf:Property rdf:ID="decomposition">
<rdfs:label>decomposition of a segment</rdfs:label>
<rdfs:domain rdf:resource="#MultimediaContent"/>
<rdfs:range rdf:resource="#Segment"/>
</rdf:Property>

<rdf:Property rdf:ID="temporal_decomposition">
<rdfs:label>temporal decomposition of a segment</rdfs:label>
<rdfs:subPropertyOf rdf:resource="#decomposition"/>
</rdf:Property>

Figure 3: Valid decomposition relationships between MPEG-7 Segment Classes (from Figure 32 [19])

If we consider the decomposition of a VideoSegment then, we would like to constrain the
temporal decomposition of VideoSegments into either smaller VideoSegments or StillRegions.

<rdf:Property rdf:ID="videoSegment_temporal_decomposition">
<rdfs:label>temporal decomposition of a video segment</rdfs:label>
<rdfs:subPropertyOf rdf:resource="#temporal_decomposition"/>
<rdfs:domain rdf:resource="#VideoSegment"/>
<rdfs:range rdf:resource="#VideoSegment"/>
<rdfs:range rdf:resource="#StillRegion"/>
</rdf:Property>
However this is illegal within RDF Schema because of the inability to specify multiple
range constraints on a single property. This limitation was first recognized in [23] when RDF
Schema was being considered as a candidate for the MPEG-7 DDL. The only way to express
this within RDF Schema is to define a new superclass which merges the permissable range
classes into a single common class.
DAML+OIL [17] permits multiple range statements but interprets the resulting range to
be the intersection of the specified classes. In this case, we want to specify that the range will
be an instance from the union of the two classes (VideoSegment and StillRegion). In order to
do this we must use daml:unionOf to define a class which is the union of these two classes and
then specify this new class as the range. For example:
<rdfs:Class rdf:ID="#VideoSegmentsOrStillRegions">
<daml:unionOf rdf:parseType="daml:collection">
<rdfs:Class rdf:about="#VideoSegment"/>
<rdfs:Class rdf:about="#StillRegion"/>
</daml:unionOf>
</rdfs:Class>


<rdfs:range rdf:resource="#VideoSegmentsOrStillRegions"/>
</rdf:Property>

Also associated with the segment classes are the properties which define the location of a
segment within its containing media object. These include such properties as: mediaLocator,
spatialLocator, mediaTime (temporal locator) and spatioTemporalLocator. If the segment is
non-continuous (i.e., the union of connected components) , then the spatialMask,
temporalMask, spatio-TemporalMask and mediaSpaceMask properties may be applicable.
These are sequences of spatial, temporal or spatiotemporal locators. Below we represent the
temporalLocator or mediaTime property (which has two components, the mediaTimePoint
(start of a segment) and the mediaDuration (length of the segment)):
<rdf:Property rdf:ID="mediaTime">
<rdfs:label>temporal location of a video or audio segment</rdfs:label>
<rdfs:domain rdf:resource="#Segment"/>
<rdfs:range rdf:resource="#MediaTime"/>
</rdf:Property>
<rdfs:Class rdf:ID="MediaTime">
<rdfs:label>time point or interval within media</rdfs:label>
<rdfs:subClassOf rdf:resource="#Time"/>
</rdfs:Class>
<rdf:Property rdf:ID="mediaTimePoint">
<rdfs:label>time point</rdfs:label>
<rdfs:domain rdf:resource="#MediaTime"/>
<rdfs:range rdf:resource="http://www. mpeg7.org/2001/MPEG-7_Schema# basicTimePoint"/>
</rdf:Property>
<rdf:Property rdf:ID="mediaDuration">
<rdfs:label>temporal length of segment</rdfs:label>
<rdfs:domain rdf:resource="#MediaTime"/>
<rdfs:range rdf:resource=" http://www. mpeg7.org/2001/MPEG-7_Schema#basicDuration"/>
</rdf:Property>
2.3 Basic Non-multimedia Entities within MPEG-7
As well as the multimedia entities described above, MPEG-7 defines a number of basic non-
multimedia entities which are used in different contexts across MPEG-7. These include:
• Agent
o Person
o PersonGroup
o Organisation
• Role
• Place
• Time
• Instrument
The RDF Schema representations of these classes can be found in Appendix A. The code
below shows both the XML Schema definition for the Person complexType. Figure 4 shows
corresponding the RDF model for the Person Class. This example illustrates how, in generating
the RDF Schema, we have translated the children elements of the XML Schema complexType to
properties attached to the RDF Schema class.

<complexType name="PersonType">
<complexContent>
<extension base="mpeg7:AgentType">
<sequence>
<element name="Name" type="mpeg7:PersonNameType"/>
<element name="Affiliation" minOccurs="0" maxOccurs="unbounded">
<complexType>
<choice>
<element name="Organization" type="mpeg7:OrganizationType"/>
<element name="PersonGroup" type="mpeg7:PersonGroupType"/>
</choice>
</complexType>
<element name="Address" type="mpeg7:PlaceType"/>
</sequence>
</extension>
</complexContent>
</complexType>
<complexType name="PersonNameType">
<sequence>
<choice minOccurs="1" maxOccurs="unbounded">
<element name="GivenName" type="string"/>
<element name="FamilyName" type="string"/>
</choice>
</sequence>
</complexType>

Again we have the situation where we would like to be able to say that the Affiliation property
can have values which are instantiations of either the Organisation or PersonGroup class i.e., we
would like to be able to define multiple possible ranges. DAML+OIL provides a way of doing
this through the unionOf mechanism as shown below:
<rdfs:Class rdf:ID="Affiliation">
<rdfs:comment>Either an Organisation or a PersonGroup</rdfs:comment>
<rdfs:Class rdf:about="#Organisation"/>
<rdfs:Class rdf:about="#PersonGroup"/>
</daml:unionOf>
</rdfs:Class>

<rdf:Property rdf:ID="affiliation">
<rdfs:label>affiliation</rdfs:label>
<rdfs:domain rdf:resource="#Person"/>
<rdfs:range rdf:resource="#Affiliation"/>
</rdf:Property>

Figure 4: RDF Class and Property Representation of PersonDS

2.4 Multimedia Description Schemes
Figure 5 provides an overview of the organization of MPEG-7 Multimedia DSs into the
following six categories: Basic Elements, Content Description, Content Management, Content
Organization, Navigation and Access, and User Interaction. The MPEG-7 DSs in Figure 5
define descriptions which provide:
• Information describing the creation and production processes of the content (director, title,
short feature movie);
• Information related to the usage of the content (copyright pointers, usage history, broadcast
schedule);
• Media information of the storage features of the content (storage format, encoding);
• Structural information on spatial, temporal or spatio-temporal components of the content
(scene cuts, segmentation in regions, region motion tracking);
• Information about low level features in the content (colors, textures, sound timbres,
melody description);
• Conceptual, semantic information of the reality captured by the content (objects and
events, interactions among objects);
• Information about how to browse the content in an efficient way (summaries, views,
variations, spatial and frequency subbands);

• Organization information about collections of objects and models, which allows
multimedia content to be characterized on the basis of probabilities, statistics and
examples;
• Information about the interaction of the user with the content (user preferences, usage
history)

Figure 5 - Overview of MPEG-7 Multimedia DSs (from Figure 1 [19])

We will not cover all of these DSs in this paper but have chosen to represent only the
CreationDS in order to demonstrate RDF Schema’s ability to model a typical MPEG-7 DS.
Figure 6 illustrates the RDF Schema classes and properties corresponding to the CreationDS
(expressed in XML Schema) below.

<complexType name="CreationType">
<complexContent>
<extension base="mpeg7:DSType">
<sequence>
<element name="Title" type="mpeg7:TitleType"/>
<element name="Abstract" type="mpeg7:TextAnnotationType"/>
<element name="Creator">
<complexContent> <extension base="mpeg7:AgentType">
<complexType>
<sequence>
<element name="Role" type="mpeg7:ControlledTermType"/>
<element name="Instrument" type="mpeg7:CreationToolType"/>
</sequence>
</complexType>
</extension></complexContent>
</element>
<element name="CreationLocation" type="mpeg7:PlaceType"/>
<element name="CreationDate" type="mpeg7:DateType"/>
</sequence>
</extension>
</complexContent>
</complexType>

Figure 6 – RDF Class and Property Representation of MPEG-7 Creation DS

2.5 Low Level Visual and Audio Descriptors
The set of features or properties which is specific to the visual entities (Image, Video,
AudioVisual, StillRegion, MovingRegion, VideoSegment) include:
• Colour
• Texture
• Motion
• Shape
Each of these features can be represented by a choice of descriptors. Table II below lists the
visual features and their corresponding MPEG-7 descriptors. Precise details of the structure
and semantics of these visual descriptors are provided in ISO/IEC 15938-3 FCD Multimedia
Content Description Interface - Part 3 Visual [20].

Feature Descriptors
Color DominantColor
ScalableColor
ColorLayout
ColorStructure
GoFGoPColor
Texture HomogeneousTexture
TextureBrowsing
EdgeHistogram
Shape RegionShape
ContourShape
Shape3D
Motion CameraMotion
MotionTrajectory
ParametricMotion
MotionActivity
Table II: Visual features and their corresponding Descriptors
Similarly there is a set of audio features which is applicable to MPEG-7 entities containing
audio (Video, AudioVisual, Audio, AudioSegment):
• Silence
• Timbre
• Speech
• Melody
ISO/IEC 15938-3 FCD Multimedia Content Description Interface - Part 4 Audio [21]
describes in detail the XML Schema specifications of the audio descriptors. Each of these
audio features can be represented by one or more audio descriptors. Table III below lists the
audio descriptors which correspond to each audio feature.

Feature Descriptors
Silence Silence
Timbre InstrumentTimbre
HarmonicInstrumentTimbre
PercussiveInstrumentTimbre
Speech Phoneme
Articulation
Language
MusicalStructure MelodicContour
Rhythm
SoundEffects Reverberation, Pitch, Contour, Noise
Table III: Audio features and their corresponding Descriptors

Only certain low-level visual and audio descriptors are applicable to each segment type. Table
IV below illustrates the association of visual and audio features to different segment types.
RDF Schema must be able to specify the constraints on these property-to-entity relationships.

Feature Video Still Moving Audio
Segment Region Region Segment
Time X - X X
Shape - X X -
Color X X X -
Texture - X - -
Motion X - X -
Audio X - - X
Table IV: Relationships between Segment types and Visual and Audio Features
Using the color descriptor, we demonstrate in Figure 7, how RDF Schema is able express these
constraints through the domain and range values in the color property definitions.

Figure 7: RDF Class and Property Representation of the MPEG-7 Color Descriptor

3. Linking the MPEG-7 XML and RDF Schemas

In a previous paper [22] we outlined the advantages of separating the semantics of domain-
specific metadata terms from the recommended encodings by defining both an RDF Schema
and an XML Schema in the domain’s registered namespace. The RDF Schema file defines the
domain-specific semantic knowledge by specifying type hierarchies and definitions - based on
the ISO/IEC 11179 standard for the description of data elements. The XML Schema file
specifies the recommended encodings of metadata elements and
descriptions by defining types and elements, and their content models, structures, occurrence
constraints and datatypes. In addition, the XML Schema file contains links to the
corresponding semantic definitions in the RDF Schema file. Because the underlying semantics
will remain relatively stable compared to the syntax, which will be application-dependent, we

choose to make the RDF Schema the base schema and to point to the base RDF Schema from
the application-specific XML Schemas, rather than the other direction.
The most concise and flexible method for implementing the link from the XML Schema
definitions to their corresponding RDF Schema definitions is to exploit the openness of XML
Schema attributes. Since nearly all types are extended from the openAttrs type in the Schema
for Schemas in [3], it is possible to extend XML Schema simpleType and complexType
definitions with a "semantics" attribute defined in another namespace e.g.,
"XMLRDFSchemaBridge". Using this approach, the value of the "semantics" attribute is set to
the RDF Property or Class which defines the semantics of the corresponding simple or
complex type. We have chosen to link the semantics to XML Schema type definitions, rather
than element declarations. This is because restrictions, extensions, redefinitions and elements
are all built on top of XML Schema types, so the most logical and flexible approach is to
attach the semantics to the type rather than the element. The XML Schema code below
demonstrates an implementation of this approach.
<schema xmnls="http://www.w3.org/2001/10/XMLSchema"
targetNamespace="http://www.mpeg7.org/2001/MPEG-7_Schema"
xmlns:mpeg7="http://www.mpeg7.org/2001/MPEG-7_Schema"
xmlns:xx="http://www.example.org/XMLRDFSchemaBridge">

<annotation>
<documentation>
XML Schema for MPEG-7
</documentation>
</annotation>

<simpleType name="Person"
xx:semantics="http://www.mpeg7.org/2001/MPEG7_Schema/mpeg7.rdf#Person">
<extension base="Agent"/>
</simpleType>

<simpleType name="Organisation"
xx:semantics="http://www.mpeg7.org/2001/MPEG-7_Schema/mpeg7.rdf#Organisation>
<extension base="Agent"/>
</simpleType>
...

</schema>

4. Balancing Metadata Interoperability, Extensibility and Diversity
By making the semantic knowledge of a domain or community available in a machine-
understandable RDF Schema, it becomes possible to merge separate ontologies or metadata
vocabularies from different communities into a single encompassing ontology expressed using
DAML+OIL. Using the ABC vocabulary ([24][28]), developed within the Harmony project
[29], as the top-level or umbrella, we have manually developed a draft version of such a
"super-ontology" - the MetaNet ontology [26]. MetaNet expresses the semantic relationships
(e.g., equivalent, narrower, broader) between metadata terms from different domains. By
linking the semantic knowledge provided by MetaNet with XSLT [25], we have been able to
perform both the semantic and the structural and syntactic mapping required to map between
XML-encoded metadata descriptions from different domains. The overall architecture of a
system, which should enable the coexistence of metadata interoperability together with
extensibility and diversity, is illustrated in Figure 8. The key components are:

• Domain-specific namespaces which express each domain’s metadata model and vocabulary
using both an RDF Schema and an XML Schema. Each XML Schema contains links to the
corresponding RDF Schema;
• MetaNet - a single "super" metadata ontology, generated by merging the domain-specific
ontologies (RDF Schemas) from different namespaces. This is expressed using
DAML+OIL and will be based on a common underlying, extensible vocabulary such as the
ABC vocabulary being developed within the Harmony project [24];
• XSLT - a language suitable for transforming between XML-encoded metadata
descriptions. Combined with the semantic knowledge of MetaNet, XSLT [25] is capable of
flexible dynamic mappings between application profile instantiations;
• Application Profiles - XML Schema definitions which combine, restrict, extend and
redefine elements from multiple existing namespaces. Application profiles could also
embed RDF Schema definitions of new classes or properties which are derived from
classes and properties defined in the domain-specific RDF Schemas.

Figure 8: The Proposed Web Metadata Architecture

5. Conclusions

In this paper, we first outlined the reasons for why an RDF Schema representation of MPEG-7
is desirable. We then described the methodology, problems encountered and results of
manually building an RDF Schema representation for a core subset of MPEG-7. Our

conclusion from this exercise is that, although RDF Schema is capable of expressing the
semantics of MPEG-7 Description Schemes and Descriptors, it does have certain serious
limitations. RDF Schema’s property-centricity makes it difficult to generate property
definitions and domain constraints from the class-centric XML Schema definitions. The
inability to specify multiple range constraints or class-specific property constraints are other
major limitations of RDF [23] within this context. However, these can be overcome through
the use of DAML+OIL extensions to RDF Schema including multiple range constraints,
boolean combinations of classes and class-specific constraints on properties. In addition, the
lack of cardinality and datatyping constraints in RDF Schema can be overcome by maintaining
the XML Schema definitions and linking them to the RDF Schema semantic definitions.
Whilst generating the RDF Schema representation of a subset of MPEG-7, we have also
been able to determine certain repetitive patterns and other information which can be derived
from the XML Schema definitions (baseTypes, comments, annotation, textual semantic
descriptions, the DOM). We believe that by exploiting this information, it may be possible to
automate the generation of an RDF Schema/DAML+OIL representation of MPEG-7 from the
existing XML Schema definitions.
So our future work plan is to attempt to develop programmatic tools capable of
automatically processing an MPEG-7 XML Schema document and converting this to a
DAML+OIL ontology which correctly represents the semantics of MPEG-7 description
schemes and descriptors and which is compatible and consistent with the corresponding XML
Schema. Links to this ontology can then be added to the MPEG-7 XML Schema definitions.
Once the MPEG-7 ontology is complete, we will then investigate ways of merging this
with the ABC/MetaNet ontology [28] as well as other metadata ontologies from other domains
(rights management, museums (CIDOC CRM)), to enable a common understanding of
descriptive terms across domains and the sharing and exchange of multimedia content over the
semantic web.

Acknowledgements
The work described in this paper has been carried out as part of the Harmony Project. It has
been funded by the Cooperative Research Centre for Enterprise Distributed Systems
Technology (DSTC) through the Australian Federal Government’s CRC Programme
(Department of Industry, Science and Resources).

References
[1] J. Martinez, "Overview of the MPEG-7 Standard (version 5.0)", ISO/IEC JTC1/SC29/WG11 N4031,
Singapore, March 2001. <http://www.cselt.it/mpeg/standards/mpeg-7/mpeg-7.htm>
[2] XML Schema Part 0: Primer, W3C Recommendation, 2 May 2001,
<http://www.w3.org/TR/xmlschema-0>
[3] XML Schema Part 1: Structures, W3C Recommendation, 2 May 2001,
<http://www.w3.org/TR/xmlschema-1/">
[4] XML Schema Part 2: Datatypes, W3C Recommendation, 2 May 2001,
<http://www.w3.org/TR/xmlschema-2/>
[5] RDF Schema Specification 1.0, W3C Candidate Recommendation 27 March
2000. <http://www.w3.org/TR/rdf-schema/>
[6] TV-Anytime Forum, <http://www.tv-anytime.org/>
[7] MPEG-21 Multimedia Framework,
<http://www.cselt.it/mpeg/public/mpeg-21_pdtr.zip>
[8] NewsML <http://www.newsml.org/>

[9] ISO/IEC 15938-2 FCD Information Technology - Multimedia Content Description Interface - Part 2:
Description Definition Language, March 2001, Singapore
[10] Dublin Core Metadata Element Set, Version 1.1, 2 July, 1999.
<http://www.purl.org/dc/documents/rec-dces-19990702.htm>
[11] G. Rust, M. Bide, "The indecs Metadata Schema Building Blocks", Indecs Metadata Model, November,
1999. <http://www.indecs.org/results/model.htm>
[12] Content Standard for Digital Geospatial Metadata (CSDGM),
<http://www.fgdc.gov/metadata/contstan.html>
[13] The Gateway to Educational Materials <http://www.the gateway.org>
[14] IEEE Learning Technology Standards Committee’s Learning Object Meta-data Working Group. Version 3.5
Learning Object Meta-data Scheme.
[15] ICOM/CIDOC Documentation Standards Group, Revised Definition of the CIDOC Conceptual Reference
Model, September 1999. <http://www.geneva-city.ch:80/musinfo/cidoc/oomodel>
[16] RDF Model and Syntax Specification, W3C Recommendation 22 February 1999.
<http://www.w3.org/TR/REC-rdf-syntax/>
[17] DAML+OIL Revised Language Specification, March 2001.
<http://www.daml.org/2001/03/daml+oil-index>
[18] UML Resource Center, <http://www.rational.com/uml/index.jsp>
[19] ISO/IEC 15938-5 FCD Information Technology - Multimedia Content Description Interface - Part 5:
Multimedia Description Schemes, March 2001, Singapore
[20] ISO/IEC 15938-3 FCD Information Technology - Multimedia Content Description Interface - Part 3: Visual,
March 2001, Singapore
[21] ISO/IEC 15938-4 FCD Information Technology - Multimedia Content Description Interface - Part 4: Audio,
March 2001, Singapore
[22] J. Hunter, C.Lagoze, "Combining RDF and XML Schemas to Enhance Metadata Interoperability Between
Application Profiles", WWW10, HongKong, May 2001.
<http://www10.org/cdrom/papers/572/index.html>
[23] J. Hunter, L.Armstrong, "A Comparison of Schemas for Video Metadata Representation", WWW8, Toronto,
May 1999 <http://archive.dstc.edu.au/RDU/staff/jane-hunter/www8/paper.html>
[24] C.Lagoze, J. Hunter, D. Brickley, "An Event-Aware Model for Metadata Interoperability", ECDL 2000,
Lisbon, September 2000.
[25] XSL Transformations (XSLT) Version 1.0 W3C Recommendation 16 November 1999,
<http://www.w3.org/TR/xslt.html>
[26] J. Hunter, "MetaNet – A Metadata Term Thesaurus to Enable Semantic Interoperability Between Metadata
Domains", Journal of Digital Information, Volume 1, Issue 8, April 2001
<http://jodi.ecs.soton.ac.uk/Articles/v01/i08/Hunter/>
[27] J. van Ossenbruggen et al., "Towards Second and Third Generation Web-Based Multimedia", WWW10,
HongKong, May 2001
[28] C. Lagoze, J. Hunter, "The ABC Ontology and Model", <http://metadata.net/harmony/dc_paper.pdf>
[29] The Harmony International Digital Library Project, <http://metadata.net/harmony/>

Appendix A: An MPEG-7 Ontology Expressed as a DAML+OIL Schema
<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:daml="http://www.daml.org/2001/03/daml+oil#"
xmlns:xsd="http://www.w3.org/2000/10/XMLSchema#"
xmlns:mpeg7="http://www.mpeg7.org/2001/MPEG-7_Schema#"
xmlns="http://www.mpeg7.org/2001/MPEG-7_Schema#">

<rdfs:Class rdf:ID="MultimediaContent">
<rdfs:label>MultimediaContent</rdfs:label>
<rdfs:comment>The class of multimedia data</rdfs:comment>

<rdfs:subClassOf rdf:resource="http://www.w3.org/2000/01/rdf-schema#Resource"/>
</rdfs:Class>
<rdfs:Class rdf:ID="Image">
<rdfs:label>Image</rdfs:label>
<rdfs:comment>The class of images</rdfs:comment>
<rdfs:subClassOf rdf:resource="#MultimediaContent"/>
</rdfs:Class>
<rdfs:Class rdf:ID="Video">
<rdfs:label>Video</rdfs:label>
<rdfs:comment>The class of videos</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="Audio">
<rdfs:label>Audio</rdfs:label>
<rdfs:comment>The class of audio resources</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="AudioVisual">
<rdfs:label>AudioVisual</rdfs:label>
<rdfs:comment>The class of audiovisual resources</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="Multimedia">
<rdfs:label>Multimedia</rdfs:label>
<rdfs:comment>The class of multimedia resources</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="Segment">
<rdfs:label>Segment</rdfs:label>
<rdfs:comment>The class of fragments of multimedia content</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="StillRegion">
<rdfs:label>StillRegion</rdfs:label>
<rdfs:comment>2D spatial regions of an image or video frame</rdfs:comment>
<rdfs:subClassOf rdf:resource="#Segment"/>
<rdfs:subClassOf rdf:resource="#Image"/>
</rdfs:Class>
<rdfs:Class rdf:ID="ImageText">
<rdfs:label>ImageText</rdfs:label>
<rdfs:comment>Spatial regions of an image or video frame that correspond to text or
captions</rdfs:comment>
<rdfs:subClassOf rdf:resource="#StillRegion"/>
</rdfs:Class>
<rdfs:Class rdf:ID="Mosaic">
<rdfs:label>Mosaic</rdfs:label>
<rdfs:comment>Mosaic or panaoramic view of a video segment</rdfs:comment>
<rdfs:subClassOf rdf:resource="#StillRegion"/>
</rdfs:Class>
<rdfs:Class rdf:ID="StillRegion3D">
<rdfs:label>StillRegion3D</rdfs:label>
<rdfs:comment>3D spatial regions of a 3D image</rdfs:comment>
<rdfs:subClassOf rdf:resource="#Image"/>
</rdfs:Class>
<rdfs:Class rdf:ID="VideoSegment">
<rdfs:label>VideoSegment</rdfs:label>

<rdfs:comment>Temporal intervals or segments of video data</rdfs:comment>
<rdfs:subClassOf rdf:resource="#Video"/>
</rdfs:Class>
<rdfs:Class rdf:ID="MovingRegion">
<rdfs:label>MovingRegion</rdfs:label>
<rdfs:comment>2D spatio-temporal regions of video data</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="VideoText">
<rdfs:label>VideoText</rdfs:label>
<rdfs:comment>Spatio-temporal regions of video data that correspond to text or captions</rdfs:comment>
<rdfs:subClassOf rdf:resource="#MovingRegion"/>
</rdfs:Class>
<rdfs:Class rdf:ID="AudioSegment">
<rdfs:label>AudioSegment</rdfs:label>
<rdfs:comment>Temporal intervals or segments of audio data</rdfs:comment>
<rdfs:subClassOf rdf:resource="#Audio"/>
</rdfs:Class>
<rdfs:Class rdf:ID="AudioVisualSegment">
<rdfs:label>AudioVisualSegment</rdfs:label>
<rdfs:comment>Temporal intervals or segments of audiovisual data</rdfs:comment>
<rdfs:subClassOf rdf:resource="#AudioVisual"/>
</rdfs:Class>
<rdfs:Class rdf:ID="AudioVisualRegion">
<rdfs:label>AudioVisualRegion</rdfs:label>
<rdfs:comment>Arbitrary spatio-temporal segments of AV data</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="MultimediaSegment">
<rdfs:label>MultimediaSegment</rdfs:label>
<rdfs:comment>Segment of a composite multimedia presentation</rdfs:comment>
<rdfs:subClassOf rdf:resource="#Multimedia"/>
</rdfs:Class>
<rdfs:Class rdf:ID="EditedVideoSegment">
<rdfs:label>EditedVideoSegment</rdfs:label>
<rdfs:comment>Video segment that results from editing work</rdfs:comment>
<rdfs:subClassOf rdf:resource="#VideoSegment"/>
</rdfs:Class>
<rdf:Property rdf:ID="decomposition">
<rdfs:label>decomposition of a segment</rdfs:label>
</rdf:Property>
<rdf:Property rdf:ID="spatial_decomposition">
<rdfs:label>spatial decomposition of a segment</rdfs:label>
</rdf:Property>
<rdf:Property rdf:ID="temporal_decomposition">
<rdfs:label>temporal decomposition of a segment</rdfs:label>

</rdf:Property>
<rdf:Property rdf:ID="spatio-temporal_decomposition">
<rdfs:label>spatio-temporal decomposition of a segment</rdfs:label>
</rdf:Property>
<rdf:Property rdf:ID="mediaSource_decomposition">
<rdfs:label>media source decomposition of a segment</rdfs:label>
</rdf:Property>
<rdf:Property rdf:ID="videoSegment_spatial_decomposition">
<rdfs:label>spatial decomposition of a video segment</rdfs:label>
<rdfs:subPropertyOf rdf:resource="#spatial_decomposition"/>
<rdfs:range rdf:resource="#MovingRegion"/>
</rdf:Property>
<rdfs:Class rdf:ID="VideoSegmentsOrStillRegions">
<rdfs:Class rdf:about="#VideoSegment"/>
</daml:unionOf>
</rdfs:Class>
<rdfs:range rdf:resource="#VideoSegmentsOrStillRegions"/>
</rdf:Property>

<rdfs:Class rdf:ID="MovingOrStillRegions">
<rdfs:Class rdf:about="#MovingRegion"/>
</daml:unionOf>
</rdfs:Class>
<rdf:Property rdf:ID="videoSegment_spatio-temporal_decomposition">
<rdfs:label>spatio-temporal decomposition of a video segment</rdfs:label>
<rdfs:subPropertyOf rdf:resource="#spatio-temporal_decomposition"/>
<rdfs:range rdf:resource="#MovingOrStillRegions"/>
</rdf:Property>
<rdf:Property rdf:ID="videoSegment_mediaSource_decomposition">
<rdfs:label>media source decomposition of a video segment</rdfs:label>
<rdfs:subPropertyOf rdf:resource="#mediaSource_decomposition"/>
<rdfs:range rdf:resource="#VideoSegment"/>
</rdf:Property>
<rdfs:Class rdf:ID="Agent">
<rdfs:label>Agent</rdfs:label>
<rdfs:comment>Agent - person, organisation or group which performs
an act.</rdfs:comment>
</rdfs:Class>

<rdfs:Class rdf:ID="Person">
<rdfs:label>Person</rdfs:label>
<rdfs:comment>An individual person.</rdfs:comment>
<rdfs:subClassOf rdf:resource="#Agent"/>
</rdfs:Class>
<rdfs:Class rdf:ID="PersonGroup">
<rdfs:label>PersonGroup</rdfs:label>
<rdfs:comment>A group of persons with a collective title.</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="Organisation">
<rdfs:label>Organisation</rdfs:label>
<rdfs:comment>Organisation.</rdfs:comment>
</rdfs:Class>
<rdf:Property rdf:ID="role">
<rdfs:label>The Role played by an agent or place in an event</rdfs:label>
<rdfs:domain rdf:resource="#Agent"/>
<rdfs:domain rdf:resource="#Place"/>
</rdf:Property>
<rdfs:Class rdf:ID="Place">
<rdfs:label>Place</rdfs:label>
<rdfs:comment>Describes real, fictional, historical locations.</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="Time">
<rdfs:label>Time</rdfs:label>
<rdfs:comment>Describes date/time points and durations</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="Instrument">
<rdfs:label xml:lang="en">Instrument</rdfs:label>
<rdfs:comment>Describes instrument or tool used to perform an action.</rdfs:comment>
</rdfs:Class>
<rdf:Property rdf:ID="name">
<rdfs:label>name</rdfs:label>
<rdfs:range rdf:resource="#PersonName"/>
</rdf:Property>
<rdfs:Class rdf:ID="Affiliation">
<rdfs:comment>An affiliation is either an Organisation or a PersonGroup </rdfs:comment>
<rdfs:Class rdf:about="#Organisation"/>
<rdfs:Class rdf:about="#PersonGroup"/>
</daml:unionOf>
</rdfs:Class>
<rdf:Property rdf:ID="affiliation">
<rdfs:label>affiliation</rdfs:label>
<rdfs:range rdf:resource="#Affiliation"/>
</rdf:Property>
<rdf:Property rdf:ID="address">
<rdfs:label>address</rdfs:label>
<rdfs:range rdf:resource="#Address"/>
</rdf:Property>

<rdfs:Class rdf:ID="Address">
<rdfs:label>Address</rdfs:label>
<rdfs:comment>Address of person, organisation or person group.</rdfs:comment>
<rdfs:subClassOf rdf:resource="#Place"/>
</rdfs:Class>
<rdfs:Class rdf:ID="PersonName">
<rdfs:label>PersonName</rdfs:label>
<rdfs:comment>Name of an individual person.</rdfs:comment>
</rdfs:Class>
<rdf:Property rdf:ID="givenName">
<rdfs:label>givenName</rdfs:label>
<rdfs:domain rdf:resource="#PersonName"/>
<rdfs:range rdf:resource="#Literal"/>
</rdf:Property>
<rdf:Property rdf:ID="familyName">
<rdfs:label>familyName</rdfs:label>
<rdfs:domain rdf:resource="#PersonName"/>
<rdfs:range rdf:resource="#Literal"/>
</rdf:Property>
<rdfs:Class rdf:ID="Creation">
<rdfs:label>Creation</rdfs:label>
<rdfs:comment>A multimedia content creation.</rdfs:comment>
</rdfs:Class>
<rdf:Property rdf:ID="title">
<rdfs:label>title</rdfs:label>
<rdfs:subPropertyOf rdf:resource="#multimediaDescriptor"/>
<rdfs:domain rdf:resource="#Creation"/>
<rdfs:range rdf:resource="#Title"/>
</rdf:Property>
<rdf:Property rdf:ID="abstract">
<rdfs:label>abstract</rdfs:label>
<rdfs:range rdf:resource="#TextAnnotation"/>
</rdf:Property>
<rdf:Property rdf:ID="creator">
<rdfs:label>creator</rdfs:label>
<rdfs:range rdf:resource="#Creator"/>
</rdf:Property>
<rdf:Property rdf:ID="creationLocation">
<rdfs:label>creationLocation</rdfs:label>
<rdfs:range rdf:resource="#Place"/>
</rdf:Property>
<rdf:Property rdf:ID="creationDate">
<rdfs:label>creationDate</rdfs:label>
<rdfs:range rdf:resource="#Time"/>
</rdf:Property>
<rdfs:Class rdf:ID="Creator">
<rdfs:label>Creator</rdfs:label>

<rdfs:comment>Person, organisation or person group who created the content.</rdfs:comment>
</rdfs:Class>
<rdf:Property rdf:ID="role">
<rdfs:label>role</rdfs:label>
<rdfs:domain rdf:resource="#Creator"/>
<rdfs:range rdf:resource="#ControlledTerm"/>
</rdf:Property>
<rdf:Property rdf:ID="creationTool">
<rdfs:label>instrument</rdfs:label>
<rdfs:comment>Instrument used by creator to create multimedia content.</rdfs:comment>
<rdfs:domain rdf:resource="#Creator"/>
<rdfs:range rdf:resource="#Instrument"/>
</rdf:Property>
<rdfs:Class rdf:ID="Color">
<rdfs:label>Color</rdfs:label>
<rdfs:comment>Color of a visual resource</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="DominantColor">
<rdfs:label>DominantColor</rdfs:label>
<rdfs:comment>The set of dominant colors in an arbitrarily-shaped region.</rdfs:comment>
<rdfs:subClassOf rdf:resource="#Color"/>
</rdfs:Class>
<rdfs:Class rdf:ID="ScalableColor">
<rdfs:label>ScalableColor</rdfs:label>
<rdfs:comment>Color histogram in the HSV color space.</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="ColorLayout">
<rdfs:label>ColorLayout</rdfs:label>
<rdfs:comment>Spatial distribution of colors.</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="ColorStructure">
<rdfs:label>ColorStructure</rdfs:label>
<rdfs:comment>Describes color content and the structure of this content.</rdfs:comment>
</rdfs:Class>
<rdfs:Class rdf:ID="GoFGoPColor">
<rdfs:label>GoFGoPColor</rdfs:label>
<rdfs:comment>Group of frames/pictures color descriptor.</rdfs:comment>
<rdfs:subClassOf rdf:resource="#ScalableColor"/>
</rdfs:Class>

<rdf:Property rdf:ID="color">
<rdfs:label>color</rdfs:label>
<rdfs:comment>Color descriptor - applicable to video segments, still regions and moving
regions.</rdfs:comment>
<rdfs:subPropertyOf rdf:resource="#visualDescriptor"/>
<rdfs:domain rdf:resource="#StillRegion"/>
<rdfs:domain rdf:resource="#MovingRegion"/>
<rdfs:range rdf:resource="#Color"/>
</rdf:Property>
</rdf:RDF>

Swws

More Related Content

Viewers also liked

Similar to Swws

More from StephanieLeBadezet

Recently uploaded

Swws