Requirements for multimedia metadata schemes in surveillance applications for security

J. Van Rest, F. A. Grootjen, M. Grootjen, R. Wijn, O. Aarts, M. L. Roelofs, G. J. Burghouts, H. Bouma, L. Alic, W. Kraaij

Research output: Contribution to journalArticle

5 Citations (Scopus)


Surveillance for security requires communication between systems and humans, involves behavioural and multimedia research, and demands an objective benchmarking for the performance of system components. Metadata representation schemes are extremely important to facilitate (system) interoperability and to define ground truth annotations for surveillance research and benchmarks. Surveillance places specific requirements on these metadata representation schemes. This paper offers a clear and coherent terminology, and uses this to present these requirements and to evaluate them in three ways: their fitness in breadth for surveillance design patterns, their fitness in depth for a specific surveillance scenario, and their realism on the basis of existing schemes. It is also validated that no existing metadata representation scheme fulfils all requirements. Guidelines are offered to those who wish to select or create a metadata scheme for surveillance for security.

Original languageEnglish
Pages (from-to)573-598
Number of pages26
JournalMultimedia Tools and Applications
Issue number1
Publication statusPublished - May 2014



  • Action
  • Annotation
  • Event
  • Human behaviour
  • MPEG-7
  • Metadata representation scheme
  • Multi-sensor
  • Multimodal
  • PETS
  • Surveillance

ASJC Scopus subject areas

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

Van Rest, J., Grootjen, F. A., Grootjen, M., Wijn, R., Aarts, O., Roelofs, M. L., Burghouts, G. J., Bouma, H., Alic, L., & Kraaij, W. (2014). Requirements for multimedia metadata schemes in surveillance applications for security. Multimedia Tools and Applications, 70(1), 573-598.