MPEG-7 audio and beyond : audio content indexing and retrieval

Advances in technology, such as MP3 players, the Internet and DVDs, have led to the production, storage and distribution of a wealth of audio signals, including speech, music and more general sound signals and their combinations. MPEG-7 audio tools were created to enable the navigation of this data, by providing an established framework for effective multimedia management. MPEG-7 Audio and Beyond: Audio Content Indexing and Retrieval is a unique insight into the technology, covering the following topics: the fundamentals of MPEG-7 audio, principally low-level descriptors and sound classification and similarity; spoken content description, and timbre, melody and tempo music description tools; existing MPEG-7 applications and those currently being developed; examples of audio technology beyond the scope of MPEG-7.

Essential reading for practising electronic and communications engineers designing and implementing MPEG-7 compliant systems, this book will also be a useful reference for researchers and graduate students working with multimedia database technology.

Author Notes

Hyoung-Gook Kim , Researcher of the MPEG-7 Audio Project at the Communication Systems Group, Technical University of Berlin, Communication Systems Group, Sekr. EN 1, Einsteinufer 17, D-10587 Berlin

Nicolas Moreau , Researcher of the MPEG-7 Audio Project at the Communication Systems Group, Technical University of Berlin, Communication Systems Group, Sekr. EN 1, Einsteinufer 17, D-10587 Berlin

Thomas Sikora , Professor and head of the Communication Systems Group, Technical University of Berlin, Communication Systems Group, Sekr. EN 1, Einsteinufer 17, D-10587 Berlin

List of Acronyms

List of Symbols

1 Introduction

1.1 Audio Content Description

1.2 MPEG-7 Audio Content Description - An Overview

1.2.1 MPEG-7 Low-Level Descriptors

1.2.2 MPEG-7 Description Schemes

1.2.3 MPEG-7 Description Definition Language (DDL)

1.2.4 BiM (Binary Format for MPEG-7)

1.3 Organization of the Book

2 Low-Level Descriptors

2.1 Introduction

2.2 Basic Parameters and Notations

2.2.1 Time Domain

2.2.2 Frequency Domain

2.3 Scalable Series

2.3.1 Series of Scalars

2.3.2 Series of Vectors

2.3.3 Binary Series

2.4 Basic Descriptors

2.4.1 Audio Waveform

2.4.2 Audio Power

2.5 Basic Spectral Descriptors

2.5.1 Audio Spectrum Envelope

2.5.2 Audio Spectrum Centroid

2.5.3 Audio Spectrum Spread

2.5.4 Audio Spectrum Flatness

2.6 Basic Signal Parameters

2.6.1 Audio Harmonicity

2.6.2 Audio Fundamental Frequency

2.7 Timbral Descriptors

2.7.1 Temporal Timbral: Requirements

2.7.2 Log Attack Time

2.7.3 Temporal Centroid

2.7.4 Spectral Timbral: Requirements

2.7.5 Harmonic Spectral Centroid

2.7.6 Harmonic Spectral Deviation

2.7.7 Harmonic Spectral Spread

2.7.8 Harmonic Spectral Variation

2.7.9 Spectral Centroid

2.8 Spectral Basis Representations

2.9 Silence Segment

2.10 Beyond the Scope of MPEG-7

2.10.1 Other Low-Level Descriptors

2.10.2 Mel-Frequency Cepstrum Coefficients

References

3 Sound Classification and Similarity

3.1 Introduction

3.2 Dimensionality Reduction

3.2.1 Singular Value Decomposition (SVD)

3.2.2 Principal Component Analysis (PCA)

3.2.3 Independent Component Analysis (ICA)

3.2.4 Non-Negative Factorization (NMF)

3.3 Classification Methods

3.3.1 Gaussian Mixture Model (GMM)

3.3.2 Hidden Markov Model (HMM)

3.3.3 Neural Network (NN)

3.3.4 Support Vector Machine (SVM)

3.4 MPEG-7 Sound Classification

3.4.1 MPEG-7 Audio Spectrum Projection (ASP) Feature Extraction

3.4.2 Training Hidden Markov Models (HMMs)

3.4.3 Classification of Sounds

3.5 Comparison of MPEG-7 Audio Spectrum Projection vs. MFCC Features

3.6 Indexing and Similarity

3.6.1 Audio Retrieval Using Histogram Sum of Squared Differences

3.7 Simulation Results and Discussion

3.7.1 Plots of MPEG-7 Audio Descriptors

3.7.2 Parameter Selection

3.7.3 Results for Distinguishing Between Speech, Music and Environmental Sound

3.7.4 Results of Sound Classification Using Three Audio Taxonomy Methods

3.7.5 Results for Speaker Recognition

3.7.6 Results of Musical Instrument Classification

3.7.7 Audio Retrieval Results

3.8 Conclusions

References

4 Spoken Content

4.1 Introduction

4.2 Automatic Speech Recognition

4.2.1 Basic Principles

4.2.2 Types of Speech Recognition Systems

4.2.3 Recognition Results

4.3 MPEG-7 Spoken Content Description

4.3.1 General Structure

4.3.2 Spoken Content Header

4.3.3 Spoken Content Lattice

4.4 Application: Spoken Document Retrieval

4.4.1 Basic Principles of IR and SDR

4.4.2 Vector Space Models

4.4.3 Word-Based SDR

4.4.4 Sub-Word-Based Vector Space Models

4.4.5 Sub-Word String Matching

4.4.6 Combining Word and Sub-Word Indexing

4.5 Conclusions

4.5.1 MPEG-7 Interoperability

4.5.2 MPEG-7 Flexibility

4.5.3 Perspectives

References

5 Music Description Tools

5.1 Timbre

5.1.1 Introduction

5.1.2 Instrument Timbre

5.1.3 Harmonic Instrument Timbre

5.1.4 Percussive Instrument Timbre

5.1.5 Distance Measures

5.2 Melody

5.2.1 Melody

5.2.2 Meter

5.2.3 Scale

5.2.4 Key

5.2.5 Melody Contour

5.2.6 Melody Sequence

5.3 Tempo

5.3.1 Audio Tempo

5.3.2 Audio BPM

5.4 Application Example: Query-by-Humming

5.4.1 Monophonic Melody Transcription

5.4.2 Polyphonic Melody Transcription

5.4.3 Comparison of Melody Contours

References

6 Fingerprinting and Audio Signal Quality

6.1 Introduction

6.2 Audio Signature

6.2.1 Generalities on Audio Fingerprinting

6.2.2 Fingerprint Extraction

6.2.3 Distance and Searching Methods

6.2.4 MPEG-7-Standardized Audio Signature

6.3 Audio Signal Quality

6.3.1 Audio Signal Quality Description Scheme

6.3.2 Broadcast Ready

6.3.3 IsOriginal Mono

6.3.4 Background Noise Level

6.3.5 Cross Channel Correlation

6.3.6 Relative Delay

6.3.7 Balance

6.3.8 DcOffset

6.3.9 Bandwidth

6.3.10 Transmiss

Available:*

On Order

Summary

Summary

Author Notes

Table of Contents