Ohm Multimedia Content Analysis

1. Auflage 2016
ISBN: 978-3-662-52828-0
Verlag: Springer
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 425 Seiten

Reihe: Signals and Communication Technology

This textbook covers the theoretical backgrounds and practical aspects of image, video and audio feature expression, e.g., color, texture, edge, shape, salient point and area, motion, 3D structure, audio/sound in time, frequency and cepstral domains, structure and melody. Up-to-date algorithms for estimation, search, classification and compact expression of feature data are described in detail. Concepts of signal decomposition (such as segmentation, source tracking and separation), as well as composition, mixing, effects, and rendering, are discussed. Numerous figures and examples help to illustrate the aspects covered. The book was developed on the basis of a graduate-level university course, and most chapters are supplemented by problem-solving exercises. The book is also a self-contained introduction both for researchers and developers of multimedia content analysis systems in industry.   

Jens Rainer Ohm graduated in Electrical Engineering at TU Berlin. After his habilitations he became project coordinator at Fraunhofer Heinrich Hertz Institute, Berlin. Since 2000 Jens Ohm is Chair for Communications Engineering and Head of the Institute for Communications Engineering at Aachen University.

Autoren/Hrsg.

Ohm, Jens-Rainer

Ohm, Jens

Inhaltsverzeichnis

1;Preface;6
2;Table of Contents;7
3;1 Introduction;11
3.1;1.1 Context;11
3.2;1.2 Applications;13
4;2 Preprocessing;19
4.1;2.1 Nonlinear filters;21
4.1.1;2.1.1 Median filters and rank-order filters;21
4.1.2;2.1.2 Morphological filters;25
4.1.3;2.1.3 Polynomial filters;29
4.2;2.2 Amplitude-value transformations;30
4.2.1;2.2.1 Amplitude mapping characteristics;31
4.2.2;2.2.2 Probability distribution modification and equalization;32
4.3;2.3 Interpolation;34
4.3.1;2.3.1 Zero and first order interpolation basis functions;35
4.3.2;2.3.2 LTI systems as interpolators;37
4.3.3;2.3.3 Spline, Lagrangian and polynomial interpolation;38
4.3.4;2.3.4 Interpolation on 2D grids;43
4.4;2.4 Multi-resolution representation;47
4.5;2.5 Locally adaptive filters;53
4.5.1;2.5.1 Steerable smoothing filters;53
4.5.2;2.5.2 Iterative smoothing (diffusion filters);55
4.6;2.6 Problems;58
5;3 Signal and Parameter Estimation;61
5.1;3.1 Expected values and probability description;61
5.2;3.2 Observation and degradation models;66
5.3;3.3 Estimation based on linear filters;67
5.3.1;3.3.1 Inverse filters;67
5.3.2;3.3.2 Wiener filters;68
5.4;3.4 Least-squares estimation;70
5.5;3.5 Singular value decomposition;75
5.6;3.6 ML and MAP estimation;77
5.7;3.7 Parameter estimation and fitting;79
5.8;3.8 Outlier rejection;81
5.9;3.9 Correspondence analysis;84
5.10;3.10 State modeling and estimation;87
5.10.1;3.10.1 Markov processes and random fields;87
5.10.2;3.10.2 Hidden Markov models;90
5.10.3;3.10.3 Kalman filters;91
5.10.4;3.10.4 Particle filters;94
5.11;3.11 Problems;94
6;4 Features of Multimedia Signals;97
6.1;4.1 Color;97
6.1.1;4.1.1 Color space transformations;98
6.1.2;4.1.2 Representation of color features;107
6.2;4.2 Texture;112
6.2.1;4.2.1 Texture analysis based on occurrence counts;114
6.2.2;4.2.2 Texture analysis based on statistical models;117
6.2.3;4.2.3 Spectral features of texture;120
6.2.4;4.2.4 Inhomogeneous texture analysis;124
6.3;4.3 Edge analysis;125
6.3.1;4.3.1 Edge detection by gradient operators;125
6.3.2;4.3.2 Edge characterization by second derivative;129
6.3.3;4.3.3 Edge finding and consistency analysis;131
6.3.4;4.3.4 Edge model fitting;134
6.3.5;4.3.5 Description and analysis of edge properties;135
6.4;4.4 Salient feature detection;137
6.5;4.5 Contour and shape analysis;142
6.5.1;4.5.1 Contour fitting;142
6.5.2;4.5.2 Contour description by orientation and curvature;146
6.5.3;4.5.3 Geometric features and binary shape features;150
6.5.4;4.5.4 Projection and geometric mapping;154
6.5.5;4.5.5 Moment analysis of region shapes;164
6.5.6;4.5.6 Region shape analysis by basis functions;168
6.6;4.6 Motion analysis;169
6.6.1;4.6.1 Projection of 3D motion into the image plane;169
6.6.2;4.6.2 Motion estimation by the optical flow principle;173
6.6.3;4.6.3 Motion estimation by matching;178
6.6.4;4.6.4 Estimation of non-translational motion parameters;188
6.6.5;4.6.5 Estimation of motion vector fields at object boundaries;190
6.7;4.7 Disparity and depth analysis;193
6.7.1;4.7.1 Coplanar stereoscopy;193
6.7.2;4.7.2 Epipolar geometry;196
6.7.3;4.7.3 Camera calibration;199
6.8;4.8 Audio signal features;203
6.8.1;4.8.1 Audio feature extraction on the timeline;204
6.8.2;4.8.2 Time domain features;206
6.8.3;4.8.3 Spectral domain features;212
6.8.4;4.8.4 Cepstral domain features;216
6.8.5;4.8.5 Harmonic features;217
6.8.6;4.8.6 Multi-channel features;222
6.8.7;4.8.7 Perceptual features;223
6.8.8;4.8.8 Semantic features;225
6.9;4.9 Problems;227
7;5 Feature Transforms and Classification;233
7.1;5.1 Feature value normalization and transforms;233
7.2;5.2 Distance metrics;248
7.3;5.3 Compressed representation of feature data;261
7.4;5.4 Feature-based comparison;263
7.5;5.5 Reliability;267
7.6;5.6 Classification methods;274
7.7;5.7 Belief, plausibility and evidence;299
7.8;5.8 Problems;302
8;6 Signal Decomposition;305
8.1;6.1 Spatial segmentation of pictures;306
8.1.1;6.1.1 Segmentation based on sample classification;307
8.1.2;6.1.2 Region-based methods;312
8.1.3;6.1.3 Contour-based methods;314
8.1.4;6.1.4 Segmentation based on ‘energy minimization’;315
8.2;6.2 Segmentation of video signals;321
8.2.1;6.2.1 Key picture and shot transition detection;322
8.2.2;6.2.2 Segmentation by background differencing;323
8.2.3;6.2.3 Object tracking and spatio-temporal segmentation;324
8.2.4;6.2.4 Combined segmentation and motion estimation;330
8.3;6.3 3D surface and volume reconstruction;331
8.3.1;6.3.1 3D point cloud generation;332
8.3.2;6.3.2 3D surface reconstruction;333
8.3.3;6.3.3 3D volume reconstruction;335
8.3.4;6.3.4 Projection based description of 3D shapes;336
8.4;6.4 Decomposition of audio signals;339
8.4.1;6.4.1 Temporal segmentation of audio;339
8.4.2;6.4.2 Audio source separation;339
8.5;6.5 Problems;341
9;7 Signal Composition, Rendering and Presentation;343
9.1;7.1 Composition and mixing of multimedia signals;343
9.2;7.2 Mosaicking and stitching;348
9.3;7.3 Synthesis of picture content;351
9.4;7.4 Warping and morphing;355
9.5;7.5 Virtual view synthesis;357
9.6;7.6 Frame rate conversion;362
9.7;7.7 View-adaptive and stereoscopic rendering of image and video signals;366
9.8;7.8 Composition and rendering of audio signals;369
9.8.1;7.8.1 Sound effects;371
9.8.2;7.8.2 Spatial (room) features;374
10;A Fundamentals and definitions;377
10.1;A.1 Fundamentals of signal processing and signal analysis;377
10.2;A.2 Fundamentals of stochastic analysis and description;386
10.3;A.3 Vector and matrix algebra;395
11;B Symbols and Variables;401
12;C Glossary and Acronyms;406
13;D References;408
14;E Index;421