Assunção / Gotchev 3D Visual Content Creation, Coding and Delivery

1. Auflage 2019
ISBN: 978-3-319-77842-6
Verlag: Springer International Publishing
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 328 Seiten

Reihe: Signals and Communication Technology

This book covers the different aspects of modern 3D multimedia technologies by addressing several elements of 3D visual communications systems, using diverse content formats, such as stereo video, video-plus-depth and multiview, and coding schemes for delivery over networks. It also presents the latest advances and research results in regards to objective and subjective quality evaluation of 3D visual content, extending the human factors affecting the perception of quality to emotional states. The contributors describe technological developments in 3D visual communications, with particular emphasis on state-of-the-art advances in acquisition of 3D visual scenes and emerging 3D visual representation formats, such as:
 multi-view plus depth and light field;evolution to freeview and light-field representation;compression methods and robust delivery systems; andcoding and delivery over various channels. Simulation tools, testbeds and datasets that are useful for advanced research and experimental studies in the field of 3D multimedia delivery services and applications are covered. The international group of contributors also explore the research problems and challenges in the field of immersive visual communications, in order to identify research directions with substantial economic and social impact. 3D Visual Content Creation, Coding and Delivery provides valuable information to engineers and computer scientists developing novel products and services with emerging 3D multimedia technologies, by discussing the advantages and current limitations that need to be addressed in order to develop their products further. It will also be of interest to students and researchers in the field of multimedia services and applications, who are particularly interested in advances bringing significant potential impact on future technological developments.

?Pedro A. Amado Assunção received the Licenciado and M.Sc. degrees in Electrical Engineering from the University of Coimbra, Portugal, in 1988 and 1993, respectively, and the Ph.D. in Electronic Systems Engineering from the University of Essex, UK, in 1998. He is currently Professor of Electrical and Computer Engineering, Electronics and Multimedia Communication Systems at the Polytechnic Institute of Leiria and senior researcher at Instituto de Telecomunicacoes, Portugal. He has been involved in several projects in the field of multimedia communications and he served as a reviewer and/or technical programme committee of many international conferences and journals. He is author/co-author of more than one hundred papers published in international scientific conferences and journals, one book, eight book chapters and four US patents. His current research interests include ultra-high definition 3D multiview and light-field video coding, communications and processing, video codec complexity control and networking adaptation of coded video streams, error concealment and quality assessment. He was the Chair of the EU COST Action IC1105, 3D Content Creation, Coding and Transmission over Future Media Networks (3D-ConTourNet) and General Chair of 3DTV-Con 2015. He is a Senior Member of the IEEE.
Atanas Gotchev received M.Sc. degrees in radio and television engineering (1990) and applied mathematics (1992) and Ph.D. degree in telecommunications (1996) from the Technical University of Sofia, and D.Sc. (Tech.) degree in information technologies from Tampere University of Technology (2003). He is Professor at Tampere University of Technology and Director of the Centre for Immersive Visual Technologies. He has published more than 30 articles in referred scientific journals, 4 book chapters and more than 145 papers in refereed scientific edited volumes and conference proceedings. He has actively contributed to the organization of the 3DTV series of conferences (being twice General Chair and four times Technical Program Chair), and served as a committee member and reviewer in a number of conferences such as EUSIPCO, IEEE ICME, IEEE ICIP, etc. He was the Scientific Coordinator of the EU FP7 project Mobile3DTV and currently he is the Coordinator of the European Training Network of Full Parallax Imaging. His recent research concentrates on methods and algorithms for multi-sensor 3-D scene capture, transform-domain light-field reconstruction, and Fourier analysis of 3-D displays.

Autoren/Hrsg.

Assunção, Pedro Amado

Gotchev, Atanas

Inhaltsverzeichnis

1;Contents;6
2;1 Introduction;12
3;2 Emerging Imaging Technologies: Trends and Challenges;15
3.1;Abstract;15
3.2;2.1 Introduction;16
3.3;2.2 Multiview Video Plus Depth;19
3.4;2.3 Standardization—The Status and Current Activities;24
3.4.1;2.3.1 Standardization in Multimedia;24
3.4.2;2.3.2 Basic Technologies;26
3.4.3;2.3.3 Multiview Video Coding;27
3.4.4;2.3.4 3D Video Coding;28
3.4.5;2.3.5 New Standardization Projects;31
3.5;2.4 Lightfield Super-Multiview with Camera Array;31
3.6;2.5 Lightfield with Microlens Array;35
3.6.1;2.5.1 Lightfield Raw Data-Based Approach;37
3.6.2;2.5.2 Multiview-Based Approach;38
3.6.3;2.5.3 Subsampled Grid of MIs Plus Disparity Approach;40
3.7;2.6 Free Navigation and Free Viewpoint Television;41
3.8;Acknowledgements;43
3.9;References;43
4;3 3D Content Acquisition and Coding;50
4.1;Abstract;50
4.2;3.1 Introduction;51
4.3;3.2 Effect of an Incorrect Camera Alignment on the Accuracy of the Spatial Reconstruction and Stereo Perception;52
4.3.1;3.2.1 The Influence of Inaccurate Camera Alignment;53
4.3.2;3.2.2 Influence of the Camera System Parameters and Spatial Position of the Object;57
4.3.3;3.2.3 Remarks;64
4.4;3.3 Compression Tools for Stereoscopic and Multiview Video;64
4.4.1;3.3.1 Stereoscopic Frame-Compatible Formats;65
4.4.2;3.3.2 Compression Tools for Stereoscopic Video;67
4.4.3;3.3.3 Performance Analysis of Compression Tools;70
4.4.4;3.3.4 Remarks;76
4.5;3.4 Multiview Video Compression for Arbitrary Camera Locations;77
4.5.1;3.4.1 Adaptation of 3D-HEVC to Nonlinear Camera Arrangements;77
4.5.2;3.4.2 Methodology of Evaluation;80
4.5.3;3.4.3 Results;81
4.5.4;3.4.4 Remarks;82
4.6;3.5 Recent Developments in Video Compression with Capabilities Beyond HEVC;82
4.6.1;3.5.1 UltraHD Video Compression Performance Beyond HEVC;84
4.6.2;3.5.2 Conversion and Coding for HDR/WCG Video;90
4.6.3;3.5.3 Projection Conversions and Coding for 360° Video;95
4.7;Acknowledgements;101
4.8;References;101
5;4 Efficient Depth-Based Coding;105
5.1;Abstract;105
5.2;4.1 Introduction;106
5.3;4.2 Depth Map Coding for Efficient Virtual View Synthesis;106
5.3.1;4.2.1 Algorithm Overview;107
5.3.2;4.2.2 Flexible Block Partitioning;108
5.3.3;4.2.3 Directional Intra Prediction;108
5.3.4;4.2.4 Constrained Depth Modelling Mode;110
5.3.5;4.2.5 Residual Signal Coding;111
5.3.6;4.2.6 Rate-Distortion Performance;112
5.4;4.3 Depth Compression Using Standard Coding Techniques;113
5.4.1;4.3.1 Bitrate Distribution;113
5.4.2;4.3.2 Depth Map Quality;114
5.4.3;4.3.3 Bitrate Distribution Between Texture and Depth;115
5.4.4;4.3.4 Coding Depth with Reduced Resolution;119
5.5;4.4 Conclusion;121
5.6;Acknowledgements;121
5.7;References;121
6;5 Error Concealment Methods for Multiview Video and Depth;123
6.1;Abstract;123
6.2;5.1 Introduction;124
6.3;5.2 Error Concealment for Multiview Video;125
6.3.1;5.2.1 Basic Methods Using Neighbouring Regions;125
6.3.2;5.2.2 Recent Advances in EC for Multiview Video;128
6.4;5.3 Methods for Error Concealment of Depth Maps;136
6.5;5.4 Conclusions;146
6.6;References;147
7;6 Light Field Image Compression;150
7.1;Abstract;150
7.2;6.1 Introduction;151
7.3;6.2 Light Field Image Representation;152
7.4;6.3 Light Field Image Coding Formats;153
7.4.1;6.3.1 Light Field Image Coding Using HEVC;154
7.4.1.1;6.3.1.1 Coding Efficiency;157
7.5;6.4 Scalable Light Field Coding for Backward Display Compatibility;159
7.5.1;6.4.1 Display Scalable Coding Architecture;162
7.5.2;6.4.2 Hierarchical Content Generation;163
7.5.3;6.4.3 Efficient LF Enhancement Layer Coding Solution;164
7.5.3.1;6.4.3.1 Self-similarity (SS) Prediction;164
7.5.3.2;6.4.3.2 Inter-layer (IL) Prediction;165
7.5.3.3;6.4.3.3 Intra Prediction;167
7.5.3.4;6.4.3.4 Header Formatting and CABAC;167
7.5.4;6.4.4 Performance Assessment;168
7.5.4.1;6.4.4.1 Overall DS-LFC RD Performance;170
7.5.4.2;6.4.4.2 Quality of Rendered Views;171
7.6;6.5 Sparse Set of Micro-lens Images and Disparities for an Efficient Scalable Coding of Light Field Images;173
7.6.1;6.5.1 Scalability;173
7.6.2;6.5.2 Displacement Intra and Inter Prediction Scheme;174
7.6.3;6.5.3 Encoding;174
7.6.3.1;6.5.3.1 Sparse Set of Micro-lens Images;175
7.6.3.2;6.5.3.2 Disparity Maps;176
7.6.3.3;6.5.3.3 Refinement by Inter and Intra Prediction;177
7.6.4;6.5.4 Decoding and Reconstruction;177
7.6.5;6.5.5 Evaluation;179
7.6.6;6.5.6 Remarks;180
7.7;6.6 Conclusions;180
7.8;References;181
8;7 Impact of Packet Losses in Scalable Light Field Video Coding;184
8.1;Abstract;184
8.2;7.1 Introduction;184
8.3;7.2 Scalable Light Field Coding;186
8.4;7.3 Mitigation of Packet Loss Impact on Scalable Light Field Coding;187
8.4.1;7.3.1 Relevant Factors for the Inter-layer Prediction Accuracy;188
8.4.2;7.3.2 Proposed Error Concealment Algorithm;190
8.5;7.4 Experimental Results;192
8.6;7.5 Conclusions;199
8.7;Acknowledgements;199
8.8;References;199
9;8 Transmission of 3D Video Content;201
9.1;Abstract;201
9.2;8.1 Introduction;202
9.3;8.2 DVB-T/T2, C/C2, and S/S2 Systems;202
9.3.1;8.2.1 DVB-T;203
9.3.2;8.2.2 DVB-T2;205
9.3.3;8.2.3 DVB-S/S2;207
9.3.4;8.2.4 DVB-C/C2;208
9.3.5;8.2.5 Transport of 3D Video in DVB Systems;210
9.4;8.3 Hybrid Broadcast/Broadband 3DTV;214
9.5;8.4 3D Video Delivery Over IP;216
9.5.1;8.4.1 HTTP and RTP-Based 3D/Multi-view Streaming;216
9.5.2;8.4.2 3D Video Distribution Over P2P Networks;218
9.6;8.5 3D Video Distribution in ICN;220
9.7;8.6 3D Stereo and Multi-view Video in Wireless Networks;221
9.8;8.7 Conclusion;224
9.9;References;224
10;9 3D Video Tools;228
10.1;Abstract;228
10.2;9.1 Introduction;229
10.3;9.2 Software Tools for 3D Video Compression;230
10.3.1;9.2.1 H.264 and 3D Extensions;230
10.3.2;9.2.2 HEVC and 3D Extensions;232
10.3.3;9.2.3 FFmpeg;233
10.3.3.1;9.2.3.1 FFmpeg as a Research Tool;234
10.4;9.3 Streamers and 3D Video Players;236
10.4.1;9.3.1 OpenSVC Decoder;236
10.4.1.1;9.3.1.1 3D Open SVC Decoder Extensions;237
10.4.2;9.3.2 VLC Player;239
10.5;9.4 Network Simulators, Emulators, Testbeds and Network Analysis Tools;239
10.5.1;9.4.1 Simulators;240
10.5.1.1;9.4.1.1 Sirannon;240
10.5.1.2;9.4.1.2 DVB-T Simulator in Simulink;241
10.5.1.3;9.4.1.3 DVB-T2 Simulator in MATLAB;242
10.5.1.4;9.4.1.4 NS-2;242
10.5.1.5;9.4.1.5 NS-3;244
10.5.1.6;9.4.1.6 ndnSIM;248
10.5.2;9.4.2 Emulators;248
10.5.2.1;9.4.2.1 Network Link Emulators;248
10.5.2.2;9.4.2.2 Virtual Network Emulators;249
10.5.2.3;9.4.2.3 Omnet++;249
10.5.2.4;9.4.2.4 MANE;250
10.5.3;9.4.3 Testbeds;252
10.5.3.1;9.4.3.1 PlanetLab;252
10.5.3.2;9.4.3.2 Network-Impairing Multimedia Testbed;253
10.5.4;9.4.4 Network Analysis Tools;255
10.5.4.1;9.4.4.1 Wireshark;255
10.6;9.5 3D Video Evaluation Tools;257
10.6.1;9.5.1 Generator of Degradations in 3D SBS Video Sequences;257
10.6.2;9.5.2 Crowd3D;259
10.6.3;9.5.3 3D MOS Using DSCQS;262
10.7;9.6 Conclusion;266
10.8;References;266
11;10 Quality of Experience and Quality of Service Metrics for 3D Content;271
11.1;Abstract;271
11.2;10.1 Introduction;272
11.3;10.2 Perceptual Characteristics of Multiview Content;275
11.3.1;10.2.1 Previous Work on QoE and QoS Assessment;278
11.3.2;10.2.2 Quality Assessment Based on Geometric and Spatial Distortions;279
11.3.3;10.2.3 Quality Based on Depth Map Analysis and Distortion;281
11.4;10.3 Subjective Quality Evaluation;284
11.4.1;10.3.1 Standard Methods for Subjective Quality Evaluation;284
11.4.2;10.3.2 Psychology/Neuroscience-Based Methodologies;286
11.4.3;10.3.3 High-Level QoE Factors;288
11.5;10.4 Conclusions and Future Directions;292
11.5.1;10.4.1 Measurement of Different Perceptual Attributes;293
11.5.2;10.4.2 Lack of 3D Image/Video Databases;293
11.5.3;10.4.3 Visual Attention Models to Develop RR and NR Quality Metrics;294
11.5.4;10.4.4 Need for a Standard for Subjective Experiments;294
11.6;References;295
12;11 3D Visual Content Datasets;302
12.1;Abstract;302
12.2;11.1 Introduction;303
12.3;11.2 Stereoscopic and Multiview Visual Content Datasets;304
12.3.1;11.2.1 Stereo Dataset Generation for Different Scene Cases;304
12.3.2;11.2.2 Multiview Camera Content for 3D Reconstruction, Modeling, and Visualization;307
12.4;11.3 Characterization and Selection of Light-Field Content for Perceptual Assessment;311
12.5;11.4 Special Point-Cloud and Holographic Content Datasets;314
12.5.1;11.4.1 JPEG Pleno Database: Point-Cloud Datasets;316
12.5.2;11.4.2 JPEG Pleno Database: Holographic Datasets;317
12.6;11.5 Datasets Annotated with Ratings from Subjective Experiments;317
12.6.1;11.5.1 3D Image Quality Databases;318
12.6.2;11.5.2 3D Video Quality Databases;320
12.6.3;11.5.3 3D Models Quality Databases;322
12.6.4;11.5.4 Eye-Tracking 3D Databases;322
12.7;11.6 Conclusions;324
12.8;References;324