Excerpts

In the past decade or so, there have been fascinating developments in multimedia representation and communications. First of all, it has become very clear that all aspects of media are "going digital"; from representation to transmission, from processing to retrieval, from studio to home. Second, there have been significant advances in digital multimedia compression and communication algorithms, which make it possible to deliver high-quality video at relatively low bit rates in today's networks. Third, the advancement in VLSI technologies has enabled sophisticated software to be implemented in a cost-effective manner. Last but not least, the establishment of half a dozen international standards by ISO/MPEG and ITU-T laid the common groundwork for different vendors and content providers. At the same time, the explosive growth in wireless and networking technology has profoundly changed the global communications infrastructure. It is the confluence of wireless, multimedia, and networking that will fundamentally change the way people conduct business and communicate with each other. The future computing and communications infrastructure will be empowered by virtually unlimited bandwidth, full connectivity, high mobility, and rich multimedia capability. As multimedia becomes more pervasive, the boundaries between video, graphics, computer vision, multimedia database, and computer networking start to blur, making video processing an exciting field with input from many disciplines. Today, video processing lies at the core of multimedia. Among the many technologies involved, video coding and its standardization are definitely the key enablers of these developments. This book covers the fundamental theory and techniques for digital video processing, with a focus on video coding and communications. It is intended as a textbook for a graduate-level course on video processing, as well as a reference or self-study text for researchers and engineers. In selecting the topics to cover, we have tried to achieve a balance between providing a solid theoretical foundation and presenting complex system issues in real video systems. SYNOPSIS Chapter 1 gives a broad overview of video technology, from analog color TV system to digital video. Chapter 2 delineates the analytical framework for video analysis in the frequency domain, and describes characteristics of the human visual system. Chapters 3-12 focus on several very important sub-topics in digital video technology. Chapters 3 and 4 consider how a continuous-space video signal can be sampled to retain the maximum perceivable information within the affordable data rate, and how video can be converted from one format to another. Chapter 5 presents models for the various components involved in forming a video signal, including the camera, the illumination source, the imaged objects and the scene composition. Models for the three-dimensional (3-D) motions of the camera and objects, as well as their projections onto the two-dimensional (2-D) image plane, are discussed at length, because these models are the foundation for developing motion estimation algorithms, which are the subjects of Chapters 6 and 7. Chapter 6 focuses on 2-D motion estimation, which is a critical component in modern video coders. It is also a necessary preprocessing step for 3-D motion estimation. We provide both the fundamental principles governing 2-D motion estimation, and practical algorithms based on different 2-D motion representations. Chapter 7 considers 3-D motion estimation, which is required for various computer vision applications, and can also help improve the efficiency of video coding. Chapters 8-11 are devoted to the subject of video coding. Chapter 8 introduces the fundamental theory and techniques for source coding, including information theory bounds for both lossless and lossy coding, binary encoding methods, and scalar and vector quantization. Chapter 9 focuses on waveform-based methods (including transform and predictive coding), and introduces the block-based hybrid coding framework, which is the core of all international video coding standards. Chapter 10 discusses content-dependent coding, which has the potential of achieving extremely high compression ratios by making use of knowledge of scene content. Chapter 11 presents scalable coding methods, which are well-suited for video streaming and broadcasting applications, where the intended recipients have varying network connections and computing powers. Chapter 12 introduces stereoscopic and multiview video processing techniques, including disparity estimation and coding of such sequences. Chapters 13-15 cover system-level issues in video communications. Chapter 13 introduces the H.261, H.263, MPEG-1, MPEG-2, and MPEG-4 standards for video coding, comparing their intended applications and relative performance. These standards integrate many of the coding techniques discussed in Chapters 8-11. The MPEG-7 standard for multimedia content description is also briefly described. Chapter 14 reviews techniques for combating transmission errors in video communication systems, and also describes the requirements of different video applications, and the characteristics of various networks. As an example of a practical video communication system, we end the text with a chapter devoted to video streaming over the Internet and wireless network. Chapter 15 discusses the requirements and representative solutions for the major subcomponents of a streaming system. SUGGESTED USE FOR INSTRUCTION AND SELF-STUDY As prerequisites, students are assumed to have finished undergraduate courses in signals and systems, communications, probability, and preferably a course in image processing. For a one-semester course focusing on video coding and communications, we recommend covering the two beginning chapters, followed by video modeling (Chapter 5), 2-D motion estimation (Chapter 6), video coding (Chapters 8-11), standards (Chapter 13), error control (Chapter 14) and video streaming systems (Chapter 15). On the other hand, for a course on general video processing, the first nine chapters, including the introduction (Chapter 1), frequency domain analysis (Chapter 2), sampling and sampling rate conversion (Chapters 3 and 4), video modeling (Chapter 5), motion estimation (Chapters 6 and 7), and basic video coding techniques (Chapters 8 and 9), plus selected topics from Chapters 10-13 (content-dependent coding, scalable coding, stereo, and video coding standards) may be appropriate. In either case, Chapter 8 may be skipped or only briefly reviewed if the students have finished a prior course on source coding. Chapters 7 (3-D motion estimation), 10 (content-dependent coding), 11 (scalable coding), 12 (stereo), 14 (error-control), and 15 (video streaming) may also be left for an advanced course in video, after covering the other chapters in a first course in video. In all cases, sections denoted by asterisks (*) may be skipped or left for further exploration by advanced students. Problems are provided at the end of Chapters 1-14 for self-study or as homework assignments for classroom use. Appendix D gives answers to selected problems. The website for this book ( www.prenhall.com/wang ) provides MATLAB scripts used to generate some of the plots in the figures. Instructors may modify these scripts to generate similar examples. The scripts may also help students to understand the underlying operations. Sample video sequences can be downloaded from the website, so that students can evaluate the performance of different algorithms on real sequences. Some compressed sequences using standard algorithms are also included, to enable instructors to demonstrate coding artifacts at different rates by different techniques. Excerpted from Video Processing and Communications by Yao Wang, Ya-quin Zhang, Joern Ostermann, Ya-Qin Zhang All rights reserved by the original copyright owners. Excerpts are provided for display purposes only and may not be reproduced, reprinted or distributed without the written permission of the publisher.