As the transition from analog to digital content continues in the AV (audio-visual) world, the number of new consumer devices supporting digital audio and video processing, including TV-sets, and DVD players, continues its rapid climb. As a result, it became one of the most widely adopted compression standard in the consumer space. H.264 is now poised to replace MPEG-2 as the new compression technology with better quality at lower transmission bitrate and storage requirements.

January 10, 2007
URL:http://www.drdobbs.com/embedded-systems/h264-codec-video-compression-for-consume/196802802

A New Video Compression Standard and its Paradigm
As the transition from analog to digital content continues in the AV (audio-visual) world, the number of new consumer devices supporting digital audio and video processing, including TV-sets, DVD players, Digital Still Image Cameras (DSCs), and Digital Video Camcorders (DVCs), continues its rapid climb. Every portable personal device these days seems to include audio-visual features, including mobile phone, portables media players and PDA. In addition, the PC, networking and the broadband technologies are also doing their part to fuel this transition by facilitating the creation, streaming and sharing of content.

The desire to create and share AV content in digital format was one of the key factors contributing to the creation of international compression standards. The MPEG (moving picture expert group) and JPEG (joint photographic expert group) committees created the original standards that allowed the transition from analog AV to digital AV formats, allowing the sharing and transfer of multimedia content via networks or portable storage media (e.g. DVD-disc, SD-card, etc.). However, with the major paradigm shift in the consumer space from content consumption to content creation, there is an increasing need for higher quality content creation, larger volume of content storage, and easier content transmission/sharing.

In typical AV products, the video content consumes most of the processing power, storage space and communication channel bandwidth. MPEG-2 video compression technology was instrumental in the transition from analog to digital video processing. As a result, it became one of the most widely adopted compression standard in the consumer space. H.264 is now poised to replace MPEG-2 as the new compression technology with better quality at lower transmission bitrate and storage requirements. These improvements are critical in cost-sensitive consumer products and especially in portable consumer devices.

The H.264 (also known as MPEG-4-AVC ) standard was completed at the end of 2005 by a joint effort of the ITU-T and the MPEG organizations. This standard greatly improves the MPEG-2 compression ratio in a wide range of applications from small size (mobile AV) to HD (high definition) video. Many applications of H.264 technology have already been launched, including Blu-Ray Disc, HD-DVD, AVCHD-Camcorders (HD-CAM), iTune-video, 1-seg DTV (mobile digital TV in Japan), DVB-H for European handheld DTV, among others. There are two key factors behind the wide adoption of H.264:

H.264 offers between 2-to-1 and 3-to-1 improvement over MPEG-2 in compression ratio, resulting in significant savings in video storage capacity and network bandwidth.

H.264 provides efficient implementation of Internet based application because of its well considered scheme to harmonize with network protocols. Networks are proliferating at home, office and anywhere in between! The bandwidth and quality of service of wired and wireless networks is increasing, and video streaming over these networks is already commonly seen in cable, satellite, IPTV and other applications. The combination of H.264 video compression technology and this ubiquitous broadband network connectivity is fueling an era of mobile digital life style.

All indications are that this era has already begun and is gaining momentum rapidly. The following table of AV products market in Japan supports this view:

Table-1 Consumer AV products market (Japan)

Notes:

1. Above figures (unit-sales) were derived from JEITA (Japan electronics and information technology industries association) and CIPA (Camera & Imaging Products Association).
The figures in 2006 are predicted from sales statistics of January thru October/2006.
2. Prediction of 2007 is anticipated from monthly figures of 2006.
3. In Japan, most mobile phones include DSC.
4. A percentage of 3G mobile phones contain video telephony and its services.
5. A percentage of the mobile phones are equipped with 1-seg DTV receiver.

The above table highlights the following key points:

• Portable consumer device segment (mobile phone, DSC) is a big and growing market.
• The evolution of personal portable devices is having a notable change in our life style, in particular the way we communicate.

Beside the changes in the portable market shown in table-1, video in the home is rapidly transitioning towards High Definition (HD) quality. Similarly video in portable devices is moving toward Standard Definition (SD) or VGA quality from QVGA or smaller. There is a very clear trend in consumer AV (audio-visual) world and market where many products are becoming personal mobile devices rather than desk-top and home centric devices.

H.264's advanced algorithms offer higher compression, lower communication channel bandwidth, smaller storage requirements and flexible scheme for AV networking; however, those algorithms also require more processing power to encode and decode the video content. Obviously the first requirement of any mobile device is low power consumption for longer battery life. In the following sections, we would summarize how Qpixel's H.264 compression ICs satisfy the video quality and low power requirements of portable consumer devices.

Technological Summary of H.264 with Comparison to MPEG-2
Background of H.264 standardization
H.264 development was started in 1998 as a long term project of ITU-T, and its target was to achieve half the bit-rate of the H.263 and MPEG-4 standards, for the same perceived video quality. Unlike past video compression standards (MPEG-1/-2/-4, H.261/262/263), the H.264 effort was to improve compression efficiency and not restrict the implementation complexity. This resulted in the current H.264 implementation that requires substantial amount of computational power. The resulting H.264 encoder offers improved features in which the encoder structure and compression profiles are quite different from MPEG-2.

H.264 Encoder Structure

This diagram indicates a generalized hybrid (motion-compensation and transform) H.264 encoder structure which is common to MPEG-2 and H.264 encoders; however the details in each box in the block diagram are different between Codecs. Table-2 shows the differences from MPEG-2 for those items which improve video quality significantly.

Table-2 H.264 Encoder Improvements Versus MPEG-2 Encoder

Table-3 lists the various profiles offered in the H.264 standard, along with a brief description, as well as target applications.

Table-3 H.264 Application Oriented Profiles

The first four profiles (Baseline, Main, Extended, High) are intended for consumer applications, while the following three profiles (High 10, High 4:2:2, High 4:4:4) are intended for non-consumer (professional) applications. There are also "Level" definitions that are similar to MPEG-2. Level-3 is for SD (standard definition: 720x480x30 or 720x576x25), and Level-4 is for HD (high definition: 1920x1080x30 or x25).

Qpixel's Approach and Solution to the AV Market Requirements
Considering the AV market needs and the H.264 technological characteristics, we directed our efforts as follows:

• Lowest power Codec (Encoder/Decoder)suitable for use with portable applications.
• High quality standard definition video processing (with an eye towards high definition resolutions in the future).
• Specialized H.264 Codec algorithm and architecture to meet specific market requirements. More specifically, we selected "Main-profile at Level-3" as the most suitable profile for standard definition applications with very good video quality.

Meeting the Quality Requirement (Algorithmic Efforts)
Using the basic theory of video compression technology suggests the following:

(a) The human psycho-visual system ignores the detail of high-frequency textures.

(b) The resultant bitrate can be reduced by finding the best motion-estimation match among many possible options.

(c) Entropy coding results in the least number of bits without loss of information.

Items (a) and (b) required many algorithmic iterations to find out the best human-eye perception for a given bitrate. Item (c) required an innovative implementation.

In general, the Encoder is required to generate a standard compliant bit-stream, while the video quality and compression efficiency are not specified in the H.264 standard. However, our goal is to achieve high quality at low bit-rate under low-power conditions. These criteria are the focus of our efforts. The following summarize our algorithmic efforts and direction:

• Break the highly sequential structure of H.264's syntax chain into a smaller number of syntax elements without sacrificing compression efficiency, thus allowing more parallelism where processing power becomes a bottleneck.
• Intelligently pre-select more efficient compression algorithms without sacrificing video quality, thus reducing a required overall processing power.
• Implement the bursty and content-dependent characteristics of CABAC in a highly scalable manner, thus rendering CABAC more tractable for hardware implementation.
• Implement B pictures in a way which best balances its video quality with implementation costs.
• Re-use data effectively to maximize the performance of our video algorithms.
• Highly sequential syntax structure with syntax inter-dependency.
• Long-term memory allowed among syntax elements.

Our technology comes from a thorough, careful analysis of how each coding tool in the H.264 encoder contributes to the video quality as well as the implementation cost in an ASIC environment. Starting from scratch without any bias from pre-existing coding standards such as MPEG-1/2/4, our technology group focused on the best cost-performance advantage for each block in the H.264 encoder. In addition to the video coding algorithm, a highly intelligent TDM-based scheme was adopted to reduce the amount of data traffic required to handle the large amount of video data, thus further reducing power consumption.

Meeting the Physical Requirements
The following is a summary of the architectural decisions that were made to meet our target the physical requirements:

• The Qpixel H.264 video compression IC was designed to target the portable consumer devices market. It is well understood that the H.264 implementation complexity is almost 10 times as much as MPEG2. To achieve high video quality and simultaneously reduce the system power consumption, we produced a flexible architectures that is power conscious using state of the art design techniques to reduce die size and power consumption while efficiently handling data traffic to minimize the inefficient, power consuming interactions with external memory.
• The major factor impacting power consumption is the IC clock tree. In order to fully utilize the potential of H.264 encoding tools to achieve high video quality, the clock frequency must be increased to provide more processing power. However running an IC at higher frequencies directly increases dynamic power consumption, so compromises needed to be made in choosing the performance target and power dissipation of the system.
• To achieve the highest performance with a given power target, the IC was architected so that external memory stall-cycles were minimized and bus arbitration was optimized to yield high data transfer efficiency.
• In light of the above considerations, two innovative schemes were implemented to reduce external memory data transfers. One is a frame mapping scheme and the other is TDM bus arbitration. TDM arbitration is highly optimized for our video processing pipeline and frame mapping is optimized for the highest DRAM bandwidth. The H.264 compression tools are carefully evaluated and selected for the trade off between video quality and the DRAM bandwidth requirements to meet the low power budget.

Conclusion and the Next Steps

• Our unique patented and patent-pending video algorithm and architecture made it possible to implement the Qpixel H.264 Main profile codec in a way best suited for video products requiring both low-power and high video quality.
• In next generation consumer AV products, there is a need for HD (high definition) resolution and quality. We will continue to apply our innovative algorithmic and architectural techniques to offer highly competitive solutions for HD applications.

References

• JVT-N050d1.doc
• H.264 / AVC TEXTBOOK, Authored by S. Okubo et al, Impress Limited
• Development of Video Coding Technology and Future Prospects, T. Chujo, Toshiba Review Vol.60 No.7, 2005
• A Study for Bit-rate Control of H.264/AVC Coding, Y. Akima, Graduate Paper of Waseda University, 2004

About the authors
Mingning Gu received his Masters Degree in Computer Science from University of Wisconsin, Madison. He has over 13 years of experience in designing ASICs for video processing and compression standards such as MPEG1, MPEG2, and H.264. Mr. Gu is currently the Director of Architecture at Qpixel Technology, where he's heading up all architecture design activities. Prior to Qpixel, Mr. Gu was the Manager of the ASIC team at VWeb, working on MPEG2 ICs.

Yasuhiro Yamada received his Masters Degree in Electrical Engineering from Doshisha University, Kyoto, Japan in 1971. He then joined JVC R&D Laboratory, where he spend most of his time managing the digital audio-visual technology research and development. He was an active member of MPEG Committee during MPEG-1 and MPEG-2 standardization. He retired JVC in 2005 and is currently working at Qpixel Technology Inc. for its technology strategy.

Hitoshi Watanabe received his Bachelor of Engineering degree from Department of Applied Physics, University of Tokyo, Tokyo, Japan in 1990. He then moved to Stanford University and received his Ph.D. degree from Department of Applied Physics in 1999. Following the completion of his research at Stanford, he joined the R&D team of Pulsent corp. His contributions include the development of a proprietary codec based on object-based video compression algorithm, aiming at 2x coding advantage over H.264, object-based coding algorithm, image segmentation/object extraction, and video pre/post processing. After Pulsent, Dr. Watanabe joined Qpixel as a founding member and has been leading a team to develop Qpixel's core H.264 encoder algorithm. Dr. Watanabe holds patents for H.264 as well as object-based video compression algorithm.

All of the authors can be reached at [email protected].

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.