Understanding the Human Eye's Role in Video Compression
Written on
Chapter 1: The Human Eye and Video Compression
When it comes to utilizing video compression standards, a deep understanding of the human eye isn't strictly necessary. However, grasping the basics can significantly enhance your comprehension of why these standards are structured as they are. Below are five key insights into the human eye that have a direct bearing on video compression techniques.
Section 1.1: Color Sensitivity
The human eye is particularly attuned to green hues, followed by red, with blue being the least perceived.
Impact: This sensitivity informs the design of algorithms and video equipment, which prioritize capturing green over other colors. As humans, our perception of brightness is closely linked to the amount of green present in a scene. Consequently, cameras are engineered to capture more green. This explains the preference for green screens in video production; they are inherently brighter and require less lighting than alternatives. Additionally, the green channel tends to hold more information, resulting in cleaner keying around subjects. The algorithms we develop often aim to preserve green more effectively due to our heightened sensitivity to it.
Section 1.2: Maximum Visual Resolution
The human eye can theoretically perceive up to 576 megapixels.
Impact: This remarkable capability drives researchers and developers to create 8K television technology and codecs for compressing such high resolutions. Yet, it’s essential to note that this peak resolution is only relevant to a small area in the center of our vision, known as the fovea, which only accounts for about 1% of the retina. In practical terms, most of what we see is at a resolution between 5 and 15 megapixels. Thus, many believe that 8K may represent the ceiling of visual quality, questioning the need for more advanced capture and compression techniques.
Video Title: Lesson 1.21 | How Human Eye Perceives Light vs. Cameras
Description: This video explores the differences between how the human eye perceives light and how cameras capture it.
Section 1.3: The Blind Spot and Predictive Coding
Your brain compensates for the blind spot created by the optic nerve.
Impact: Video compression algorithms utilize a similar concept of 'filling in' information through predictive coding. Many compression techniques rely on keyframes, which contain complete data for specific frames. Subsequent frames are encoded based on their differences from these keyframes. When there is movement, algorithms predict the motion and adjust their data storage accordingly, mirroring how our brains fill in gaps in visual information.
Section 1.4: Sensitivity to Luminance
Humans are more responsive to luminance (brightness) than to chrominance (color).
Impact: The YUV color encoding framework is structured around human vision, recognizing that we are less sensitive to color variations. This allows for reduced bandwidth usage for color data. Therefore, any transmission errors or compression artifacts tend to be less noticeable compared to those encountered with the RGB color model. The decisions made in video compression involving YUV are based on our varying sensitivities to different aspects of visual information.
Video Title: How Your Eyes Work
Description: This video delves into the mechanics of vision, explaining how the human eye processes light and color.
Section 1.5: Frame Rate Perception
The human eye can detect approximately 30 to 60 frames per second (FPS).
Impact: Most content is produced at 24, 30, or 60 FPS, with ongoing debates about the true limits of our frame rate perception. Gamers often assert that the human eye can discern more than 60 FPS, a common misconception that confuses FPS with Hz (Hertz), which refers to refresh rates rather than frame rates. Regardless of these discussions, the limitations of human vision often dictate that most streaming content operates at 60 FPS or lower. Video games may push past this limit, but evidence remains inconclusive regarding human capacity to perceive beyond 60 FPS.