There’s More to Video Quality than Resolution: 1080p vs. 4K


Is resolution the only factor?

With no doubt higher resolution is able to give us more fine details in images we captured, and higher resolution requires higher pixel number of course. That’s the reason we saw image sensor vendors put more and more pixels in their devices, year after year, to increase resolution. You even can see some cell phone already carried over 20Mp sensor. Yes, high resolution is supposed to be good, but why iPhone, as the most popular camera, insisted to offer us only 8Mp (Mega pixel) till last year. Apple must be mean? Maybe, but I have to admit my iPhone6 plus gives me nice pictures always… DXO even considered iphone6s was the best smartphone camera in 2014, it beat all other higher resolution cameras. Therefore, there should be some other factor impacts on quality, not only pixel number.

Let’s turn to look at CANON, as the world class camera vendor; they must know all the factors.

PowerShot G series, this semi-pro solution offers us very good quality with compact size (compare to DSLR), over 15 years.

Here’s a figure shows its resolution evolution.

Till 2008, this figure looks quite reasonable, Canon increased pixel to 14Mp to catch up the resolution of film. But the count dropped from 2009. It seems they tend to agree with Apple, pixel number isn’t everything. Fortunately CANON spec tells us more: the image sensor size. We can get pixel size roughly by divide sensor size to H/V resolution.

What it tells us? Since 2000, the pixel size gets smaller and smaller, so that Canon can integrate more pixels into compact body. While they regret in 2010, they increase pixel size even it hurt the resolution. (You can google tons of complain about G10 actually, here’s an interesting comparison)

Look at the latest G serial, they increased sensor size when offering more pixels in order to avoid using too small pixel.

Yes, Pixel Size, this is the hidden factor.

 

Why pixel size matters?

Pixel number is very easy to understand for people, it stands for resolution in camera spec war, higher is better. But how understand pixel size, why it matters? Let’s check how image sensor works at first.

The sensor in today camera used is almost all CMOS sensor, which consists of 2D pixel array. Each pixel absorbs photons and generates electrons so that latter circuit can digitize it. To help you to understand easily, just consider one pixel is a bucket. After light pass through the lens (limited by aperture), it starts raining, photon rain. (This visual term is given by Clark, who described this more accurate)

It’s easy to image the benefits from the bucket size: a bigger bucket has more chances to collect photon than a small one, so it collects more photons in a given time. Additionally, a big bucket can hold more photons a time, less chance to overflow.

Now let’s use some scientific terms to define the benefits we get:  Sensitivity

In a given time, usually we call it exposure time in camera system, sensitivity is how much the sensor outputs based on the input quantity being measured. (Assume FNO & ISO are same) this is how we define sensitivity. Since our pixel size is bigger, it collects more photons (input quantity) to generate more electrons so that A/D circuit can output higher level.

 

SNR (signal-to-noise ratio)

The accuracy of the signal measured is directly proportional to the size of the signal. In the physics of photon counting, the noise in the signal is equal to the square root of the number of photons. This benefit is not that straightforward, since you can see more photons introduce more noise, but we don’t like noise… well, check a sample below, a step chart encoded by a real camera.

You must think it’s noisier in dark patches, while cleaner in whiter patches, right? However the truth is sensor noise tends to increase with brightness, e.g. more photons.

Here’s the answer, when we consider an image is noisy, usually it’s because lack of SNR, not the absolute count of noise. Therefore, higher SNR signal gives us cleaner image, which leads you think whiter patches look clearer. Let’s do some simple calculation here:

Now you see the benefit if we can collect more photons.

(Please note SNR isn’t the only thing impacts on our feeling, noise pattern also matters.)

 

Dynamic range

Dynamic range is important in photography, it could be defined how many stops we can record from real world.

Think we are capturing an image with rich tones, and we don’t want pitch black in shadow so we set enough exposure time to record shadow. Then what happens in highlight? Overexposure, if we don’t have big enough pixel! Because the bucket there is full and overflow, we cannot digitize its value anymore.

How about increase the wall height of bucket, to holds more water? Unfortunately that doesn’t work. Electrons get generated only if photon can hit the bottom of pixel, the energy of photons hit to wall is wasted. A pixel isn’t a real bucket after all. Instead, sensor vendors keep working on lower the wall, to improve light gathering.

Now we know the pixel size really matters. When we talk about image quality, we must consider it, which defines quality with resolution together. General speaking, bigger is better.

 

Video quality requires more

Frame rate

Video is composed of frames which is an electronically coded still image. So other than image resolution, we have another term “frame rate” to define video standard, like 1080p60, 4Kp30. Frame rate is the number of frame per second (well known as fps) we record still images. Generally, the higher frame rate, the smoother of motion. Only offering high quality still image isn’t good enough in video case, since the reason we record video is we want to capture motion after all.

Therefore, when talking about video quality, we need figure out what’s the fps, which is a major factor as well as resolution. 4K sounds good, but how about 4Kp15? Trust me, you won’t like to walk as zombie in your own video. For a motion camera, 30fps is far away from enough. If I have to choose, I would like to take fps as priority instead of resolution, because I care about motion more in video application.

 

Big image sensor

Once we raise the fps bar higher, we definitely need to consider about sensor carefully. At first, we need a sensor is able to output fps as we required at least. (This is part of reason CCD got replaced by CMOS, not fast enough)

Second, in video application, the exposure time is limited by frame rate, for instance, the maximum exposure time is 1/30s in 30fps case, otherwise sensor output cannot catch up the frame rate we want. Then what else we can do to maintain a bright video? A faster lens? Yes, it helps in some case. But fast lens usually suffers from flare, also hurts depth of field.

From what we learn from pixel size, a better solution is a big sensor. We can collect enough photons in limited time, and maintain good SNR so that we can keep each frame bright and clean.

 

High performance DSP

Each camera system has a DSP to process RAW images from sensor, then encode to JPEG(still camera) or H.264 video stream(video camera). Unlike still camera, its DSP has plenty time to process RAW image, since there’s no frame rate requirement. The performance of video camera is more critical, the process has to be done in 1/fps second. Once it misses, a frame drop case happens, then we can see a motion jumping in video, very bad experience.

Also, its encoding efficiency must be good. RAW image size is huge, so people invented JPEG to encode it so that we store more. While video stream keep generating frames per second, higher frame rate generates more. JPEG isn’t good enough, so we have H.264 standard here to record frames as low bits as possible, but keep the quality as high as JPEG. See how a good encoder helps here? To save our money for storage.

If the DSP is more powerful, has extra performance to do something else, we can use it to do oversampling. What is oversampling? It’s sampling signals with much higher sampling frequency than Nyquist Rate. In image process, it’s using much higher resolution than 1080p, like 4Mp, 6Mp, then downscale to 1080p.

You can do it on your laptop to see the advantage, zoom out a noisy, big resolution image. What do you see then? You get a sharp and clean small size image for free!

 

Do You Have Any Questions?

If you have any questions regarding our video quality, please do not hesitate to contact us at info@waylens.com

Thank you for your support!