|
When super resolution doesn't work
|
If you've already read
how super-resolution works, you know that it uses subpixel accurate
motion compensation to find similar areas in neighbor frames in order
to intelligenlty merge them combining the details. There are cases when this
can't be done successfully or cannot bring new details.
Stills
First of all, if an object doesn't move or change at all, if it's identical
in several frames, then there's no additional information to extract. The
quality will be as of simple spatial upsize.
Very fast motion or change
Opposite extreme is very fast motion or change. If an object moves too fast,
it's difficult to track its motion, and often it becomes blurred due to two
reasons. One reason is motion blur - when camera's exposure is set so that
object moves a notable distance during shooting of one frame. The other reason
is compression. If the video is compressed with a method that uses delta-frames
(like MPEG1-2-4, WMV and many others), intense motion creates a lot of
differences between frames so the codec quantizes data stronger to fit into
required bitrate, so more details are lost on the way.
Just too strong compression
If your video is compressed to a low bitrate, in many cases this is very bad
for super-resolution. There are two types of lossy video codecs: those that
use delta-frames and those that use only key-frames.
If the video is compressed heavily by a
key-frame-only codec like MJPEG (DV is also such codec, but DV video always
has high enough bitrate), then each frame is compressed independently and a
lot of details are lost in each frame. This usually leads to an artefact
called "blocking" (visible squares in the picture), you can see it in strongly compressed JPEG images.
When an object moves in the video, the blocks don't move, so the object
is changing a lot during this motion making it impossible to accurately
track its motion and get some details. What you can get from other frames
is their blocking artefacts, making picture even worse.
If the video is heavily compressed by a codec with delta-frames (most
common codecs), then the codec performs motion compensation, gets the
difference between frames and quantizes it losing all details and leaving
only major differences. When we try to apply super-resolution, it performs
similar motion compensation to use differences between frames and it finds
almost no details, only gross changes that cannot be used to improve the frame.
One practical conclusion is that there's no sense in trying to use
Video Enhancer on video shot with mobile phones.
Other articles...
|
|
|
