Nvidia optical flow accelerators can also use AI frame generation for video encoding

Something to look forward to: Many consumers are looking forward to this year’s new graphics cards for their gaming capabilities, but they are also introducing new tools for video encoding. Nvidia’s RTX 4000 series GPUs add an extra trick to double frame rates when encoding video.

The optical flow accelerators behind Nvidia’s new DLSS 3 feature won’t just increase video game frame rates. Content creators can also use technology to artificially create increase frame rates in the videos they encode.

While DLSS 2.0 uses the Tensor cores of Nvidia’s RTX 2000 and 3000 GPUs to generate new pixels through machine learning, DLSS 3 uses the 4000 series optical flow accelerators to create new frames. PC games that support DLSS 3 can double their frame rates in addition to the performance gains of DLSS 2.0, but Nvidia’s technology can bring the same improvements to videos.

Motion vectors are a tool used by DLSS to improve game frame rates, and Nvidia also uses them in what it calls engine-assisted frame rate up-conversion (FRUC). It’s basically a form of hardware-assisted motion interpolation. The concept is similar to how TVs can smooth and interpolate motion, but the RTX 4000’s CUDA cores and optical flow accelerators make the process faster and more accurate. When interpolated images have artifacts, image-domain hole-filling techniques can fill them in to create an accurate final image.

The FRUC library APIs support both ARGB and NV12 input surface formats. They are also compatible with all DirectX and CUDA applications.

The enhanced motion interpolation could differentiate Lovelace from Intel’s Arc Alchemist series and AMD’s RDNA3 GPUs, as all three introduce GPU-based AV1 encoding. Early tests show that AV1 has great advantages over H.264 encoding in terms of speed, data usage, and image quality. The new format allows streamers and content creators to more efficiently encode high-resolution video. Unlike H.265, AV1 is also royalty free.

Google is also pushing AV1 encoding as the format becomes increasingly prominent on YouTube. This week the company published a significant update for its open source AV1 encoder – AOM-AV1 3.5 – which now supports frame parallel encoding for more threads. Depending on video resolution and CPU thread count, the update could reduce encoding times by 18-34%.

About Debra D. Johnson

Check Also

Luca Guadagnino: 5 films that I consider to be cinematic masterpieces

On paper, it’s easy to guess what attracted Luca Guadagnino to lead bones and all. …