• AnyStream is having some DRM issues currently, Netflix is not available in HD for the time being.
    Situations like this will always happen with AnyStream: streaming providers are continuously improving their countermeasures while we try to catch up, it's an ongoing cat-and-mouse game. Please be patient and don't flood our support or forum with requests, we are working on it 24/7 to get it resolved. Thank you.

Guide Video Enhancement for NVIDIA RTX20xx (or later) GPU Owners

tectpro

Translator (ms_MY)
Thread Starter
Joined
Feb 27, 2011
Messages
1,414
Likes
949
Hello Community,

After many questions, I created a separate thread focusing on NVIDIA video enhancement from the original thread here:
https://forum.redfox.bz/threads/reencoding-videos-via-free-command-line-tools-and-gpu.88489/

This thread is mainly for those who own NVIDIA RTX20xx series GPUs (or later) and prefer command-line interfaces to GUI-based tools.


Why This Might Be for You:
  • Cost-Effective: Utilizes your existing NVIDIA hardware; no need for additional expensive AI software.
  • Quality Enhancement: Leverages NVIDIA's AI to upscale and denoise videos.

What You Need:
  1. NVIDIA RTX 20xx GPU or later
  2. Latest NVIDIA Drivers: Ensure your GPU drivers are up to date.
  3. Video Effect Models and Runtime Dependencies: Download the latest versions suitable for your GPU from NVIDIA's Resource Page
    Code:
     https://www.nvidia.com/en-sg/geforce/broadcasting/broadcast-sdk/resources/
  4. Rigaya NVENC: Get the latest release from Rigaya's GitHub
    Code:
     https://github.com/rigaya/NVEnc/releases

Installation & Setup:
  • Install the NVIDIA drivers and Video Effect models.
  • Extract the Rigaya NVENC files to a designated folder.
  • In the same folder, create a text file named enhance.txt.
  • Copy and paste the following command-line arguments into enhance.txt:

    Code:
    --avsw
    --codec hevc
    --cqp 20:22:25
    --output-res -2x1080
    --vpp-resize nvvfx-superres
    --preset quality
    --output-depth 10
    --lookahead 32
    --bframes 5
    --ref 6
    --multiref-l0 7
    --multiref-l1 3
    --bref-mode each
    --mv-precision Q-pel
    --level auto
    --profile main10
    --colormatrix auto
    --colorprim auto
    --transfer auto
    --chromaloc auto
    --max-cll copy
    --master-display copy
    --atc-sei auto
    --dhdr10-info copy
    --aud
    --repeat-headers
    --pic-struct
    --split-enc auto_forced
    --audio-copy
    --audio-disposition copy
    --audio-metadata copy
    --chapter-copy
    --vpp-deband
    --vpp-nvvfx-denoise
    --vpp-nvvfx-artifact-reduction


    A brief overview of the settings.
    • --avsw: Read input file using avformat + ffmpeg's software decoder.
    • --codec hevc: Set the output video codec to HEVC (H.265).
    • --cqp 20:22:25: Use the Constant Quantization Parameter (CQP) rate control method with specified QP values for I, P, and B frames.
    • --output-res -2x1080: Set the output resolution, maintaining the aspect ratio and setting the height to 1080 pixels.
    • --vpp-resize nvvfx-superres: Use NVIDIA's VFX super-resolution for resizing.
    • --preset quality: Use the "quality" preset for encoding, prioritising video quality over encoding speed.
    • --output-depth 10: Set the output video bit depth to 10 bits per color channel.
    • --lookahead 32: Enable lookahead with 32 frames to improve quality by analyzing future frames.
    • --bframes 5: Set the maximum number of consecutive B-frames to 5.
    • --ref 6: Use 6 reference frames for prediction.
    • --multiref-l0 7 and --multiref-l1 3: Set the number of multi-references for list 0 and list 1 in B-frames.
    • --bref-mode each: Use each B-frame as a reference for other frames.
    • --mv-precision Q-pel: Set the motion vector precision to quarter-pixel.
    • --level auto: Automatically set the encoder level based on other settings and input.
    • --profile main10: Use the Main 10 profile for HEVC, which supports 10-bit color depth.
    • --colormatrix auto, --colorprim auto, --transfer auto, --chromaloc auto: Copy color matrix, color primaries, transfer characteristics, and chroma sample location based on the input or defaults.
    • --max-cll copy, --master-display copy: Copy the Max CLL (Maximum Content Light Level) and Mastering Display Color Volume metadata from the source if available.
    • --atc-sei auto, --dhdr10-info copy: Automatically handle ATC SEI (Alternative Transfer Characteristics SEI) and copy dynamic HDR10+ information if present.
    • --aud, --repeat-headers: Insert Access Unit Delimiters and repeat VPS, SPS, and PPS headers at every IDR frame.
    • --pic-struct: Signal picture structure (frame or field) in the encoded stream.
    • --split-enc auto_forced: Automatically decide on using split encoding.
    • --audio-copy, --audio-disposition copy, --audio-metadata copy: Copy audio tracks, their dispositions, and metadata from the source.
    • --chapter-copy: Copy chapter information from the source.
    • --vpp-deband: Apply a debanding filter to reduce banding artifacts.
    • --vpp-nvvfx-denoise: Use NVIDIA VFX denoising.
    • --vpp-nvvfx-artifact-reduction: Use NVIDIA MAXINE VideoEffects SDK's artefact reduction filter to decrease encoding artefacts.

    You can modify any of the settings. These settings you should consider changing.
  • --avsw change to --avhw - Reads using avformat + cuvid hardware decoder. This mode will provide maximum performance since the entire transcoding process will run on the GPU
  • --cqp 20:22:25 - Adjust values to increase or decrease compression: Lower setting equals less compression and larger files, and vice versa.
  • --vpp-resize nvvfx-superres - The strength can be adjusted example: --vpp-resize algo=nvvfx-superres,superres-mode=1 more here
    Code:
     https://github.com/rigaya/NVEnc/blob/master/NVEncC_Options.en.md#--vpp-resize-string-or-param1value1param2value2
  • --vpp-nvvfx-denoise - The strength of the filter can be adjusted --vpp-nvvfx-denoise strength=1 ,1 =Stronger effect, which places a higher emphasis on noise removal. More here
    Code:
     https://github.com/rigaya/NVEnc/blob/master/NVEncC_Options.en.md#--vpp-nvvfx-denoise-param1value1param2value2
  • --vpp-nvvfx-artifact-reduction - Using `mode=0` will result in a less aggressive filter better suited for higher bitrates.
  • You can adjust any setting and see the effects; all settings are located here:
    Code:
     https://github.com/rigaya/NVEnc/blob/master/NVEncC_Options.en.md#
Next, create a file called encode_enhance.bat and copy and paste the details from the text below
Code:
@echo off

REM Set the paths directly without prompting for user input
set "EncoderPath=E:\rigaya\NVEncC\NVEncC64.exe"
set "TxtFilePath=E:\rigaya\enhance.txt"
set "OutputPath=D:\encode"

REM This batch file encodes all MP4 files in the current directory using NVEncC64.exe
REM Subtitles must be named exactly like the video files but with an .srt extension.
REM The output will mirror the folder structure of the input files in the specified output directory.

for %%i in (*.mp4) do (
    REM Encoding each MP4 file with subtitles
    "%EncoderPath%" -i "%%i" --option-file "%TxtFilePath%" --sub-source "%%~dpni.srt" -o "%OutputPath%\%%~pni.mkv"
)

echo.
echo Encoding complete. Press Enter to exit...
pause > nul

Things you need to adjust
Code:
set "EncoderPath=E:\rigaya\NVEncC\NVEncC64.exe" change to the folder where your NVEncC64.exe is located.
set "TxtFilePath=E:\rigaya\enhance.txt" change to the folder where your enhance.txt is located.
set "OutputPath=D:\encode" change to the folder where you want to encode to.

Please note that the setting --sub-source "%%~dpni.srt will look for a subtitle with the ending .srt in the folder of the MP4s with the same name as the MP4.
If your subtitle is ending like .en-us.srt, you need to adjust it to --sub-source "%%~dpni.en-us.srt
You can find additional subtitle settings here:
Code:
 https://github.com/rigaya/NVEnc/blob/master/NVEncC_Options.en.md#--sub-source-stringintparam1value1

You can copy and place the bat file into the folder with the MP4 and execute it to start the encoding process. Ensure that you have enough hard disk space available.


Considerations:
  • Pros: Free, no additional costs, bypasses the need for complex software like Avisynth or Vaporsynth or expensive AI software.
  • Cons: There is no GUI; preview requires encoding a specific frame amount; settings may need adjustments based on the video; and enhancements might not work correctly with cartoons or animation. Compared to Avisynth or Vaporsynth, it has certain limitations.
Remember, this isn't a magical solution that can transform poor-quality footage into high-definition content, but it's a powerful tool for enhancing and upscaling videos with hardware.
 

Encoding Speed Expectations with NVEncC​

When using NVEncC for video encoding, the speed can vary significantly based on several factors, including the complexity of the filters applied, the resolution of the input and output videos, and the capabilities of the hardware, particularly the GPU.

Example Speeds with an RTX 3080:​

  • Upscaling from 720p to 1080p with Noise Reduction: In scenarios where both upscaling and noise reduction filters are applied, encoding speeds around 85 frames per second (fps) have been achieved. This is considering the use of an RTX 3080.

  • Only Noise Reduction on 1080p Video: When the encoding process involves only noise reduction without any upscaling, the speed can increase significantly. With the same settings and hardware (RTX 3080), the speeds been around 140 fps.
These speeds are indicative and can vary based on the specific settings used, the complexity of the video content, and other running processes that might be utilizing GPU resources.

Known Issues​

  • White Lines in Improperly Deinterlaced Videos: One common issue encountered during encoding is the appearance of white lines in videos that have not been properly deinterlaced. This can detract from the overall quality of the output video.

  • Deinterlacing Challenges: Currently I am checking if there is a way to fix bad deinterlaced videos within NVENC instead of Avisynth or Vaporsynth. ALternatively you can use non AI upscaling and noise reduction.

Future Updates and Innovations​

  • AI HDR Calculation for SDR Material: Nvidia has recently showcased advancements in AI HDR calculation, initially aimed at converting Standard Dynamic Range (SDR) material to High Dynamic Range (HDR) in gaming. This technology holds promise for video encoding as well, potentially allowing for automatic and intelligent enhancement of SDR videos to HDR quality.
    While currently limited to gaming applications, there is hope that similar capabilities will be extended to video encoding, offering new possibilities for enhancing video quality through advanced AI algorithms.
 
I've discovered a potential solution that could enhance the quality of bad deinterlaced content.
Although I haven't had the chance to test it in all scenarios, it seems promising and worth considering.

For upscaling these videos, it is recommended to use NVIDIA's Performance Primitives (NPP) libraries.
To use these libraries, you need the following DLL files: nppc64_10.dll, nppif64_10.dll, and nppig64_10.dll.
These files should be placed in the same directory as your NVEncC64.exe executable.
The NPP DLLs are essential for enabling advanced upscaling algorithms and can be downloaded from the following link and filename npp64_10_dll_7zip.7z.
Code:
 https://github.com/rigaya/NVEnc/releases/tag/7.00
You can find more information about NVIDIA's Performance Primitives libraries here
Code:
 https://developer.nvidia.com/npp

In this scenario please comment out the following line: --vpp-resize nvvfx-superres by adding a # in front so it looks like this #--vpp-resize nvvfx-superres and add a new line below it: --vpp-resize lanczos

While the solution has shown potential, its effectiveness may vary depending on the specific characteristics of the material you're working with.
I will be further investigating.
 
NOTE:
I did not include sample settings for the following filters in the initial script, as some of these filters seem now to adjust according to NVENC and video material, but will need to confirm this.

Please find sample settings below.
--vpp-resize nvvfx-superres,superres-mode=0,superres-strength=0.4
--vpp-nvvfx-denoise strength=0
--vpp-nvvfx-artifact-reduction mode=0


You can use this to test any material that contains grain or noise or that you want to upscale.
If you can't think of any, this might give a hint to one
Code:
 ...Its five-year mission: to explore strange new worlds; to seek out new life and new civilizations...
 
Its five-year mission: to explore strange new worlds; to seek out new life and new civilizations
Fascinating. Make it so and Engange! 🖖

(If it doesn't want to be upscaled: Resistance is futile!)

Jokes aside, I immediately thought that I will try it on Voyager. Just haven't found the time yet.
 
Jokes aside, I immediately thought that I will try it on Voyager.
You might need to adjust the filters for that one. Last time I watched it, it looked like something was off with the deinterlace.
 
You might need to adjust the filters for that one. Last time I watched it, it looked like something was off with the deinterlace.
I have five and a half different sources for it (but I will only try two of them right now because I don't want to pull out the other drives). So maybe there are some differences between them. I will try your preset first and then see what happens if I change things.
 
I have five and a half different sources for it (but I will only try two of them right now because I don't want to pull out the other drives). So maybe there are some differences between them. I will try your preset first and then see what happens if I change things.
Ensure you do not use an NTSC source with a 720x576 resolution.
 
It's 768x576. I don't think I even have NTSC for that. The DVDs should be PAL, I only copied them, but never watched them. But I have DS9 NTSC DVDs. (Not that I would use them, I have it from two other sources as well).

What would happen when I use this program on NTSC format? Issues with deinterlancing as you mentioned?
 
It's 768x576. I don't think I even have NTSC for that. The DVDs should be PAL, I only copied them, but never watched them. But I have DS9 NTSC DVDs. (Not that I would use them, I have it from two other sources as well).
Ok. DVD PAL should be appropriately done (hopefully).

What would happen when I use this program on NTSC format? Issues with deinterlancing as you mentioned?
It's not what happens with NTSC. It's about the combination of NTSC and 720x576.
720x576 was never an accurate resolution for NTSC. Looking just now at the AMZ P+ Voyager, I see lousy deinterlacing.
The combination will introduce very hard-to-remove "issues" in the video. NTSC + 720x756 + lousy deinterlacing.
  • Combing: When motion occurs between the fields of a single frame, it creates a comb-like appearance on moving objects.
  • Ghosting: Slight trails or duplication of moving objects caused by field blending instead of correctly reconstructing full frames.
  • Judder: Stuttering and uneven motion, particularly noticeable in panning shots, can occur due to incorrect field merging.
 
DVD PAL should be appropriately done (hopefully).
I can post a comparison screenshot of all versions if you want. And maybe I will also try other normal DVDs. Some things are not available in HD or I just don't have them yet, it might be useful to have them upscaled. But for starters, I will only try the two or three Voyager versions I have on hand.


720x576 was never an accurate resolution for NTSC.
Oh, I always wondered why some videos are available in different resolutions, but one site is the same for both options. Could not find a good answer online and it appears no one really knows anymore. I believe there was a thread at one point, but I would probably have no luck finding it.

Anyway, since I have both PAL downloads and the NTSC DVD of DS9, I will check it out just to see what issues appear. At some point.
 
NOTE:
I did not include sample settings for the following filters in the initial script, as some of these filters seem now to adjust according to NVENC and video material, but will need to confirm this.

Please find sample settings below.
--vpp-resize nvvfx-superres,superres-mode=0,superres-strength=0.4
--vpp-nvvfx-denoise strength=0
--vpp-nvvfx-artifact-reduction mode=0


You can use this to test any material that contains grain or noise or that you want to upscale.
If you can't think of any, this might give a hint to one
Code:
 ...Its five-year mission: to explore strange new worlds; to seek out new life and new civilizations...
AV1 hw encoding is only supported on the RTX40xx series video cards right?
 
I can post a comparison screenshot of all versions if you want. And maybe I will also try other normal DVDs. Some things are not available in HD or I just don't have them yet, it might be useful to have them upscaled. But for starters, I will only try the two or three Voyager versions I have on hand.
Usually MediaInfo will tell you if it's NTSC and what resolution.
Oh, I always wondered why some videos are available in different resolutions, but one site is the same for both options. Could not find a good answer online and it appears no one really knows anymore. I believe there was a thread at one point, but I would probably have no luck finding it.
Yes, I think so too. But the one thing I noticed I have forgotten is that when there is lousy deinterlacing and the resolution is wrongly set, it will be hard to remove, if removable, that "stuff" again.

Anyway, since I have both PAL downloads and the NTSC DVD of DS9, I will check it out just to see what issues appear. At some point.
If you have the NTSC DVD, I would start with that but don't forget to deinterlace. NVENC can do that as well.
--interlace auto and either one of these deinterlacing methods --vpp-afs or --vpp-nnedi or --vpp-yadif
More info here:
Code:
 https://github.com/rigaya/NVEnc/blob/master/NVEncC_Options.en.md#--vpp-afs-param1value1param2value2

AV1 hw encoding is only supported on the RTX40xx series video cards right?
You can find the whole Encode/Decode matrix for Nvidia Cards here:
Code:
 https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new
But yes, only the 40xx series has AV1 encoding for consumer cards.
 
@DeepSpace
If you want a better analysis of the DVD video, I'd recommend dgmpgdec.
https://www.videohelp.com/software/DGMPGDec
Helps tremendously to choose the right deinterlacing filter for DVD content.
 
Note:
Please do not blindly use the script.


Denoising is unnecessary if there is no noise/grain in the video.
Comment --vpp-nvvfx-denoise out via a # like #--vpp-nvvfx-denoise

If you apply denoising to a video without noise or grain, it may lose detail, making it appear excessively smooth or artificial.
This is because the denoising process can mistakenly identify fine details as noise and remove them.

Same for the --vpp-nvvfx-artifact-reduction Only use it when needed; always do a test run!
 
@DeepSpace
Alternatively, for removing noise/grain removal, you might want to try out --vpp-denoise-dct step=1
Since you seem to be a🖖, I suggest testing it on TNG HD or TOS HD version.

Comment out --vpp-nvvfx-denoise
 
--vpp-nvvfx-denoise only works up to 1080p :cry:
And there I thought I could denoise some 4K content :rolleyes:
 
--vpp-nvvfx-denoise only works up to 1080p :cry:
And there I thought I could denoise some 4K content :rolleyes:
You can try the --vpp-denoise-dct step=1

Let me know if you need more settings.
For DCT, there are more settings here: https://github.com/rigaya/NVEnc/blob/master/NVEncC_Options.en.md#--vpp-denoise-dct-param1value1param2value2
 
@cartman0208 and @DeepSpace

Most of the filters can be combined.
Meaning you can use more than one denoise filter for example.
So instead of tuning one filter to the max you can try add another one.

For deinterlacing filters it is not possible to add more than one.

Also filters will be applied in a fixed order and you can't shift them around.
 
Here is an example of how more than one denoise filter can be combined.
--vpp-denoise-dct step=1
--vpp-convolution3d
It is not always necessary to use the --vpp-nvvfx-denoise filter.
 
Back
Top