1 Reply by cws

(2,252 replies, posted in Using SVP)

pensioner600 wrote:

I also noticed that ALL versions of V2 give the entire picture a shake, which is very clearly visible on a static scene on small light objects on a dark background. I noticed a long time ago that something was wrong, but I didn’t have time to watch movies. Versions NOT V2 and lite no jitter. I’m definitely switching to 15lite or 16lite, even if it’s x3, not x4. V2 is complete nonsense. Today I devoted half a day to tests, a lot became clear))

I think this has been fixed in newer TensorRT 9.x testing releases (note the current release that is part of SVP is 8.x.) If you can still see it with 9.x, it should be reported so developers are aware and hopefully it can be addressed.

Go to forum: Using SVP

2 Reply by cws

(2,252 replies, posted in Using SVP)

It's true there's no significant performance improvement, but there may be fixes and other minor improvements.

Specifically, when using v13 (with TensorRT 8.5 / CUDA 11.8) with the v2 models, I can see some blurriness and pixels shifting in the bottom right corner if there is some obvious static content there like a logo. If this is something you can reproduce, try using the latest v14 (with TensorRT 9.x / CUDA 12.x)

Go to forum: Using SVP

3 Reply by cws

(2,252 replies, posted in Using SVP)

Right, TensorRT 9.x is in "beta" and not officially released. Still, v13 was released when TensorRT 8.5 was the latest version, when there is a new release of TensorRT 8.x (which is 8.6.)

There may be fixes and enhancements in the latest v14 releases which is not in v13. If you're willing to help test, I would try downloading https://github.com/AmusementClub/vs-mlr … 4.test2.7z and unzipping 

vstrt.dll

and the folder 

vsmlrt-cuda

into 

C:\Program Files (x86)\SVP 4\rife

Go to forum: Using SVP

4 Reply by cws

(2,252 replies, posted in Using SVP)

Does it still happen when using the latest version from https://github.com/AmusementClub/vs-mlrt/releases? SVP might be using the last non-test release (v13) instead of the latest testing release (v14).

Go to forum: Using SVP

5 Reply by cws

(2,252 replies, posted in Using SVP)

Are you sure you followed all the steps?

You should not be replacing or renaming any files in `C:\Program Files (x86)\SVP 4\rife\models\rife`.

You should put the v2 models in the new directory `C:\Program Files (x86)\SVP 4\rife\models\rife_v2`.

After replacing `vsmlrt.py` with the latest you downloaded from GitHub, then modifying the engine path, then modifying it to use `_implementation=2`, you should set the "rife_trt_model" property in SVP to specify the file you want to load (like setting it to "49") instead of renaming files.

Go to forum: Using SVP

6 Reply by cws

(2,252 replies, posted in Using SVP)

You can update `vsmlrt.py` by getting a copy from the `AmusementClub/vs-mlrt` repo, then modifying:

    alter_engine_path = os.path.join(
        tempfile.gettempdir(),
        os.path.splitdrive(engine_path)[1][1:]
    )

into

    alter_engine_path = os.path.join(
        os.path.expandvars("%APPDATA%\\SVP4\\cache\\"),
        os.path.splitdrive(engine_path)[1][1:]
    )

Then you can force the v2 implementation by either:

1) modifying `def RIFE()` in `vsmlrt.py` to make the implementation argument default to 2, or

2) modifying line 20 of `C:\Program Files (x86)\SVP 4\script\base.py` to pass in an additional implementation argument: `smooth = RIFE(input_m,multi=rife_num,model=rife_mnum,backend=trt_backend,_implementation=2)`

Note I have downloaded the v2 models to the folder `C:\Program Files (x86)\SVP 4\rife\models\rife_v2`. You should also set `rife_trt_model` to 49 in SVP's Advanced settings in the "Video profiles options" section that corresponds to the profile you're using, so that it uses rife_v4.9.onnx (without file renaming shenanigans.)

Go to forum: Using SVP

7 Reply by cws

(2,252 replies, posted in Using SVP)

I think in `def RIFE()` of `vsmlrt.py`, you can set `_implementation` to 2, and it'll look for the model in the `C:\Program Files (x86)\SVP 4\rife\models\rife_v2` directory.

Go to forum: Using SVP

8 Reply by cws

(2,252 replies, posted in Using SVP)

If anyone else is interested in doing some experimenting and/or testing, I've been trying to use the v2 implementation of rife v4.6. This seems to noticeably improve seek times for me on a 4090 with 4K videos...

According to this the aliasing issue appears to be precision-related (fp16 vs fp32) so I tried using the debug tool in TensorRT to narrow it down to the first 90 layers. Maybe someone who is more knowledgeable can debug further, but I tried creating an .engine file (which you can replace in your cache folder) that uses higher precision for the first 90 layers by running:

%LOCALAPPDATA%\Programs\Python\Python39\Scripts\polygraphy.exe convert --convert-to trt --fp16 --trt-min-shapes input:[1,7,2144,3840] --tensor-dtypes input:float16 output:float16  --precision-constraints obey --trt-npps process.py -o path\to\save\your\rife.engine "C:\Program Files (x86)\SVP 4\rife\models\rife_v2\rife_v4.6.onnx"

Note you'll need to install Python support in the Windows TensorRT SDK, then install the polygraphy package in Python, then create process.py which is based on the debug tool:

import tensorrt as trt

def postprocess(network):
    N = 90
    EXCLUDE_LAYER_NAMES = ["CONSTANT"]
    EXCLUDE_LAYERS = [getattr(trt.LayerType, attr) for attr in EXCLUDE_LAYER_NAMES if hasattr(trt.LayerType, attr)]

    for index in range(0, N):
        layer = network.get_layer(index)
        def should_exclude():
            has_non_execution_output = any(
                not layer.get_output(i).is_execution_tensor for i in range(layer.num_outputs)
            )
            has_non_activation_output = any(
                layer.get_output(i).dtype not in [trt.float32, trt.float16, trt.int8]
                for i in range(layer.num_outputs)
            )
            return layer.type in EXCLUDE_LAYERS or has_non_execution_output or has_non_activation_output

        if not should_exclude():
            layer.precision = trt.float32

Go to forum: Using SVP