GSFG 1.1:实时游戏插帧GSFG 1.1: Real-Time Game Frame Generation

推出 GSFG 1.1,GameSir Frame Gen 的新一代插帧模型。相比 GSFG 1.0 它有明显提升,在 22 个测试片段上的 PSNR 全部超过 LSFG 2.0,在手机 GPU 上生成一帧只需约 1.39 ms。

Introducing GSFG 1.1, the new frame-interpolation model in GameSir Frame Gen. It's a clear upgrade over GSFG 1.0, beats LSFG 2.0 in PSNR on all 22 of our test clips, and generates a frame in about 1.39 ms on a phone GPU.

游戏画面里最难插的是快速转动的镜头和大幅移动的角色,而准星、血条又必须稳稳留在原位。GSFG 1.1 会把这些运动分开处理,再合成出两张原始帧之间任意时刻的画面。

The hardest frames to interpolate in games come from fast camera turns and large character motion, while crosshairs and health bars have to stay exactly where they are. GSFG 1.1 handles these motions separately, then combines them into a frame at any moment between two rendered ones.

测试覆盖 9 款游戏。GSFG 1.1 在每个片段上的平均 PSNR 都高于 LSFG 2.0 和 GSFG 1.0;逐帧比较,与 LSFG 2.0 的 PSNR 差距至少为 1 dB 的帧里,92% 是 GSFG 1.1 更高。在 Adreno 840 GPU 上,720p 输入、运动估计取 60% 分辨率,生成一帧的 GPU 耗时约 1.39 ms(含读入和写出),不到 120 fps 下每帧预算的五分之一。

In our testing across 9 games, GSFG 1.1 has a higher average PSNR than both LSFG 2.0 and GSFG 1.0 on every clip. Frame by frame, where the PSNR gap versus LSFG 2.0 is at least 1 dB, GSFG 1.1 scores higher in 92% of frames. On an Adreno 840 GPU, with 720p input and motion estimated at 60% resolution, a frame takes about 1.39 ms of GPU time including reads and writes, under a fifth of the per-frame budget at 120 fps.

画质基准Image quality benchmark

逐片段 PSNR · GSFG 1.1 vs LSFG 2.0Per-clip PSNR · GSFG 1.1 vs LSFG 2.0

逐帧 PSNR 差 · GSFG 1.1 − LSFG 2.0Per-frame PSNR difference · GSFG 1.1 − LSFG 2.0

左图比较各片段的平均 PSNR,虚线上方的点表示 GSFG 1.1 更高。右图展示逐帧差值:向右是 GSFG 1.1 更高,向左是 LSFG 2.0 更高。在 PSNR 相差至少 1 dB 的帧中, 是 GSFG 1.1 更高。

The left chart compares each clip's average PSNR; points above the dashed line are clips where GSFG 1.1 scores higher. The right chart shows per-frame differences: to the right GSFG 1.1 is higher, to the left LSFG 2.0 is higher. Among frames whose PSNR differs by at least 1 dB, GSFG 1.1 is higher in .

模型怎样处理运动How the model handles motion

游戏里经常同时出现几种运动:镜头带着背景移动,角色朝另一个方向走,血条和准星则留在屏幕上的固定位置。GSFG 1.1 会分别处理这些情况。

Games often mix several kinds of motion at once: the camera drags the background along, a character walks in another direction, and health bars and crosshairs stay fixed on screen. GSFG 1.1 handles each of these separately.

模型会额外读取一张历史帧,用连续三帧识别保持静止的界面区域,并约束这些区域的运动估计。上一组输入估计出的运动也会保留下来,供下一组输入使用。

The model also reads a history frame and uses three consecutive frames to find UI regions that stay still, constraining the motion estimate there. The motion estimated for the previous pair of inputs is kept and reused for the next pair.

主运动估计从低分辨率开始,逐层细化;另一路从零开始估计小幅运动,处理那些没有跟着背景一起移动的物体。融合网络为每个像素分配两路候选的权重。合成时,再利用初步生成的画面引导运动场上采样,减少边缘处的错位。

The main motion estimate starts at low resolution and is refined level by level; a second branch estimates small motion from zero, for objects that do not move with the background. A fusion network weights the two candidates for every pixel. During synthesis, a first-pass image guides the upsampling of the motion field to reduce misalignment at edges.

历史帧用于识别静止界面;上一组输入的运动估计供当前计算参考。主运动和小幅运动两路结果共同参与最终合成。

The history frame is used to detect static UI; the previous pair's motion estimate serves as a starting point. The main-motion and small-motion branches both feed the final synthesis.

任意时刻插帧Interpolation at any moment

GSFG 1.1 可以生成两张原始帧之间任意时刻的画面:时刻 t 是模型的连续输入,而不是几个固定档位。2×、3×、6× 等整数倍,以及 40→60 fps(1.5×)、24→60 fps(2.5×)这类非整数倍,都只是在不同位置取 t,用的是同一个模型。

GSFG 1.1 can generate the frame at any moment between two rendered frames: the time t is a continuous input to the model, not a handful of fixed steps. Integer multiples such as 2×, 3× and 6×, and non-integer ones such as 40→60 fps (1.5×) or 24→60 fps (2.5×), simply sample t at different positions with the same model.

拖动滑块,查看 t 从 0 到 1 的画面变化:两端是原始帧,中间 59 个时刻全部由模型生成。在 60 fps 录像上测试时,训练中没有使用过的时刻(如 1/5、1/6)与常用时刻表现一致。

Drag the slider to watch the frame change as t goes from 0 to 1: both ends are rendered frames, and all 59 moments in between are generated by the model. In tests on 60 fps footage, moments never used in training (such as 1/5 and 1/6) performed as well as the common ones.

前一帧Previous frame后一帧Next frame

画面对比Side-by-side comparison

下面是 4 个画面中 LSFG 2.0 与 GSFG 1.1 的对比。上方是同一区域的放大图,依次为 LSFG 2.0、GSFG 1.0、GSFG 1.1 和真实帧。下方可拖动分隔线比较整帧,也可切换到误差图;颜色越亮,与真实帧的差异越大。

Four frames comparing LSFG 2.0 and GSFG 1.1. The top row zooms into the same region: LSFG 2.0, GSFG 1.0, GSFG 1.1 and the real frame. Below, drag the divider to compare full frames, or switch to the error map, where brighter means further from the real frame.

与真实帧一致Matches real frame误差大Large error
LSFG 2.0GSFG 1.1

慢放对比Slow-motion comparison

下面只播放模型生成的中间帧,速度为 8 fps。可以观察同一物体在连续帧中的轮廓变化,以及重影是否反复出现。左侧为 LSFG 2.0,右侧为 GSFG 1.1。

These videos play only the generated intermediate frames, at 8 fps, so you can follow how an object's outline changes across consecutive frames and whether ghosting keeps reappearing. LSFG 2.0 is on the left, GSFG 1.1 on the right.

LSFG 2.0GSFG 1.1
杀手 3Hitman 3

LSFG 2.0GSFG 1.1
无畏契约Valorant

LSFG 2.0GSFG 1.1
原神Genshin Impact

在手机上运行Running on a phone

GSFG 1.1 使用 Vulkan 计算着色器在手机 GPU 上运行,网络权重以半精度编入着色器。最新基准中,Adreno 840 GPU 在 720p、运动估计宽高均设为原画面的 60% 时,生成一张中间帧的 GPU 耗时约为 1.39 ms,包含新帧读入和结果写出。

GSFG 1.1 runs on the phone GPU as Vulkan compute shaders, with the network weights baked into the shaders in half precision. In the latest benchmark, with 720p input and motion estimated at 60% of the width and height, an Adreno 840 GPU generates one intermediate frame in about 1.39 ms of GPU time, including reading the new frame and writing the result.

降低运动估计分辨率可以减少计算量,最终输出仍保持原分辨率。多倍插帧会生成更多中间帧,界面识别等可复用的计算只需做一次。在相同设置下,3× 插帧生成两张中间帧约需 2.04 ms,4× 生成三张约需 2.71 ms,6× 生成五张约需 4.05 ms。图中展示的是每组输入的总 GPU 耗时。

Lowering the motion-estimation resolution reduces computation while the output stays at full resolution. Multi-frame interpolation generates more intermediate frames, and reusable work such as UI detection runs only once. With the same settings, 3× (two frames) takes about 2.04 ms, 4× (three frames) about 2.71 ms, and 6× (five frames) about 4.05 ms. The chart shows the total GPU time per input pair.

每组输入的 GPU 耗时 · Adreno 840,720p,60% 运动估计分辨率GPU time per input pair · Adreno 840, 720p, 60% motion-estimation resolution

  1. 画质测试:9 款游戏、22 个片段、3,256 张生成帧。将 30 fps 录像隔帧抽为 15 fps,插帧后与原帧比较。Image quality: 9 games, 22 clips, 3,256 generated frames. 30 fps recordings were decimated to 15 fps, interpolated, and compared with the original frames.
  2. 图中相对误差以 LSFG 2.0 为 100,由平均 PSNR 差换算为 MSE 比值;数值越低越好,各图使用独立的截断刻度。Relative error in the charts sets LSFG 2.0 to 100, converted from the average PSNR difference into an MSE ratio; lower is better, and each chart uses its own truncated scale.
  3. 性能测试:Adreno 840,720p,60% 运动估计分辨率;取多轮 GPU 耗时中位数,不含游戏渲染和显示等待。Performance: Adreno 840, 720p, 60% motion-estimation resolution; median GPU time over multiple runs, excluding game rendering and display waits.

如何使用:保持盖世游戏APP应用联网,启动游戏后,应用会自动完成更新并启用 GSFG 1.1。

How to use: Keep the GameHub app connected to the internet. Launch a game, and the app will automatically update and enable GSFG 1.1.