RTX 20, 30 and 40 Series GPUs Can Nearly Double Tensor Performance in AI Workloads With a Firmware Change

It turns out that several generations of GeForce RTX graphics cards have significantly more potential than NVIDIA makes available by default for certain AI workloads. After RTX 40 series cards, similar unlocks have now been achieved on older RTX 20 and RTX 30 cards, with performance increases of 50% to nearly 100% in some operations.

This is not traditional overclocking or a way to speed up games. The changes mainly affect FP16, BF16, TF32 and FP8 Tensor operations used for matrix calculations and AI. The projects use low-level changes to the GPU configuration before the standard NVIDIA driver loads. There is no need to flash a modified BIOS or use a modified driver.

RTX 20 Series

The unlock has also been achieved on the Turing architecture. The RTX 2080 Ti showed significant differences in FP16 calculations with FP32 accumulation:

• 59.9 → 111.0 TFLOPS, or +85.3%
• In the broader MAMF test: 57.71 → 88.24 TFLOPS, or +52.9%
• FP16 → FP16 remained practically unchanged: 119.5 → 118.5 TFLOPS

This also shows that the unlock does not automatically speed up all Tensor operations. It depends on the specific type of calculation.

RTX 30 Series

The effect is even more interesting on Ampere cards. The GPU unlock 1.0.0 project aims to increase the rate at which FP16/BF16 instructions are issued for operations with FP32 accumulation.

Results include:

• RTX 3090: 73.03 → 122.76 TFLOPS in FP16 → FP32, or +68.1%
• RTX 3090 Ti FP16: 87.46 → 118.39 TFLOPS, approximately +35%
• RTX 3090 Ti BF16: 87.50 → 157.41 TFLOPS, approximately +80%
• RTX 3090 Ti TF32: 42.58 → 82.61 TFLOPS, approximately +94%

According to the authors, the mechanism doubles the rate at which certain FP16/BF16 FMA instructions are issued on supported GA102, GA103, GA104 and GA106 chips. However, this does not automatically mean twice the performance in an entire application.

RTX 40 Series

A similar situation has emerged with the Ada Lovelace architecture. Tests show significant headroom, particularly in FP16, BF16 and FP8 Tensor operations:

• RTX 4090: approximately 165 → 331 TFLOPS in FP16/BF16, or nearly double the performance
• RTX 4080 SUPER: FP16 performance increased by approximately 83%, while FP8 came close to doubling in some tests
• RTX 4070 Ti SUPER: FP16/BF16 performance nearly doubled, while FP8 matrix calculations increased by approximately 95%
• RTX 4060 Ti: initial tests indicate potential performance gains of approximately 40%

Even in this case, the graphics card’s performance does not universally double. The actual speedup depends on which Tensor operations a particular AI application uses.

Not a Simple Free Performance Boost

The current solutions are experimental and are mostly used under Linux. They require changes to the system startup process, disabling Secure Boot and, in some cases, a full cold start of the graphics card.

The authors also warn that these are low-level GPU experiments. An incorrect procedure can cause system crashes, startup problems or, in extreme cases, hardware damage. This is definitely not a modification intended for everyday users.

Still, the results are interesting. They show that at least three generations of GeForce RTX cards—Turing, Ampere and Ada Lovelace—have headroom in certain Tensor operations that the standard configuration does not use.

This does not practically change performance for gamers, but the difference could be significant for local AI models, image generation, calculations and other Tensor workloads. In particular, some BF16, FP16, TF32 and FP8 operations see performance gains of tens of percent, and in extreme cases come close to doubling.

You can find the unlocks on GitHub:

• GPU Unlock Ada - RTX 40
• GitHub - Zagitta - RTX 30
• GitHub - Turing - RTX 20

Comments

Name:

:)