16:25Machine-translated from Azerbaijani by AIRead in Azerbaijani
Experts Bypass AI Limit on Nvidia RTX 20, 30 and 40 Graphics Cards
Mod developers have bypassed the restriction on artificial intelligence computations in Nvidia’s RTX 20, RTX 30 and RTX 40 series graphics cards. A software vulnerability allows the tensor cores—the components that perform artificial intelligence computations—to operate at full speed. As a result, queries can be processed more quickly and images can be generated faster. However, gaming performance is not expected to increase by 2 times.
3 separate projects have been developed for different generations. gpu-unlock works on Ampere-based RTX 30 models. The gpu-unlock-ada project covers the RTX 4090 and some other Ada graphics cards. turing-unlock works on the RTX 2080 Ti. Support for the RTX 4090 D, RTX 4070 Ti Super and the RTX 4060 Ti with 16 GB of memory is still being tested. The RTX 50 series is not supported.
The report says that “theoretical FP16 computing power on the RTX 4090 increased by 2 times.” The figure rose from 165.6 TFLOPS to 331.6 TFLOPS. However, the results were lower in real-world artificial intelligence computations. Performance increased by 12–34% in FP16 format and by no more than 20% in BF16 format.
In testing of the RTX 4070 Ti, the increase in FP16 performance reached 80.1%. The result for BF16 was 76.4%. The RTX 3090 Ti delivered 35% higher performance in FP16 computations and 80% higher performance in BF16. The increase in TF32 computations reached 94%.
BF16 computations on other RTX 30 models became 25–81% faster. The speed at which a large language model processes initial data increased by 14–36%. Image generation and FlashAttention operations also became faster. However, the speed at which chatbots generate word fragments, or tokens, while preparing responses remained almost unchanged. This is because the process mainly depends on the video memory’s data transfer rate and the graphics card’s power limit.
The method exploits a memory overflow flaw in Nvidia’s signed SEC2 boot program. In other words, it takes advantage of a vulnerability in how the software handles memory. Special code is run in protected mode to change 2 parameters responsible for performance. The device is then restarted before the graphics card’s main driver loads, removing the restriction. This method is based on a vulnerability first discovered in the CMP 170HX graphics card.
The projects have unofficial modules for Linux. Windows versions are still being tested. The developers warn that this modification can cause the graphics card and computer to freeze. The system may fail to boot, data may become corrupted and even the hardware may fail. Many Turing and Ada models have not yet been tested. No specific plan has been announced to increase the number of supported models.