19:06Machine-translated from Azerbaijani by AIRead in Azerbaijani

Lawyer Builds AI Server With 12 Used Nvidia Cards for AZN 16,000

Lawyer Nick Mermiges has installed 12 used Nvidia V100 graphics cards in his personal artificial intelligence (AI) server, replacing the previous 8 cards. The system cost $9,450 (approximately AZN 16,070). All expert modules of the DeepSeek V4.1 Flash model were stored in memory in the more compact 4-bit NVFP4 format. During testing, the code generation speed reached 157–184 tokens, or text units, per second. According to Mermiges, the server outperformed systems assembled from 4 DGX Spark devices when processing a single query. Moreover, it costs half as much as 3 such devices.Hüquqşünas 12 işlənmiş Nvidia kartlı AI serverini 16 min manata yığıb

The previous 8 cards had a total of 254 GiB of memory. Storing the model’s expert components in NVFP4 format required 278.9 GiB. Therefore, all 15,360 expert modules were stored in a more highly compressed 2-bit format. The 5,949 most frequently used modules were then converted to the 4-bit format. After the number of cards was increased, all the modules fit into memory in NVFP4 format without additional compression. Expert modules are small subsections that the model selects and uses based on the query.

V100 cards do not have dedicated tensor cores that accelerate FP4 and FP8 calculations. Therefore, data in NVFP4 format is unpacked by software and converted into a half-precision format that the cards can process. Nevertheless, because less data is read in the new format, it can be used even on hardware from 2018. The system consists of 3 boards, each with 4 cards. The cards are connected to one another via NVLink, while the boards are connected via PCIe. The PCIe hop adds latency.

After the upgrade, the power limit for each card was raised from 150 W to 175 W. The clock speed was also reduced from 1,522 MHz to 1,452 MHz. Generating the first code segment for a 256K text previously took 108.4 seconds. That time has now fallen to 58 seconds. Beyond the 32K threshold, the reading speed increased from approximately 2,200 tokens per second to 4,500 tokens per second. A 128K file was read in 28 seconds. Code generation speed increased by 23% on a 1K query. With larger queries, particularly beyond 32K, the main advantage is that the first response arrives sooner.

When the power limit for each card was raised to 200 W, the system experienced stability problems. The host system stopped once while running 4 long chats simultaneously. When 3 separate 128K queries were processed in parallel, the aggregate speed in coding tasks was 360 tokens per second. For legal texts, this figure reached 213 tokens. Mermiges plans to process several queries together and store chat data on disk in the future. He also emphasized that the speed figures do not prove the quality of the responses. Nevertheless, he is already using this server as his main DeepSeek computer.