InfluxDB 3.0 vs 1.8: A Surprising Decline in Ingestion Speed

#database #opensource

Highlights

TSBS testing demonstrates that for datasets with fewer than 1 million devices, InfluxDB OSS 3.0 actually ingests data more slowly than InfluxDB OSS 1.8.

For datasets of 1 million or more devices, InfluxDB OSS 3.0’s data ingestion performance is faster than InfluxDB OSS 1.8.

InfluxDB OSS 3.0’s data ingestion interfaces are not consistent with InfluxDB 1.8, meaning that users cannot migrate smoothly.

When InfluxDB 3.0 was released to enterprise customers back in 2023, it was claimed to be a major performance improvement over previous versions. At the time, InfluxData published a benchmark report comparing the new enterprise version with InfluxDB OSS 1.8, which showed great results across the board for InfluxDB 3.0.

We wanted to try it for ourselves, but we’re not enterprise customers — so like the rest of you, we had to wait for a year and a half till InfluxDB OSS 3.0 finally came out this January. And although the current release is admittedly just a “public alpha,” we still had high hopes for its performance. Considering that InfluxData rearchitected their entire product for the second time, causing great frustration to existing users hoping for a smooth upgrade process, and even deprecating their Flux language, they must have been able to produce amazing results. Otherwise, it would hardly be worth upgrading, right?

Putting InfluxDB to the Test

To prove how much better InfluxDB 3.0 performs, we decided to compare it against InfluxDB 1.8 using the Time Series Benchmark Suite (TSBS) originally developed by InfluxData and now maintained, to some extent, by Timescale. Because InfluxDB 3.0 supports InfluxQL and InfluxDB’s line protocol, in theory the test suite for 1.8 should be able to run on the new public alpha. In fact, we did run into several compatibility issues during the testing process and had to find workarounds; this is discussed in the Methodology section below.

We wanted to use TSBS because it would give more comprehensive results than the somewhat simplistic benchmarks run by QuestDB and also be a fair comparison on an open and well-established framework. The results, however, were surprising.

TSBS includes two use cases, a DevOps scenario involving CPU monitoring, and an IoT scenario involving vehicle tracking. For this comparison, we used TSBS to generate the data for both use cases and ingest it into InfluxDB 3 0.1.0 revision v2.5.0-14345, InfluxDB OSS 1.8, and TDengine OSS 3.3.5.8. The following tables show the data ingestion performance of each system in metrics per second.

TSBS DevOps Use Case

TSBS IoT Use Case

In the largest-scale datasets, ingestion performance increased by 5.4x in DevOps and 4.9x in IoT — a far cry from the “45x better write throughput” that InfluxData’s benchmark report claims for InfluxDB 3.0 Enterprise. More alarmingly, in the three datasets with 100,000 devices or fewer, InfluxDB 1.8 actually outperformed InfluxDB 3.0. We can see that for data ingestion, the claimed performance improvements in InfluxDB 3.0 are either limited to enterprise customers or do not hold up against independent testing.

InfluxData’s claims from their website

Although InfluxDB 3.0 has been rebuilt from the ground up, its ingestion performance still cannot match a modern time-series database like TDengine. The test results show that TDengine‘s ingestion was between 4.4x and 11.3x faster than InfluxDB 3.0, and between 3.1x and 22.8x faster than InfluxDB 1.8.

Methodology

All testing was performed on a 40-core machine with 256 GB of RAM. While this is slightly smaller than the machine used in InfluxDB’s benchmark report and larger than the machine used by QuestDB, we do not believe that upgrading or downgrading hardware would affect the overall trends in performance.

Because TSBS has not been updated to support InfluxDB 3.0, we were forced to make certain modifications to enable testing. In the interest of fairness, we tried to limit our modifications as much as possible, but the following issues needed to be addressed:

1.The commands to show, create, and delete databases used in TSBS for InfluxDB 1.8 were not operational in InfluxDB 3.0. Instead, we used the InfluxDB v3 API as follows:

GET /api/v3/configure/database to show databases
POST /api/v3/configure/database to create databases
DELETE /api/v3/configure/database to delete databases

2.With more than one worker, InfluxDB 3.0 would experience failures to write data and throw the error Invalid write response (status 409): catalog update error: table already exists. To avoid this, we modified TSBS to retry writes that failed due to this error, instead of exiting. We used the /api/v3/write_lp interface to write data to InfluxDB 3.0.

You can review all modifications on our fork of TSBS and also run the test suite yourself using our scripts.

Conclusion

While the newly redesigned InfluxDB 3.0 promises much, when it comes to ingestion, at least for open-source users, it fails to deliver. Comprehensive testing demonstrates that its ingestion performance is only somewhat superior to InfluxDB 1.8 for the largest datasets that most users will never encounter, and even falls behind for sites that include fewer than a million devices. Although the currently released version is a public alpha not ready for production, after more than a year, we have to ask: is this the best that InfluxData can offer to the open-source community?

With the difficulty of upgrading existing deployments to InfluxDB 3.0, open-source users need to decide whether it’s worth the trouble. If ingestion performance has been a bottleneck for you, it’s clear that, at least for now, upgrading to 3.0 is not a solution. Instead, consider a time-series database like TDengine that respects its community by consistently releasing high-performance, full-featured software that is available to everyone.