This article is focused on describing the latest performance benchmarking executed on the latest release of MySQL and Percona. 
In this set of tests I have used the machine described here.
Assumptions
There are many ways to run tests, and we know that results may vary depending on how you play with many factors, like the environment or the MySQL server settings. However, if we compare several versions of the same product on the same platform, it is logical to assume that all the versions will have the same “chance” to behave well or badly unless we change the MySQL server settings.
Because of this, I ran the tests changing only things in a consistent way, with the intent to give the same opportunity to each solution., with the clear assumption that if you release your product based on the defaults, that implies you had tested with them and consider them the safest for generic use.
I also applied some modifications and ran the tests again to see how optimization would impact performance.
What tests do we run?
High level, I run one set of test:
- TPC-C (https://www.tpc.org/tpcc/) like
The full methodology and test details can be found here, while actual commands are available:
Why do I only run TPC-C tests?
Well I am, normally, more interested in testing scenarios that are closer to reality than a single function test as we normally do with sysbench.
This is it, while it is not possible to get the perfect benchmark test fitting all real usage, we need to keep in mind the rule of 80%.
If you use MYSQL/InnoDB I expect that you have an OLTP kind of traffic, more than Key/Value or else. Given that while testing the single function, as we do with sysbench, may be useful to identify regression points or so. To get the wide scenario, TPC-C is a better way to go, given it implies not only a more intense write load, TPC-C test is 50/50 r/w, but also a schema structure with relations, foreign keys and constraints. In short, it is closer to the common use of a relational database management system.
Results
The tests done have two different kinds of isolation levels. Repeatable Read and Read Committed. The first is the default in MySQL/InnoDB, while the second is the default in many other very well known RDBMS.
As usual an image is more descriptive than many words:
You can also compare these trends (not the values) with the previous results published here.
Let us comment a bit on these images.
The first comment we should make is that nowadays our systems must be ready to scale. Period, no discussion, also doing benchmarks up to 1024 threads is not enough. In reality we have 4000 or even more connections, given that doing benchmarking exercises and stopping the load at 128 threads or lower, makes no sense. Here it is quite clear that doing something like that could be very misleading.
Old MySQL versions are still faster than newer with low level of concurrency, but they do not scale. So if I stop my tests at 64 or 128 threads I will conclude that MySQL 5.7 is always better than newer versions, while if I go on loading, I can see that old version performance drop after 64 concurrent threads.
The second comment is that while in previous tests (see mentioned article) we saw that newer versions were not performing better or even consistently. With the latest releases MySQL had not only stabilized the server behaviour, but done significant fixes to the performance issues it had.
If we remove the 5.7 version from the graphs we can see clearer what is going on:
If you notice the servers lines after 32 threads and especially after 64 threads diverge and we have two groups one see Percona and MySQL 8.0.40 in the lower set while Percona and MySQL 8.4 and 9.x are in the upper group.
This is quite great news and very nice to see, because if I have to read it in a way I see a positive sign indicating how Oracle/MySQL is progressively resolving some performance issues and gaining ground again.
Conclusions
The tests performed using TPC-C like tests confirms the initial finding of my colleague here, and give us a picture that is more positive of what we had. At the same time they indicate that the race to get better performance is open again, and I am intrigued to see what will come next.
For now we can say that MySQL 9.2 is the better performing MySQL version currently available, for its stability and scalability.
Great job!