Real World Apple Silicon Benchmarks Part 7: Renaissance Java Benchmark Suite

For the last benchmark I am going to explore the performance of Java assembeled for the purposes of benchmarking Java-based compute systems, called Renaissance . There has been a change since I started this (see this previous post ) though. Azul , a company that specializes in Java and JVM infrastructure, has released a version of OpenJDK that is compiled for Apple Silicon. I have therefore run the benchmarks both using AdoptJDK Intel installation running under Rosetta as well as the Apple Silicon Native M1 one by Azul. Let’s see how Orekit runs in these three environments. The full project and results is documented here .

The Renaissance benchmark system is itself a composite benchmark assembled from other existing open sourced benchmark systems. It’s designed to stress test the JVM for workloads like big data processing, machine learning, and functional programming. The principle person behind it is Alexander Prokopec who works for Oracle Labs on Graal Virtual Machine project. These sorts of tests can help them in the design of better compilers and runtimes, which is pretty central to the purpose of the Graal project. Using this suite we were able to really stress test the JVM both running the Intel build of OpenJDK from AdoptJDK under Rosetta and the native build by Azul Systems running natively on M1. The suite itself is comprised of 24 inidividual benchmarks. We attempted to run each of them an all of the platforms.

We did run into a problem with one suite running natively, the “db-shootout” suite. This test calls into the libraries by the Java Native Runtime Project. Those libraries are specifically wrapping system libraries. Because there is no Apple Silicon version of this dependency this test fails running natively. All the other tests however are fully JVM code therefore run perfectly. We’ll explore the Rosetta v. Intel results for db-shootout after looking at the rest of the test suite. The full results (besides db-shootout) completely with tabular output can be found here .

Looking at the total sum execution time of every single test in the benchmark we can see that compared to the Intel MBP the JVM under Rosetta was slightly slower, about 14 percentage points, but the native JVM was 36 percentage points faster. The former result is acceptable but the latter result is wonderful. Watching the tests execute I’m not 100% surprised by this either. When running in Rosetta it didn’t seem that the JVM was ever using more than 50% of the total CPU of the laptop in the highly parallelize tests. When running natively however it used up almost all of the CPUs. I know that there is a disparity between the 4 high power and 4 low power CPU cores. I’m not sure what exactly “low power means” or what they can or can’t do. The fact the JVM running natively could exercise them though is promising.

If you look at the fine grained details of each test you’ll find a mixed bag for Rosetta-based JVM performance. Some were slightly faster, others were slightly slower. The only one which truly stands out as a very poor performing benchmark under Rosetta was the “movie lens” test. This one was a whopping 4x slower than on the Intel MBP. For the native JVM the performance was almost uniformily faster and noticeably so, consistent with the overall benchmark results. The one test which the Intel MBP edged out the native M1 running JVM was the “gauss-mix” test, where the M1 MBP was 16% slower.

Doubling back to the “db-shootout” benchmark to look at how Rosetta matched up to Intel Native MBP, this happened to be one benchmark that the M1 MBP beat out the Intel MBP even with the Rosetta penalty. It clocked in 17% faster on that test suite.

Overall the result looks very promising for the native JVM again, with a substantial speed improvement over the Intel MBP. Again, with the availability of the Azul OpenJDK and the performance numbers here, I’d say it makes for a solid Java runtime platform and the fastest one you can find in the Apple ecosystem.