Measuring performances of an emulator is always a bit tricky. For box86 (and box64), I decided to use benchs of opensource softwares, and compare the results between native and i386 version. A few months ago, I did some benchs and wrote the results for the Fosdem. The article can still be found here. The result was that CPU intensive tasks were at around 50% of the native speed, and graphics intensive apps were close to 100% speed, because most work is done in the libs, so in native space because of the wrapping of the libs (the “twist of box86”). The conclusion was that a game should probably be somewhere in between.
Let’s answer that question: how does that translate to actual games. There are not a lot of AAA games that are opensource. Not actual ones at least, so the testing is a bit difficult. But in the past, some games went opensource. A major one is Quake3. It’s a highly optimized engine, that also contain a virtual machine with a JIT, and this game has an integrated benchs! So, using the fully opensource OpenArena, I can do some benchs on different architectures and see how fast box86 and box64 can be.
For that, I took the official OpenArena 0.8.8 build for i386 and x86_64, and I added an armhf and aarch64 build from the Mageia distribution. Then I take the bench setup from Phoronix, wrote a small script and I’m ready to bench!
I chose to bench using a fullscreen 1024×768 resolution, to avoid being bottlenecked too much by the GPU. The Phoronix script uses high detail, putting some pressure on the RK3399 board I’m using for this test. So I have 4 architectures: ARM64 (or aarch64), ARM32 (or armhf), i386 and x86_64.
Here are the rough results:
ARM64 version: 3398 frames 108.5 seconds 31.3 fps 16.0/31.9/68.0/6.7 ms ARMHF version: 3398 frames 106.4 seconds 31.9 fps 15.0/31.3/67.0/6.6 ms i386 version: 3398 frames 130.2 seconds 26.1 fps 17.0/38.3/157.0/11.0 ms x86_64 version: 3398 frames 128.2 seconds 26.5 fps 16.0/37.7/168.0/9.9 ms
The first thing that comes to mind is: the ARM64 version is NOT faster than the ARM32 one! Despite the legend that arm64 must be faster than 32bits version, here it’s not the case. While comparing binaries from the RPi repo might lead to this, that’s probably because the armhf binaries use conservative build option to stay compatible with the whole range of RPi, while those binaries comes from a distrib that is optimized for armv8, even for the 32bits build. So, while the ARMv8 ISA have many advantage over the ARMv7 (especially for 64bits integer handling!), it seems it’s not enough here to compensate for the 64bits address management…
The other analysis is, the speed of the emulated OpenArena is not that far from the native version! On 32bits, it’s 26.1 vs 31.9, so around 80% of native speed! It’s even better for the 64bits with 26.5 vs 31.3: almost 85% of native speed. And box64 is young, with some more space for optimisations!
I tried to run the benchmark on a PI/400, using the beta of the 64bits OS. But I ran in some issues: the 32bits package of libSDL1.2 has some weird dependencies that makes it impossible to install without removing critical components, and the OS also uses glibc 2.27, where the aarch64 version of OpenArena I use needs 2.29+. I cheated a bit and still got the 64bits version running. But on PI, at those settings, the game is completely GPU bound, giving just 9.3 fps for both ARM64 and x86_64 version. So, not a good test as-is for the PI.
So yeah, right between 50% and 100%, the emulated performances of OpenArena is at 80% (or 85% on 64bits). That’s pretty good! If some of you want to test, the package with the binary and the “bench.sh” script is available here (it will download the OpenArena binaries at the first launch).
14 replies on “Game performances”
ptitseb, if i were king, there would be statues made of you. <3
has apple or nvidia offered you a job based on this?
Thank you! Ah no, neither Apple nor NVidia contacted me, but that’s not a surprise 😉
[…] most sense as the long-term solution to me as it would open the door to allowing all games to work. box86 and box64 now have 80% the native performance of running AMD/Intel programs on Arm, and that’s the same performance of Rosetta 2 on the new Apple M1 Macs! […]
Great job! you are the man!
Thank you very much!!!
I was trying box64 on Manjaro (64bit kernel and userland) yesterday on a Raspberry pi 4 4GB and found that your benchmark script works almost flawlessly (just had to change the ‘unzip’ command for a ‘7z x’) 🙂
It ran at 9FPS as expected, but for me everything involving dynarec is a mix between magic and esoterism… And it’s an amazing start!
If you consider this useful, I was using glibc 2.32-2. If there is any more info that you’re interested about let me know. Thank you again!
Did you had enough precision to see some difference in performances between x86_64 and arm64 build?
Hey, not that much. x86_64 was in low 9s (between 8.9 and 9.2) and aarch64 was always over 9. As soon as I can check I’ll tell you
Worth noting: your arm64 vs arm32 comments aren’t taking into account that OpenArena likely isn’t doing much 64-bit math! Most games are float-heavy, not integer; it’s also highly likely that most integers are 32-bit-or-smaller.
Yes all Integer are 32bits there. With 64bits integer, no doubt the Arm64 would be much faster.
How to get the fps for this app, what tools are used?
The link is dead. Is there anywhere else I can read this article ?
Yes, I nopticed. I’ll repost the article here later.
[…] test wasn’t present last year. But I did write an article about it there.The benchmark use some heavy graphics settings, making it quite GPU-limited on low-end machines, so […]