Ironically, one of the biggest announcements at last year’s Re:Invent conference was also one of the quietest. A small blog post that appeared on the first day announced that Amazon has made available the first non-Intel based compute instances, known as “A1.” Based on the ARM architecture, these instances are Amazon’s own chip design (Graviton), which are optimized for performance and cost.
Having a custom high-performance chip is great. But you also need a viable operating system for these chips to be fully utilized as a compute platform. For this, AWS has ported over their version of Linux, in addition to offerings from Red Hat and Ubuntu. This combination makes the A1 instances a great contender for running web servers and compiled apps written in languages such as Java.
So, this had me thinking. How can I compare performance of these new instances?
Testing environment
For comparison-testing, I launched two EBS-based EC2 instances, of comparable size. Both contain 2 CPUs, 4 GB of RAM, and EBS disk performance up to 3,500 Mbps. For the A1 series, the size would be an a1.large. For the Intel-based system, this equates to a c5.large.
The systems were configurated identically with the following software:
- AWS Linux 2 – Kernel version 4.14.97-90.72
- Java – OpenJDK 1.8.0_191
- Apache Maven 3.6.0
Below is a screenshot showing each system with its configuration. The Intel system is on the left, the ARM is on the right.
For testing software, I settled on the Open Source Java benchmarking tool: JMH. You can read more about JMH here. JMH allowed me to run a simple Java program and compile-in the JMH performance libraries to measure thread execution. Compilation of the test program was performed individually on each server.
To kick-off the tests, I logged into each server and synced the SSH windows using tmux. Tmux let me kick-off the benchmark simultaneously to ensure both processes were started at exactly the same time.
The tests executed with five warmup threads prior to recording the performance of 25 threads for the actual benchmark.
Test Results
I was surprised (and not surprised) by the results. Below, you can see the overall execution time between both systems to be an almost perfect match, with the ARM system taking only a second longer. The same goes for the average score (measured in microseconds), which, rounded-down, equates to two seconds. In other words, near-identical performance.
Cost
So now that we have (somewhat) scientifically shown the performance between the ARM and Intel instances to be comparable, let’s look at the cost of the instances used in this test. Considering on-demand prices for the AWS region us-east-1, we can see the A1 cost is 40% lower than their Intel counterparts. This is a significant savings for nearly identical performance.
| Instance Type | Hourly Cost | Monthly Cost(750 Hours) | Yearly Cost |
| a1.large | $0.051 | $38.25 | $459 |
| C5.large | $0.085 | $68.75 | $765 |
Conclusion
Most of today’s compute workloads in the cloud are web-based static content or run open-source, compiled languages such as Java, Python and Node.js, to name a few. Additionally, with the growing popularity of Containers, their very portable nature makes those workloads contenders as well. Immediate savings can be had for these installations by simply changing the underlying hardware.
If you enjoyed this blog, I recommend taking a look at some related Anexinet blogs, including a series on the pillars of Amazon’s Well Architected Framework. And to learn how your own application stacks up, please feel free to schedule your FREE Well-Architected Review with a Certified AWS Solutions Architect from Anexinet.
Share on
