Instaclustr recently released support for Apache Cassandra 2.1 and our benchmark testing has demonstrated significant performance improvements delivered by this version of the database. Instaclustr customers simply need to log a support ticket to get their cluster upgraded to take advantage.
This article provides some of the results of recent benchmark testing that we have run with Cassandra 2.1 as well as some examples of real-world performance we are seeing with our customers.
Cassandra 2.1 Benchmarks
For these benchmarks we tested 3 node clusters of each of our AWS node sizes running Cassandra 2.1.7. We used the cassandra-stress tool to test basic read, write and mixed load scenarios. For the m3.xlarge size we also tested a 6 node cluster to confirm scale-out. The cluster used the default configuration provisioned by the Instaclustr system.
In each scenario, the keyspace was created with a replication factor of three and operations were performed at quorum consistency level. The number of threads used by cassandra-stress was (roughly) tuned to achieve optimum throughput. The average operations per second were calculated over a 10-minute period (following a warm-up). Each scenario was run twice and the results below represent the average operations per second (although results were remarkably consistent between runs).
The results of this testing are summarised in the table below:
| Nodes | Operations Per Second | |||
|---|---|---|---|---|
| Type | # | Mixed Load (3:1 read:write) |
Read Only | Insert Only |
| Dev/Light Production (m3.xlarge) | 3 | 31,377 | 40,330 | 27,243 |
| 6 | 63,980 | 88,737 | 47,492 | |
| Medium (c3.2xlarge) | 3 | 55,137 | 73,172 | 45,252 |
| Large (i2.xlarge) | 3 | 32,005 | 36,851 | 27,104 |
| Extra Large (i2.2xlarge) | 3 | 59,093 | 66,226 | 44,687 |
It should be noted that these benchmarks should be seen as the maximum achievable throughput with the relatively simple default data model generated by cassandra-stress. Actual performance for a particular application will depend on a range of factors specific to the application.
For initial sizing, assuming a relatively simple data model and a well-tuned application using batching and async operations appropriately, we would recommend discounting 20% for achievable peak load and 40% for achievable ongoing load (allowing for background tasks such as compactions, repairs and backups).
These benchmarks also demonstrate the significant performance gains over Cassandra 2.0 that can be achieved with Cassandra 2.1. The following tables summarise our results for c3.2xlarge 3 node cluster.
As can be seen from the tables, Cassandra 2.1 delivers a huge improvement in read performance and a significant improvement in write performance. It’s unclear at this point why the improvements in the isolated read and write test did not translate to similar magnitude improvements in the mixed load test – we will continue to investigate and provide an update when we get to the bottom of it.
| Cassandra Version | Operations Per Second | ||
|---|---|---|---|
| Mixed Load (3:1 read:write) |
Read Only | Insert Only | |
| Cassandra 2.0.16 | 50,370 | 45,959 | 40,416 |
| Cassandra 2.1.7 | 55,137 | 73,172 | 45,252 |
| Improvement | 9.5% | 59.2% | 12.0% |
Interestingly, the improvement in read performance comes primarily from reducing the variability of the performance of each read rather than a general reduction. The following table summarises the latencies we saw during the read test:
| Cassandra Version | Latency (ms) | ||
|---|---|---|---|
| Median | 95th Percentile | 99.9th Percentile | |
| Cassandra 2.0.16 | 5.2 | 101.1 | 172.2 |
| Cassandra 2.1.7 | 12.8 | 34.9 | 80.0 |
Our testing has clearly validated that worthwhile performance improvements can be achieved with upgrading to Cassandra 2.1. If you’re an Instaclustr customer with a 2.0 based cluster and would like to upgrade to 2.1 then simply contact [email protected] and we can arrange the upgrade for you. For most customers it’s simply a matter of changing a setting in our configuration database and then performing a rolling-restart of Cassandra in your cluster – our automated provisioning system will take care of the rest. Of course, as always, we recommend testing the upgrade in a non-production environment first.
Real-world Experience
Benchmarks with a stress tool are always a bit theoretical so it’s useful to also present some statistics from some real Cassandra clusters that we are running.
A good place to start is the Cassandra cluster that we run to collect and analyze monitoring data from all our managed nodes. This is currently a 6 node c3.2xlarge cluster running Cassandra 2.0.16 (soon to be upgrade to Cassandra 2.1). It serves around 1300 batched writes per second with a batch size of 40 records, so a total of 52,000 records per second being written with average CPU usage of 25% (user). This is a constant 24×7 load.
When it comes to performance and some of the workloads that we see within our customer environments, here are some performance statistics that have been extracted from our monitoring database:
- Peak write workload: 60,000 writes / second (6 node cluster, single customer)
- Peak read workload: 90,000 reads / second (6 node cluster, single customer)
- Biggest sustained production workload (average):
- 15,000 writes/second (9 nodes, single customer)
- 20,000 reads/second (9 nodes, single customer)
If you’re considering using Instaclustr or Cassandra and need some detailed advice on sizing then contact [email protected] and we’d be happy to arrange a conversation.