Merry Christmas web framework performance aficionados! What better way to celebrate the holidays than by cheering on your favorites as they race through a variety of application fundamentals in the biggest web platform grudge match of the season? We certainly can't think of anything more festive.
Now at 90 frameworks and 230 permutations (variations on configuration), Round 8 has something for everyone. And if it doesn't have what you want, you can join the party! We have fruitcake and egg nog. Or maybe not. But we enjoy pull requests; they're almost as good as egg nog.
A veritable rainbow of holiday cheer awaits!
Round 8 notes and observations
- Go, always the scrappy competitor, flexes some performance muscle and lands a razor-thin victory in JSON serialization on i7 hardware. But be aware, the highest-performance frameworks are network limited in the JSON serialization and Plaintext tests. Anything in this "200k club" is sure to keep overhead at a bare minimum, leaving maximum headroom for your custom application logic.
- Round 7 was missing some of the Go frameworks due to configuration problems. Those problems have been resolved, and the Go frameworks have returned in Round 8 to reaffirm that Go is a viable performance rival to the JVM.
- The HipHop PHP VM with no framework and thanks in part to the MySQL driver for PHP, yields dominion over the Updates test. HHVM is impressive in the multiple query test as well. However, hhvm trails plain PHP in the Fortunes test presently. Implementation details may be at play here. If you're interested in testing HHVM with popular PHP frameworks, we would be happy to receive a pull request.
- Vert.x and Netty have wrestled the Plaintext crown from Undertow, but this rivalry isn't yet settled. Rumor has it they have more improvements in store for Round 9. Meanwhile, a newcomer named Plain (which may rival Go as the most in need of a more search-friendly name; though the irony of Go makes it uncontested champion) is right behind the leaders. Most interestingly, Plain demonstrates the highest Windows performance we've seen by a massive margin (reaching 611,095 pipelined plaintext requests per second on i7). Once again, bear in mind that these tests are network-bound by our gigabit Ethernet.
- On EC2, the Netty and Vert.x upgrades have paid huge dividends with Netty now breaking 200,000 pipelined plaintext responses per second on a humble m1.large instance.
- Grizzly performance on JSON serialization is off from its Round 7 showing, but unfortunately, we have not yet determined the cause.
- The Plaintext test requirements were clarified. It is not necessary to copy the bytes of the small response payload per request. Using a pre-rendered byte buffer for the body is acceptable as long as that is conventional for the platform or framework being tested and response headers are composed normally.
- The maximum query and update performance for Mongo on EC2 is substantially higher than MySQL. When looking at that chart in particular, consider filtering by your preferred data store to maintain a useful perspective. Related: a late change to the Mongo "schema" intended to replace "id" with "_id" caused some challenges. We further postponed Round 8 to re-run Mongo tests with a schema that provides both columns to allow all tests to complete. We want to normalize the implementations for Round 9.
- We were targeting early December for Round 8 and we're off by about two weeks. There is still room for improvement toward our goal of a monthly cycle. We will target mid-January for Round 9.
Thanks!
A big thank-you to all of the contributors who have added and improved existing test implementations for Round 8.
The contributors for Round 8 are, in no particular order: @lhotari, @methane, @pseudonom, @lucassp, @aualin, @weltermann17, @kpacha, @nareshv, @martin-g, @bclozel, @ijl, @bbrowning, @sbordet, @purplefox, @stuartwdouglas, @normanmaurer, @kardianos, @hamiltont, and @julienschmidt.
If you have questions, comments, criticism, or would like to contribute a new test or an improvement to an existing one, please join our Google Group or visit the project at Github.
About TechEmpower
We provide web and mobile application development services and are passionate about application performance. Read more about what we do.
