To contend with the huge number of tests, we’ve added filtering to the results view, allowing you to hide frameworks that do not meet your needs. You can filter by classification (full-stack, micro-framework, or platform); language; platform; front-end server; database server; ORM classification; and implementation approach.
Additionally, we’ve added our fourth test called “Fortunes” which exercises server-side templates and collections. The Fortunes test is implemented in 17 frameworks, including most mainstream contenders.
Round 4 notes and observations
- When viewing the data tables, you will notice that some frameworks show up more than once. Check the attribute columns at the right to compare the various test permutations. For example, in some cases a framework is tested both with and without its ORM (“full ORM” versus “Raw”).
- Onion, a C platform contributed by Coralbits is now included in the EC2 JSON serialization test and the results are extremely impressive, clocking in over 52,000 JSON responses per second.
- Thanks to work by Brad Fitzpatrick and others in the Go community, Go 1.1 is now a champion of database connectivity. In the previous round, Go exhibited serious problems with highly concurrent database utilization. Understatement: the Round 4 numbers for Go are impressive.
- The new Fortunes test exercises server-side templates and collections. We kept the payload small so that the test does not run into a Gigabit Ethernet wall. Thanks to contributors, the test is already implemented on 17 of the frameworks. The initial results in this round are certainly interesting, but we expect later rounds to be still more interesting as additional implementations arrive. A surprising performance pain point for some implementations is escaping of untrusted text. To create the plain Servlet implementation, we initially used an Apache StringEscapeUtils library but found its performance lacking. Since the Servlet implementation uses JSP as a view, we replaced StringEscapeUtils with JSTL’s “out” tag.
- Assigning attributes such as platform, ORM type, and classification (full-stack, micro, or platform) was an interesting challenge. We are not certain we have every framework described properly, so please contact us if you think we’ve mischaracterized any framework’s attributes.
- Django is now connecting to Postgres in this round, making this the first Postgres test in the suite. We plan to add more in time. Django performance is slightly improved on Postgres versus MySQL, presumably thanks to the use of a connection pool. However, the “Stripped” implementation of Django, which still uses MySQL, performs slightly better than the new Django-Postgres test.
- We’ve dropped the line charts in this round. We felt they did not add tremendous value to the site.
- We have not yet received the pull request with .NET tests. We hope they arrive soon so we can include them in Round 5.
Once again, thank you to all of the readers who have contributed tests and feedback. A special shout-out to Skamander whose contributions are numerous; just check the Github pull request history!
We provide web and mobile application development services and are passionate about application performance. Read more about what we do.