Third Round of the SBST Java Unit Testing Tool Contest

After two successful competitions in 2012 and 2013 we, again this year, invite developers of tools for Java unit testing at the class level—both SBST and non-SBST—to participate in the third round of our tools competition!

Competition entries are in the form of short papers (maximum of 4 pages) describing an evaluation of your tool against a benchmark supplied by the workshop organizers. In addition to comparing your tool to other popular and successful tools such as Randoop, we will manually create unit tests for the classes under test, to be able to obtain and compare benchmark scores for manual and automated test generation!

The results of the tools competition will be presented at the SBST 2015 workshop.

The Contest

The contest is targeted at developers/vendors of testing tools that generate test input data for unit testing java programs at the class level. Each tool will be applied on a set of java classes taken from open-source projects, and selected by the contest organization. The participating tools will be compared for:

  • execution time
  • achieved branch coverage
  • mutation score

Each participant should install a copy of their tool on the server where the contest will be run. To this end each participant will have SSH-access to the contest-server. The benchmark infrastructure will run the tools and measure their outputs fully automatically, therefore tools must be capable of running without human intervention.

Important Dates

  • Deadline for installing a copy of your tool on the server ready to be run and collect the results: February 6th, 2015
  • After the 6th of February the test will be run and results will be communicated ASAP.
  • Deadline for uploading the 4 page competition paper: February 18th, 2015

Find out more!