[Read] (2011) EvoSuite: Automatic Test Suite Generation for Object-Oriented Software

2011. EvoSuite - Automatic Test Suite Generation for Object-Oriented Software

What are Oracles?

[Read] Seven Principles of Software Testing

0. Overview

1. Introduction

techniques: hybrid search, dynamic symbolic execution, and testability transformation

2. Whole Test Suite Optimization

2.1 Common Systematic Approach

A common systematic approach to test generation: selection one coverage goal for a given coverage criterion at a time_ (e.g. a program branch or a control flow path_) and derive a test case that exercises this particular goal.

2.2 Whole Test Suite Generation

Do not produce individual test cases for individual coverage goals, but instead focus on test suites targeting an entire coverage criterion. -> There is no collateral coverage.

EvoSuite implements whole test suite generation as a search-based approach.

To help the generation of appropriate input data, EvoSuite also makes use of focused local searches and dynamic symbolic execution after every predefined number of generations.

Fitness example: branch coverage -> the fitness calculates the sum of the individual branch distance of all the branches in the UUT.

Branch Distance: how far an individual execution is from having a branch evaluate to true or to false (e.g. the condition x=42 with the value 10 for x has a branch distance of \(|42-10|=32\) if becoming true. fitness 0 -> cover all branches of a UUT)

At the end of the search, test suites are minimized such that only statements contributing to coverage remain.

3. Mutation-based Assertion Generation

The test oracle problem is one of the main problems that remain in traditional white-box test generation.

4. Implementation

Java bytecode -> JUnit test suite

one class at a time, maximizing branch coverage for this class variants

single threading application

dynamic symbolic execution -> improve over random testing and its variants