Building Ant System for Multi-Faceted Test Case Prioritization

This article presents the empirical study of multi-criteria test case prioritization. In this article, a test case prioritization problem with time constraints is being solved by using the ant colony optimization (ACO) approach. The ACO is a meta-heuristic and nature-inspired approach that has been applied for the statement of a coverage-based test case prioritization problem. The proposed approach ranks test cases using statement coverage as a fitness criteria and the execution time as a constraint. The proposed approach is implemented in MatLab and validated on widely used benchmark dataset, freely available on the Software Infrastructure Repository (SIR). The results of experimental study show that the proposed ACO based approach provides near optimal solution to test case prioritization problem.

Multi- Objective Crow Search and Fruit Fly Optimization for Combinatorial Test Case Prioritization

This paper proposes a novel test case prioritization technique, namely Multi- Objective Crow Search and Fruitfly Optimization (MOCSFO) for test case prioritization. The proposed MOCSFO is designed by integrating Crow search algorithm (CSA) and Chaotic Fruitfly optimization algorithm (CFOA). The optimal test cases are selected based on newly modelled fitness function, which consist of two parameters, namely average percentage of combinatorial coverage (APCC) and Normalized average of the percentage of faults detected (NAPFD). The test case to be selected is decided using a searching criterion or fitness based on sequential weighed coverage size. Accordingly, the effective searching criterion is formulated to determine the optimal test cases based on the constraints. The experimentation of the proposed MOCSFO method is performed by considering the performance metrics, like NAPFD, and APCC. The proposed MOCSFO outperformed the existing methods with enhanced NAPFD of 0.7, and APCC of 0.837.

Applying test case prioritization to software microbenchmarks

Regression testing comprises techniques which are applied during software evolution to uncover faults effectively and efficiently. While regression testing is widely studied for functional tests, performance regression testing, e.g., with software microbenchmarks, is hardly investigated. Applying test case prioritization (TCP), a regression testing technique, to software microbenchmarks may help capturing large performance regressions sooner upon new versions. This may especially be beneficial for microbenchmark suites, because they take considerably longer to execute than unit test suites. However, it is unclear whether traditional unit testing TCP techniques work equally well for software microbenchmarks. In this paper, we empirically study coverage-based TCP techniques, employing total and additional greedy strategies, applied to software microbenchmarks along multiple parameterization dimensions, leading to 54 unique technique instantiations. We find that TCP techniques have a mean APFD-P (average percentage of fault-detection on performance) effectiveness between 0.54 and 0.71 and are able to capture the three largest performance changes after executing 29% to 66% of the whole microbenchmark suite. Our efficiency analysis reveals that the runtime overhead of TCP varies considerably depending on the exact parameterization. The most effective technique has an overhead of 11% of the total microbenchmark suite execution time, making TCP a viable option for performance regression testing. The results demonstrate that the total strategy is superior to the additional strategy. Finally, dynamic-coverage techniques should be favored over static-coverage techniques due to their acceptable analysis overhead; however, in settings where the time for prioritzation is limited, static-coverage techniques provide an attractive alternative.