Combinatorial testing: using blocking to assign test cases for validating complex software systems

2021 ◽  
pp. 1-8
Author(s):  
Ryan Lekivetz ◽  
Joseph Morgan
Keyword(s):  
Software Systems ◽  
Test Cases ◽  
Combinatorial Testing

Research Notes: Binary Test-Suites Using Covering Arrays

2018 ◽  
Vol 28 (09) ◽  
pp. 1321-1337 ◽  
Author(s):  
Jose Torres-Jimenez ◽  
Himer Avila-George ◽  
Ezra Federico Parra-González
Keyword(s):  
Software Testing ◽  
Test Generation ◽  
Software Systems ◽  
Combinatorial Testing ◽  
Covering Array ◽  
Covering Arrays ◽  
Almost All ◽  
Generation Problem ◽  
Test Suites

Software testing is an essential activity to ensure the quality of software systems. Combinatorial testing is a method that facilitates the software testing process; it is based on an empirical evidence where almost all faults in a software component are due to the interaction of very few parameters. The test generation problem for combinatorial testing can be represented as the construction of a matrix that has certain properties; typically this matrix is a covering array. Covering arrays have a small number of tests, in comparison with an exhaustive approach, and provide a level of interaction coverage among the parameters involved. This paper presents a repository that contains binary covering arrays involving many levels of interaction. Also, it discusses the importance of covering array repositories in the construction of better covering arrays. In most of the cases, the size of the covering arrays included in the repository reported here are the best upper bounds known, moreover, the files containing the matrices of these covering arrays are available to be downloaded. The final purpose of our Binary Covering Arrays Repository (BCAR) is to provide software testing practitioners the best-known binary test-suites.

Combinatorial Testing

2010 ◽  
pp. 196-208 ◽  
Author(s):  
Renée C. Bryce ◽  
Yu Lei ◽  
D. Richard Kuhn ◽  
Raghu Kacker
Keyword(s):  
Economic Losses ◽  
Software Systems ◽  
Limited Time ◽  
Combinatorial Testing ◽  
Bodily Harm ◽  
Security Breaches ◽  
Test Suites

Software systems today are complex and have many possible configurations. Products released with inadequate testing can cause bodily harm, result in large economic losses or security breaches, and affect the quality of day-to-day life. Software testers have limited time and budgets, frequently making it impossible to exhaustively test software. Testers often intuitively test for defects that they anticipate while less foreseen defects are overlooked. Combinatorial testing can complement their tests by systematically covering t-way interactions. Research in combinatorial testing includes two major areas (1) algorithms that generate combinatorial test suites and (2) applications of combinatorial testing. The authors review these two topics in this chapter.

A Critical Review on Automated Test Case Generation for Conducting Combinatorial Testing Using Particle Swarm Optimization

2018 ◽  
Vol 7 (3.8) ◽  
pp. 22 ◽  
Author(s):  
Dr V. Chandra Prakash ◽  
Subhash Tatale ◽  
Vrushali Kondhalkar ◽  
Laxmi Bewoor
Keyword(s):  
Particle Swarm Optimization ◽  
Critical Review ◽  
Optimization Problem ◽  
Test Suite ◽  
Test Case Generation ◽  
Test Case ◽  
Test Cases ◽  
Combinatorial Testing ◽  
Swarm Optimization ◽  
Automated Test Case Generation

In software development life cycle, testing plays the significant role to verify requirement specification, analysis, design, coding and to estimate the reliability of software system. A test manager can write a set of test cases manually for the smaller software systems. However, for the extensive software system, normally the size of test suite is large, and the test suite is prone to an error committed like omissions of important test cases, duplication of some test cases and contradicting test cases etc. When test cases are generated automatically by a tool in an intelligent way, test errors can be eliminated. In addition, it is even possible to reduce the size of test suite and thereby to decrease the cost & time of software testing.It is a challenging job to reduce test suite size. When there are interacting inputs of Software under Test (SUT), combinatorial testing is highly essential to ensure higher reliability from 72 % to 91 % or even more than that. A meta-heuristic algorithm like Particle Swarm Optimization (PSO) solves optimization problem of automated combinatorial test case generation. Many authors have contributed in the field of combinatorial test case generation using PSO algorithms.We have reviewed some important research papers on automated test case generation for combinatorial testing using PSO. This paper provides a critical review of use of PSO and its variants for solving the classical optimization problem of automatic test case generation for conducting combinatorial testing.   

scholarly journals An Efficient Regression Testing By Computing Coverage Data For Software Evolution

2012 ◽  
pp. 176-179
Author(s):  
Machani SivaPrasad
Keyword(s):  
Software Evolution ◽  
Regression Testing ◽  
Test Suite ◽  
Software Systems ◽  
Test Cases ◽  
Dataflow Analysis ◽  
New Approach ◽  
Testing Techniques ◽  
Test Suit ◽  
Execution Tracing

Software systems is evolve continuously during development and maintenance. After software is modified regression testing is applied to software to ensure that It behaves intended and modifications not negatively impacts its original functionality .It is time consuming to rerun test suite T of program Pi on modified program Pi+1.So there are many regression testing techniques are there for doing regression testing. These are based on coverage data. So computing coverage data for Pi+1 without rerunning all test cases is the problem for doing regression testing of program Pi+1.This paper proposed a new approach that computes coverage data with selecting test cases T’ for the subsequent versions of the software .By computing coverage data for subsequent version of software on without rerunning entire test suit T we can improve overall time taken to retest the evolving software using Regression testing. This paper focus on improving the performance of regression testing for software evolve continuously during maintenance, by implementing a new approach for regression testing by computing coverage data for evolving software using dataflow analysis and execution tracing .

SETER

2012 ◽  
Vol 3 (3) ◽  
pp. 23-49 ◽  
Author(s):  
Ayda Saidane ◽  
Nicolas Guelfi
Keyword(s):  
Development Process ◽  
Security Requirements ◽  
Software Systems ◽  
Test Cases ◽  
Security Testing ◽  
Secure Systems ◽  
Architecture Model ◽  
Software Engineering Process ◽  
Secure Software Development

The quality of software systems strongly depends on their architecture. For this reason, taking into account security requirements at the architecture level is crucial for the success of secure software development. Today, systems are permanently evolving due to customer needs, technology evolution or maintenance constraints. Thus, a resilient secure system is expected to evolve towards more satisfaction of its security requirements (Guelfi 2011). In particular, such evolution process should identify and eliminate faults and vulnerabilities during the development process or runtime. This study focuses on the design phases and aims to propose a resilient software engineering process guaranteeing the development of secure systems that satisfy their critical requirements. During the development process, the system is expected to evolve until reaching satisfactory compliance against its requirements. The satisfaction computation is based on the quantification of failures and degradations. In this paper, the authors propose a novel architecture model-based security testing approach for identifying faults and vulnerabilities. The originality of the proposal resides in the usage of the architecture model for security testing and in coupling security requirements with threat model for generating both security functional test cases and malicious test cases. The assessment of the security requirements’ satisfaction and the overall system resilience is based on the test traces analysis. Throughout this study, a client-server system is used as a running example for illustrating the approach.

A Multi-Agent-Based Approach for Critical Components Identification and Testing

2017 ◽  
pp. 657-678
Author(s):  
D. Jeya Mala ◽  
R. Iswarya
Keyword(s):  
Crucial Role ◽  
Research Work ◽  
Software Systems ◽  
Test Cases ◽  
Agent Based ◽  
Multi Agent ◽  
Adequacy Criterion ◽  
Critical Components ◽  
Test Adequacy

In real time software systems, testing plays a crucial role as any of the critical components in these systems are left undetected, then inadvertent effects will happen which will lead to erroneous operations, system failure, high cost and resource wastage etc. To address this most important and the emergent problem, this research work proposes an effective method by means of multi-agents based approach to identify such critical components and execute test cases along the critical test paths which will aid in effectively covering them during testing. Finally, this paper also compared the performance with existing approaches in terms of time taken for the search process and the component coverage based test adequacy criterion to ensure quality of the software.

Optimization Driven Constraints Handling in Combinatorial Interaction Testing

2019 ◽  
Vol 10 (3) ◽  
pp. 19-37
Author(s):  
Ram Gouda ◽  
Chandraprakash V.
Keyword(s):  
Optimization Algorithm ◽  
Software Systems ◽  
Compact Set ◽  
Test Cases ◽  
Effective Manner ◽  
Constraints Handling ◽  
Configurable System ◽  
Interaction Testing ◽  
Average Size ◽  
Combinatorial Interaction Testing

The combinatorial strategy is useful in the reduction of the number of input parameters into a compact set of a system based on the combinations of the parameters. This strategy can be used in testing the behaviour that takes place when the events are allowed to be executed in an appropriate order. Basically, in the software systems, for the highly configurable system, the input configurations are based on the constraints, and the construction of this idea undergoes various kinds of difficulties. The proposed Jaya-Bat optimization algorithm is developed with the combinatorial interaction test cases in an effective manner in the presence of the constraints. The proposed Jaya-Bat based optimization algorithm is the integration of the Jaya optimization algorithm (JOA) and the Bat optimization algorithm (BA). The experimentation is carried out in terms of average size and the average time to prove the effectiveness of the proposed algorithm. From the results, it is clear that the proposed algorithm is capable of selecting the test cases optimally with better performance.

ComboRT: A New Approach for Generating Regression Test Cases for Evolving Programs

2016 ◽  
Vol 26 (06) ◽  
pp. 1001-1026 ◽  
Author(s):  
Xiaobing Sun ◽  
Xin Peng ◽  
Hareton Leung ◽  
Bin Li
Keyword(s):  
Impact Factor ◽  
Regression Testing ◽  
Software Systems ◽  
Test Case ◽  
Test Cases ◽  
Test Case Selection ◽  
Case Selection ◽  
Testing Techniques ◽  
The Impact ◽  
Ranked List

Regression testing is essential to ensure software quality during software evolution. Two widely-used regression testing techniques, test case selection and prioritization, are used to maximize the value of the continuously enlarging test suite. However, few works consider both these two techniques together, which decreases the usefulness of the independently studied techniques in practice. In the presence of changes during program evolution, regression testing is usually conducted by selecting the test cases that cover the impact results of the changes. It seldom considers the false-positives in the information covered. Hence, the effectiveness of such regression testing techniques is decreased. In this paper, we propose an approach, ComboRT, which combines test case selection and prioritization together to directly generate a ranked list of test cases. It is based on the impact results predicted by the change impact analysis (CIA) technique, FCA–CIA, which generates a ranked list of impacted methods. Test cases which cover these impacted methods are included in the new test suite. As each method predicted by FCA–CIA is assigned with an impact factor value corresponding to the probability of this method to be impacted, test cases are then ordered according to the impact factor values of the impacted methods. Empirical studies on four Java based software systems demonstrate that ComboRT can be effectively used for regression testing in object-oriented Java-based software systems during their evolution.

Sign in / Sign up

Export Citation Format

Share Document