Test Oracle Generation Based on BPNN by Using the Values of Variables at Different Breakpoints for Programs

2021 ◽  
Vol 31 (10) ◽  
pp. 1469-1494
Author(s):  
Chunyan Ma ◽  
Shaoying Liu ◽  
Jinglan Fu ◽  
Tao Zhang
Keyword(s):  
Prediction Accuracy ◽  
Recall Rate ◽  
Training Data ◽  
New Method ◽  
Test Case ◽  
Test Cases ◽  
Test Oracles ◽  
Test Input ◽  
Test Oracle ◽  
The Impact

Automatic test oracle generation is a bottleneck in realizing full automation of the entire software testing process. This study proposes a new method for automatically generating a test oracle for a new test input on the basis of several historical test cases by using a backpropagation neural network (BPNN) model. The new method is different from existing test oracle techniques. Specifically, our method has two steps. First, the values of variables are collected as training data when several historical test inputs are used to execute the program at different breakpoints. The test oracles (pass or fail) of these test cases are utilized to classify and label the training data. Second, a new test input is used to execute the program at different breakpoints, where the trained BPNN prediction model automatically generates its test oracle on the basis of the collected values of the variables involved. We conduct an experiment to validate our method. In the experiment, 113 faulty versions of seven types of programs are used as experimental objects. Results show that the average prediction accuracy rate of 74,651 test oracles is 95.8%. Although the failed test cases in the training data account for less than 5%, the overall average recall rate (prediction accuracy of test case execution failure) of all programs is 78.9%. Furthermore, the trained BPNN can reveal not only the impact of the values of variables but also the impact of the logical correspondence between variables in test oracle generation.

A FRAMEWORK FOR STATISTICAL TESTING OF SOFTWARE COMPONENTS

2007 ◽  
Vol 17 (03) ◽  
pp. 379-405
Author(s):  
RAKESH SHUKLA ◽  
PAUL STROOPER ◽  
DAVID CARRINGTON
Keyword(s):  
Probability Distribution ◽  
Statistical Testing ◽  
Test Case Generation ◽  
Test Case ◽  
Test Cases ◽  
Software Components ◽  
Test Oracles ◽  
Test Oracle ◽  
Operational Profile ◽  
Actual Operation

Statistical testing involves the testing of software by selecting test cases from a probability distribution that is intended to represent the software's operational usage. In this paper, we describe and evaluate a framework for statistical testing of software components that incorporates test case execution and output evaluation. An operational profile and a test oracle are essential for the statistical testing of software components because they are used for test case generation and output evaluation respectively. An operational profile is a set of input events and their associated probabilities of occurrence expected in actual operation. A test oracle is a mechanism that is used to check the results of test cases. We present four types of operational profiles and three types of test oracles, and empirically evaluate them using the framework by applying them to two software components. The results show that while simple operational profiles may be effective for some components, more sophisticated profiles are needed for others. For the components that we tested, the fault-detecting effectiveness of the test oracles was similar.

ComboRT: A New Approach for Generating Regression Test Cases for Evolving Programs

2016 ◽  
Vol 26 (06) ◽  
pp. 1001-1026 ◽  
Author(s):  
Xiaobing Sun ◽  
Xin Peng ◽  
Hareton Leung ◽  
Bin Li
Keyword(s):  
Impact Factor ◽  
Regression Testing ◽  
Software Systems ◽  
Test Case ◽  
Test Cases ◽  
Test Case Selection ◽  
Case Selection ◽  
Testing Techniques ◽  
The Impact ◽  
Ranked List

Regression testing is essential to ensure software quality during software evolution. Two widely-used regression testing techniques, test case selection and prioritization, are used to maximize the value of the continuously enlarging test suite. However, few works consider both these two techniques together, which decreases the usefulness of the independently studied techniques in practice. In the presence of changes during program evolution, regression testing is usually conducted by selecting the test cases that cover the impact results of the changes. It seldom considers the false-positives in the information covered. Hence, the effectiveness of such regression testing techniques is decreased. In this paper, we propose an approach, ComboRT, which combines test case selection and prioritization together to directly generate a ranked list of test cases. It is based on the impact results predicted by the change impact analysis (CIA) technique, FCA–CIA, which generates a ranked list of impacted methods. Test cases which cover these impacted methods are included in the new test suite. As each method predicted by FCA–CIA is assigned with an impact factor value corresponding to the probability of this method to be impacted, test cases are then ordered according to the impact factor values of the impacted methods. Empirical studies on four Java based software systems demonstrate that ComboRT can be effectively used for regression testing in object-oriented Java-based software systems during their evolution.

scholarly journals The impact of test case summaries on bug fixing performance: An empirical investigation

2016 ◽  
Author(s):  
Sebastiano Panichella ◽  
Annibale Panichella ◽  
Moritz Beller ◽  
Andy Zaidman ◽  
Harald C Gall
Keyword(s):  
Test Generation ◽  
Empirical Investigation ◽  
Test Case ◽  
Test Cases ◽  
Minimal Set ◽  
Automated Test Generation ◽  
Structural Coverage ◽  
Manual Testing ◽  
The Cost ◽  
The Impact

Automated test generation tools have been widely investigated with the goal of reducing the cost of testing activities. However, generated tests have been shown not to help developers in detecting and finding more bugs even though they reach higher structural coverage compared to manual testing. The main reason is that generated tests are difficult to understand and maintain. Our paper proposes an approach, coined TestScribe, which automatically generates test case summaries of the portion of code exercised by each individual test, thereby improving understandability. We argue that this approach can complement the current techniques around automated unit test generation or search-based techniques designed to generate a possibly minimal set of test cases. In evaluating our approach we found that (1) developers find twice as many bugs, and (2) test case summaries significantly improve the comprehensibility of test cases, which is considered particularly useful by developers.

Software Change Impact Analysis

2015 ◽  
Vol 5 (2) ◽  
pp. 44-55
Author(s):  
Chetna Gupta ◽  
Varun Gupta
Keyword(s):  
Impact Analysis ◽  
Software Engineer ◽  
Maintenance Cost ◽  
Test Case ◽  
Test Cases ◽  
Change Impact Analysis ◽  
Call Graph ◽  
Regression Test ◽  
Change Impact ◽  
The Impact

This paper presents an approach to prioritize program segments within the impact set computed using functional call graph to assist regression testing for test case prioritization. The presented technique will first categorize the type of impact propagation and then prioritize the impacted segments into higher and lower levels based on propagation categorization. This will help in saving maintenance cost and effort by allocating higher priority to those segments which are impacted more within the impacted set. Thus a software engineer can first run those test cases which cover segments with higher impacted priority to minimize regression test selection.

scholarly journals The impact of test case summaries on bug fixing performance: An empirical investigation

2016 ◽  
Author(s):  
Sebastiano Panichella ◽  
Annibale Panichella ◽  
Moritz Beller ◽  
Andy Zaidman ◽  
Harald C Gall
Keyword(s):  
Test Generation ◽  
Empirical Investigation ◽  
Test Case ◽  
Test Cases ◽  
Minimal Set ◽  
Automated Test Generation ◽  
Structural Coverage ◽  
Manual Testing ◽  
The Cost ◽  
The Impact

Automated test generation tools have been widely investigated with the goal of reducing the cost of testing activities. However, generated tests have been shown not to help developers in detecting and finding more bugs even though they reach higher structural coverage compared to manual testing. The main reason is that generated tests are difficult to understand and maintain. Our paper proposes an approach, coined TestScribe, which automatically generates test case summaries of the portion of code exercised by each individual test, thereby improving understandability. We argue that this approach can complement the current techniques around automated unit test generation or search-based techniques designed to generate a possibly minimal set of test cases. In evaluating our approach we found that (1) developers find twice as many bugs, and (2) test case summaries significantly improve the comprehensibility of test cases, which is considered particularly useful by developers.

scholarly journals The PAU Survey: Photometric redshifts using transfer learning from simulations

2020 ◽  
Vol 497 (4) ◽  
pp. 4565-4579
Author(s):  
M Eriksen ◽  
A Alarcon ◽  
L Cabayol ◽  
J Carretero ◽  
R Casas ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Broad Band ◽  
Training Data ◽  
Machine Learning Techniques ◽  
Test Case ◽  
Photometric Redshifts ◽  
Mixture Density ◽  
Photometric Redshift ◽  
The Galaxy ◽  
The Impact

ABSTRACT In this paper, we introduce the deepz deep learning photometric redshift (photo-z) code. As a test case, we apply the code to the PAU survey (PAUS) data in the COSMOS field. deepz reduces the σ68 scatter statistic by 50 per cent at iAB = 22.5 compared to existing algorithms. This improvement is achieved through various methods, including transfer learning from simulations where the training set consists of simulations as well as observations, which reduces the need for training data. The redshift probability distribution is estimated with a mixture density network (MDN), which produces accurate redshift distributions. Our code includes an autoencoder to reduce noise and extract features from the galaxy SEDs. It also benefits from combining multiple networks, which lowers the photo-z scatter by 10 per cent. Furthermore, training with randomly constructed coadded fluxes adds information about individual exposures, reducing the impact of photometric outliers. In addition to opening up the route for higher redshift precision with narrow bands, these machine learning techniques can also be valuable for broad-band surveys.

scholarly journals The impact of test case summaries on bug fixing performance: An empirical investigation

2015 ◽  
Author(s):  
Sebastiano Panichella ◽  
Annibale Panichella ◽  
Moritz Beller ◽  
Andy Zaidman ◽  
Harald C Gall
Keyword(s):  
Test Generation ◽  
Empirical Investigation ◽  
Test Case ◽  
Test Cases ◽  
Minimal Set ◽  
Automated Test Generation ◽  
Structural Coverage ◽  
Manual Testing ◽  
The Cost ◽  
The Impact

Automated test generation tools have been widely investigated with the goal of reducing the cost of testing activities. However, generated tests have been shown not to help developers in detecting and finding more bugs even though they reach higher structural coverage compared to manual testing. The main reason is that generated tests are difficult to understand and maintain. Our paper proposes an approach, coined TestScribe, which automatically generates test case summaries of the portion of code exercised by each individual test, thereby improving understandability. We argue that this approach can complement the current techniques around automated unit test generation or search-based techniques designed to generate a possibly minimal set of test cases. In evaluating our approach we found that (1) developers find twice as many bugs, and (2) test case summaries significantly improve the comprehensibility of test cases, which is considered particularly useful by developers.

scholarly journals Reducing the Discard of MBT Test Cases

2020 ◽  
Vol 8 ◽  
pp. 4-1 - 4:15
Author(s):  
Thomaz Diniz ◽  
Everton L G Alves ◽  
Anderson G F Silva ◽  
Wilkerson L Andrade
Keyword(s):  
Machine Learning ◽  
Distance Functions ◽  
Test Case ◽  
Test Cases ◽  
Use Case ◽  
Industrial Data ◽  
Model Based Testing ◽  
Industrial Projects ◽  
The Impact ◽  
Test Suites

Model-Based Testing (MBT) is used for generating test suites from system models. However, as software evolves, its models tend to be updated, which may lead to obsolete test cases that are often discarded. Test case discard can be very costly since essential data, such as execution history, are lost. In this paper, we investigate the use of distance functions and machine learning to help to reduce the discard of MBT tests. First, we assess the problem of managing MBT suites in the context of agile industrial projects. Then, we propose two strategies to cope with this problem: (i) a pure distance function-based. An empirical study using industrial data and ten different distance functions showed that distance functions could be effective for identifying low impact edits that lead to test cases that can be updated with little effort. We also found the optimal configuration for each function. Moreover, we showed that, by using this strategy, one could reduce the discard of test cases by 9.53%; (ii) a strategy that combines machine learning with distance values. This strategy can classify the impact of edits in use case documents with accuracy above 80%; it was able to reduce the discard of test cases by 10.4% and to identify test cases that should, in fact, be discarded.

scholarly journals Evaluating Methods of Updating Training Data in Long-Term Genomewide Selection

2016 ◽  
Author(s):  
Jeffrey L. Neyhart ◽  
Tyler Tiede ◽  
Aaron J. Lorenz ◽  
Kevin P. Smith
Keyword(s):  
Genetic Gain ◽  
Prediction Accuracy ◽  
Training Data ◽  
Training Population ◽  
Optimal Method ◽  
Phenotypic Data ◽  
Breeding Cycles ◽  
Genomewide Selection ◽  
The Impact

ABSTRACTGenomewide selection is hailed for its ability to facilitate greater genetic gains per unit time. Over breeding cycles, the requisite linkage disequilibrium (LD) between quantitative trait loci (QTL) and markers is expected to change as a result of recombination, selection, and drift, leading to a decay in prediction accuracy. Previous research has identified the need to update the training population using data that may capture new LD generated over breeding cycles, however optimal methods of updating have not been explored. In a barley (Hordeum vulgare L.) breeding simulation experiment, we examined prediction accuracy and response to selection when updating the training population each cycle with the best predicted lines, the worst predicted lines, both the best and worst predicted lines, random lines, criterion-selected lines, or no lines. In the short-term, we found that updating with the best predicted lines or the best and worst predicted lines resulted in high prediction accuracy and genetic gain, but in the long-term, all methods (besides not updating) performed similarly. We also examined the impact of including all data in the training population or only the most recent data. Though patterns among update methods were similar, using a smaller, but more recent training population provided a slight advantage in prediction accuracy and genetic gain. In an actual breeding program, a breeder might desire to gather phenotypic data on lines predicted to be the best, perhaps to evaluate possible cultivars. Therefore, our results suggest that an optimal method of updating the training population is also very practical.

Optimized Shroud Design for Axial Turbine Aerodynamic Performance

2007 ◽  
Author(s):  
L. Porreca ◽  
A. I. Kalfas ◽  
R. S. Abhari
Keyword(s):  
Flow Field ◽  
Aerodynamic Performance ◽  
Field Analysis ◽  
Test Case ◽  
Test Cases ◽  
Axial Turbine ◽  
Significant Difference ◽  
Flow Field Analysis ◽  
The Impact ◽  
Partial Shroud

This paper presents a comprehensive study of the effect of shroud design in axial turbine aerodynamics. Experimental measurements and numerical simulations has been conducted on three different test cases with identical blade geometry and tip clearances but different shroud designs. The first and the second test cases are representative of a full shroud and a non-axisymmetric partial shroud geometry while the third test case however uses an optimized partial shroud. Partial shrouds are sometimes used in industrial application in order to benefit from the advantage of shrouded configuration as well as reducing mechanical stress on the blades. However, the optimal compromise between mechanical considerations and aerodynamic performances is still an open issue due to the resulting highly 3-dimensional unsteady flow field. Aerodynamic performance is measured in a low-speed axial turbine facility and shows that there are clear differences between the test cases. In addition, steady and time resolved measurements are performed together with computational analysis in order to improve understanding of the effect of the shroud geometry on the flow field and to quantify the sources of the resultant additional losses. The flow field analysis shows that the effect of the shroud geometry is significant from 60% blade height span to the tip. Tip leakage vortex in the first rotor is originated in the partial shroud test cases while the full shroud case present only a weak indigenous tip passage vortex. This results in a significant difference in the secondary flow development in the following second stator with associated losses that varies of about 1% in this row. The analysis shows that the modified partial shroud design has improved considerably the aerodynamic efficiency of about 0.6% by keeping almost unchanged the overall weight of this component and thus blade root stresses. The work therefore presents a comprehensive flow field analysis and the shows the impact of the shroud geometry in the aerodynamic performance.

Sign in / Sign up

Export Citation Format

Share Document