IEEE Transactions on Software Engineering· 2014Q1· Review
The Oracle Problem in Software Testing: A Survey
- 1,043citations
- Q1SCImago
- 2014year
Short summary
This survey analyzes current approaches and trends for automating the 'test oracle problem' in software testing, which is the challenge of distinguishing correct system behavior from incorrect behavior given an input.
AI-generated from the title and abstract; the full text is not read.
Key points
- The 'test oracle problem' is a key bottleneck in achieving full test automation.
- Existing automation techniques include modeling, specifications, contract-driven development, and metamorphic testing.
- Human judgment, based on informal expectations and domain knowledge, is often the final source for oracle guidance.
- Balancing the cost and benefit of test oracle automation is a universal challenge.
AI-generated from the title and abstract; the full text is not read.
Abstract
Testing involves examining the behaviour of a system in order to discover potential faults. Given an input for a system, the challenge of distinguishing the corresponding desired, correct behaviour from potentially incorrect behavior is called the “test oracle problem”. Test oracle automation is important to remove a current bottleneck that inhibits greater overall test automation. Without test oracle automation, the human has to determine whether observed behaviour is correct. The literature on test oracles has introduced techniques for oracle automation, including modelling, specifications, contract-driven development and metamorphic testing. When none of these is completely adequate, the final source of test oracle information remains the human, who may be aware of informal specifications, expectations, norms and domain specific information that provide informal oracle guidance. All forms of test oracles, even the humble human, involve challenges of reducing cost and increasing benefit. This paper provides a comprehensive survey of current approaches to the test oracle problem and an analysis of trends in this important area of software testing research and practice.
The authors' abstract, as published at the source. IEEE Transactions on Software Engineering, 2014 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Software
SoftwareComputer Science