Constructing t-way test suites: a systematic survey and qualitative synthesis of the One-Parameter-at-a-Time approach

Abstract

Parameter interactions are a major source of software failures, yet the exponential growth of possible configurations makes exhaustive testing impractical. Combinatorial Interaction Testing (CIT) is a highly effective sampling technique for identifying software defects caused by parameter interactions. Over the past two decades, the One-Parameter-at-a-Time (OPAT) constructions paradigm has emerged as a cornerstone of CIT test suite generation. Although numerous OPAT strategies have been proposed to optimize test suite size and execution speed, the literature remains fragmented, and direct side-by-side quantitative benchmarking has been impeded by a lack of open-source tooling and standardized platforms. To address this, this study presents a systematic survey and qualitative synthesis of 40 primary OPAT studies published between 2000 and 2026, retrieved from four major digital libraries. Rather than attempting a direct quantitative benchmark comparison of execution times, which is shown to be mathematically invalid due to highly heterogeneous self-reported hardware environments and the closed-source nature of most academic tools, we establish a unified taxonomy that classifies OPAT research into five major categories: deterministic refinements, scaling and distribution strategies, advanced feature-enhanced strategies, metaheuristic and hybrid strategies, and specialized OPAT adaptations. Our analysis synthesizes qualitative trends, structural characteristics, parameter-order growth mechanisms, and feature-integration gaps across the literature. We conclude with a critical assessment of the tooling and reproducibility challenges in the field, outlining vital future pathways, including diversified metaheuristic and hyper-heuristic techniques, deterministic parallel algorithms, and AI-driven adaptive test generation, to transition OPAT research into industrial-grade, open-source software engineering pipelines.

The authors' abstract, as published at the source. Frontiers in Computer Science, 2026 · DOI ↗