Minimal Detectable and Identifiable Biases for quality control

Abstract

The Minimal Detectable Bias (MDB) is an important diagnostic tool in data quality control. The MDB is traditionally computed for the case of testing the null hypothesis against a single alternative hypothesis. In the actual practice of statistical testing and data quality control, however, multiple alternative hypotheses are considered. We show that this has two important consequences for one's interpretation and use of the popular MDB. First, we demonstrate that care should be exercised in using the single-hypothesis-based MDB for the multiple hypotheses case. Second, we show that for identification purposes, not the MDB, but the Minimal Identifiable Bias (MIB) should be used as the proper diagnostic tool. We analyse the circumstances that drive the differences between the MDBs and MIBs, show how they can be computed using Monte Carlo simulation and illustrate by means of examples the significant differences that one can experience between detectability and identifiability.