Significance Levels for Multiple Tests

Sergiu Hart and Benjamin Weiss

(Acrobat PDF file)

Let X1, ..., Xn be n random variables, with cumulative distribution functions F1, ..., Fn. Define ξi := Fi(Xi) for all i, and let ξ(1) ≤ ... ≤ ξ(n) be the order statistics of the (ξi)i. Let α1 ≤ ... ≤ αn be n numbers in the interval [0,1]. We show that the probability of the event R := {ξ(i)αi for all 1 ≤ in} is at most mini{n αi / i}. Moreover, this bound is exact: for any given n marginal distributions (Fi)i, there exists a joint distribution with these marginals such that the probability of R is exactly mini{n αi / i}. This result is used in analyzing the significance level of multiple hypotheses testing. In particular, it implies that the Rüger tests dominate all tests with rejection regions of type R as above.