Hypothesis Testing validation

Free hypothesis testing for environmental datasets: one- and two-sample t-tests, the Wilcoxon-Mann-Whitney test and one-way ANOVA with Bonferroni post-hoc comparisons.

Scenarios: 4
Values agreeing: 15
Values differing: 0
Validated against: R EnvStats / nortest, USEPA ProUCL

Open Hypothesis Testing

The one-sample t-test, Welch two-sample t-test and Wilcoxon-Mann-Whitney test are compared against R EnvStats. Analysis of variance is compared against USEPA ProUCL, agreeing on the F statistic and p-value to thirteen decimal places.

Scenarios

Non-parametric site vs background comparisonTypical use
One-sample t against a standard, upper tailTypical use
Site vs background, unequal variances (Welch)Typical use
One-way ANOVA across three groups of unequal sizeBoundary case

Non-parametric site vs background comparison

Typical use

Input

Group Site (8 values)

value
12.4
15.1
11.8
18.2
14.6
16.9
13.2
19.4

Group Background (7 values)

value
8.1
9.4
7.7
10.2
8.8
9.1
11.5

Settings: test = mannWhitney, tail = twoSided, significance = 0.05

Against R EnvStats / nortest (EnvStats 3.1.0, nortest 1.0-4, R 4.6.1)

Value	R EnvStats / nortest	EnviroAutomate	Difference	Agreement band	Verdict
Test statistic testStatistic	0	0	0	within 0.000001 relative	Agree
p-value pValue	0.00146006191624215	0.0014600619163627	1.21e-13	within 1e-9	Agree

Not run for this scenario

USEPA ProUCL - ProUCL does not expose this test in a form that can be compared directly, so it is checked against R instead.

One-sample t against a standard, upper tail

Typical use

Input

Group Site (8 values)

value
12.4
15.1
11.8
18.2
14.6
16.9
13.2
19.4

Settings: test = oneSampleT, tail = greater, standard = 12, significance = 0.05

Against R EnvStats / nortest (EnvStats 3.1.0, nortest 1.0-4, R 4.6.1)

Value	R EnvStats / nortest	EnviroAutomate	Difference	Agreement band	Verdict
Test statistic testStatistic	3.280049420759791	3.2800494207597914	4.44e-16	within 0.000001 relative	Agree
Degrees of freedom degreesOfFreedom	7	7	0	within 0.000001 relative	Agree
p-value pValue	0.006743070224416923	0.006743070224416892	3.12e-17	within 1e-9	Agree

Not run for this scenario

USEPA ProUCL - ProUCL does not expose this test in a form that can be compared directly, so it is checked against R instead.

Site vs background, unequal variances (Welch)

Typical use

Input

Group Site (7 values)

value
12.4
15.1
11.8
18.2
14.6
16.9
13.2

Group Background (6 values)

value
8.1
9.4
7.7
10.2
8.8
9.1

Settings: test = twoSampleTWelch, tail = twoSided, significance = 0.05

Against R EnvStats / nortest (EnvStats 3.1.0, nortest 1.0-4, R 4.6.1)

Value	R EnvStats / nortest	EnviroAutomate	Difference	Agreement band	Verdict
Test statistic testStatistic	5.945003743912736	5.945003743912735	8.88e-16	within 0.000001 relative	Agree
Degrees of freedom degreesOfFreedom	7.957568411673562	7.95756841167356	1.78e-15	within 0.000001 relative	Agree
p-value pValue	0.00035106057838176	0.000351060578381857	9.61e-17	within 1e-9	Agree

Not run for this scenario

USEPA ProUCL - ProUCL does not expose this test in a form that can be compared directly, so it is checked against R instead.

One-way ANOVA across three groups of unequal size

Boundary case

Input

Group A (5 values)

value
12.4
15.1
11.8
18.2
14.6

Group B (7 values)

value
8.1
9.4
7.7
10.2
8.8
9.1
9.9

Group C (4 values)

value
21.3
19.8
22.6
20.1

Settings: test = anova, significance = 0.05

Against R EnvStats / nortest (EnvStats 3.1.0, nortest 1.0-4, R 4.6.1)

Value	R EnvStats / nortest	EnviroAutomate	Difference	Agreement band	Verdict
Test statistic testStatistic	66.80428408351607	66.80428408351612	5.68e-14	within 0.000001 relative	Agree
Degrees of freedom degreesOfFreedom	2	2	0	within 0.000001 relative	Agree
p-value pValue	1.44742476870365e-7	1.44742476893001e-7	2.26e-17	within 1e-9	Agree

Against USEPA ProUCL (ProUCL 5.2.0)

Value	USEPA ProUCL	EnviroAutomate	Difference	Agreement band	Verdict
Test used testUsed	anova	anova	-	exact match	Agree
Test statistic testStatistic	66.80428408351628	66.80428408351612	1.56e-13	within 0.000001 relative	Agree
Degrees of freedom degreesOfFreedom	2	2	0	within 0.000001 relative	Agree
p-value pValue	1.44742476893001e-7	1.44742476893001e-7	0	within 1e-9	Agree

Evidence generated 2026-08-26 from a full comparison run against the versions named above.