Appendix 3

The arithmetic mean, $\bar{y}$ , shall be calculated by means of equation (7-155):

\bar{y} = \frac{\sum_{i = 1}^{N} y_{i}}{N}

(7-155)

The standard deviation for a non-biased (e.g., N–1) sample, σ, shall be calculated by means of equation (7-156):

σ_{y} = \sqrt{\frac{\sum_{i = 1}^{N} {(y_{i} - \bar{y})}^{2}}{(N - 1)}}

(7-156)

The root mean square, rms_y , shall be calculated by means of equation (7-157):

{rms}_{y} = \sqrt{\frac{1}{N} \sum_{i = 1}^{N} y_{2}^{i}}

(7-157)

It shall be determined if the data passes a t-test by means of the following equations and Table 7.8:

(a)

For an unpaired t-test, the t statistic and its number of degrees of freedom, v, shall be calculated by means of equations (7-158) and (7-159):

$t = \frac{\|{\bar{y}}_{ref} - \bar{y}\|}{\sqrt{\frac{σ_{2}^{ref}}{N_{ref}} + \frac{σ_{2}^{y}}{N}}}$	(7-158)
$v = \frac{{(\frac{σ_{2}^{ref}}{N_{ref}} + \frac{σ_{2}^{y}}{N})}^{2}}{\frac{{(σ_{2}^{ref} ∕ N_{ref})}^{2}}{N_{ref} - 1} + \frac{{(σ_{2}^{y} ∕ N)}^{2}}{N - 1}}$	(7-159)

(b)

For a paired t-test, the t statistic and its number of degrees of freedom, v, shall be calculated by means of equation (7-160), noting that the ε _i are the errors (e.g., differences) between each pair of y _ref _i and y_i :

t = \frac{|\bar{ε}| \times \sqrt{N}}{σ_{ε}}

v= N – 1

(7-160)

(c)

Table 7.8 shall be used to compare t to the t _crit values tabulated versus the number of degrees of freedom. If t is less than t _crit, then t passes the t-test.

v	Confidence
Table 7.8
Critical t values versus number of degrees of freedom, v
	90 %	95 %
1	6,314	12,706
2	2,920	4,303
3	2,353	3,182
4	2,132	2,776
5	2,015	2,571
6	1,943	2,447
7	1,895	2,365
8	1,860	2,306
9	1,833	2,262
10	1,812	2,228
11	1,796	2,201
12	1,782	2,179
13	1,771	2,160
14	1,761	2,145
15	1,753	2,131
16	1,746	2,120
18	1,734	2,101
20	1,725	2,086
22	1,717	2,074
24	1,711	2,064
26	1,706	2,056
28	1,701	2,048
30	1,697	2,042
35	1,690	2,030
40	1,684	2,021
50	1,676	2,009
70	1,667	1,994
100	1,660	1,984
1 000+	1,645	1,960

Linear interpolation shall be used to establish values not shown here.

The F statistic shall be calculated by means of equation (7-161):

F_{y} = \frac{σ_{2}^{y}}{σ_{2}^{ref}}

(7-161)

(a)

For a 90 % confidence F-test, Table 7.9 is used to compare F to the F _crit90 values tabulated versus (N–1) and (N _ref–1). If F is less than F _crit90, then F passes the F-test at 90 % confidence;
(b)

For a 95 % confidence F-test, Table 7.10 is used to compare F to the F _crit95 values tabulated versus (N–1) and (N _ref–1). If F is less than F _crit95, then F passes the F-test at 95 % confidence.

F1Table 7-9

Critical F values, F _crit90 , versus N – 1 and N _{ref – 1} at 90 per cent confidence

N – 1	1	2	3	4	5	6	7	8	9	10	12	15	20	24	30	40	60	120	1000+
N _ref – 1
1	39,86	49,50	53,59	55,83	57,24	58,20	58,90	59,43	59,85	60,19	60,70	61,22	61,74	62,00	62,26	62,52	62,79	63,06	63,32
2	8,526	9,000	9,162	9,243	9,293	9,326	9,349	9,367	9,381	9,392	9,408	9,425	9,441	9,450	9,458	9,466	9,475	9,483	9,491
3	5,538	5,462	5,391	5,343	5,309	5,285	5,266	5,252	5,240	5,230	5,216	5,200	5,184	5,176	5,168	5,160	5,151	5,143	5,134
4	4,545	4,325	4,191	4,107	4,051	4,010	3,979	3,955	3,936	3,920	3,896	3,870	3,844	3,831	3,817	3,804	3,790	3,775	3,761
5	4,060	3,780	3,619	3,520	3,453	3,405	3,368	3,339	3,316	3,297	3,268	3,238	3,207	3,191	3,174	3,157	3,140	3,123	3,105
6	3,776	3,463	3,289	3,181	3,108	3,055	3,014	2,983	2,958	2,937	2,905	2,871	2,836	2,818	2,800	2,781	2,762	2,742	2,722
7	3,589	3,257	3,074	2,961	2,883	2,827	2,785	2,752	2,725	2,703	2,668	2,632	2,595	2,575	2,555	2,535	2,514	2,493	2,471
8	3,458	3,113	2,924	2,806	2,726	2,668	2,624	2,589	2,561	2,538	2,502	2,464	2,425	2,404	2,383	2,361	2,339	2,316	2,293
9	3,360	3,006	2,813	2,693	2,611	2,551	2,505	2,469	2,440	2,416	2,379	2,340	2,298	2,277	2,255	2,232	2,208	2,184	2,159
10	3,285	2,924	2,728	2,605	2,522	2,461	2,414	2,377	2,347	2,323	2,284	2,244	2,201	2,178	2,155	2,132	2,107	2,082	2,055
11	3,225	2,860	2,660	2,536	2,451	2,389	2,342	2,304	2,274	2,248	2,209	2,167	2,123	2,100	2,076	2,052	2,026	2,000	1,972
12	3,177	2,807	2,606	2,480	2,394	2,331	2,283	2,245	2,214	2,188	2,147	2,105	2,060	2,036	2,011	1,986	1,960	1,932	1,904
13	3,136	2,763	2,560	2,434	2,347	2,283	2,234	2,195	2,164	2,138	2,097	2,053	2,007	1,983	1,958	1,931	1,904	1,876	1,846
14	3,102	2,726	2,522	2,395	2,307	2,243	2,193	2,154	2,122	2,095	2,054	2,010	1,962	1,938	1,912	1,885	1,857	1,828	1,797
15	3,073	2,695	2,490	2,361	2,273	2,208	2,158	2,119	2,086	2,059	2,017	1,972	1,924	1,899	1,873	1,845	1,817	1,787	1,755
16	3,048	2,668	2,462	2,333	2,244	2,178	2,128	2,088	2,055	2,028	1,985	1,940	1,891	1,866	1,839	1,811	1,782	1,751	1,718
17	3,026	2,645	2,437	2,308	2,218	2,152	2,102	2,061	2,028	2,001	1,958	1,912	1,862	1,836	1,809	1,781	1,751	1,719	1,686
18	3,007	2,624	2,416	2,286	2,196	2,130	2,079	2,038	2,005	1,977	1,933	1,887	1,837	1,810	1,783	1,754	1,723	1,691	1,657
19	2,990	2,606	2,397	2,266	2,176	2,109	2,058	2,017	1,984	1,956	1,912	1,865	1,814	1,787	1,759	1,730	1,699	1,666	1,631
20	2,975	2,589	2,380	2,249	2,158	2,091	2,040	1,999	1,965	1,937	1,892	1,845	1,794	1,767	1,738	1,708	1,677	1,643	1,607
21	2,961	2,575	2,365	2,233	2,142	2,075	2,023	1,982	1,948	1,920	1,875	1,827	1,776	1,748	1,719	1,689	1,657	1,623	1,586
20	2,949	2,561	2,351	2,219	2,128	2,061	2,008	1,967	1,933	1,904	1,859	1,811	1,759	1,731	1,702	1,671	1,639	1,604	1,567
23	2,937	2,549	2,339	2,207	2,115	2,047	1,995	1,953	1,919	1,890	1,845	1,796	1,744	1,716	1,686	1,655	1,622	1,587	1,549
24	2,927	2,538	2,327	2,195	2,103	2,035	1,983	1,941	1,906	1,877	1,832	1,783	1,730	1,702	1,672	1,641	1,607	1,571	1,533
25	2,918	2,528	2,317	2,184	2,092	2,024	1,971	1,929	1,895	1,866	1,820	1,771	1,718	1,689	1,659	1,627	1,593	1,557	1,518
26	2,909	2,519	2,307	2,174	2,082	2,014	1,961	1,919	1,884	1,855	1,809	1,760	1,706	1,677	1,647	1,615	1,581	1,544	1,504
27	2,901	2,511	2,299	2,165	2,073	2,005	1,952	1,909	1,874	1,845	1,799	1,749	1,695	1,666	1,636	1,603	1,569	1,531	1,491
28	2,894	2,503	2,291	2,157	2,064	1,996	1,943	1,900	1,865	1,836	1,790	1,740	1,685	1,656	1,625	1,593	1,558	1,520	1,478
29	2,887	2,495	2,283	2,149	2,057	1,988	1,935	1,892	1,857	1,827	1,781	1,731	1,676	1,647	1,616	1,583	1,547	1,509	1,467
30	2,881	2,489	2,276	2,142	2,049	1,980	1,927	1,884	1,849	1,819	1,773	1,722	1,667	1,638	1,606	1,573	1,538	1,499	1,456
40	2,835	2,440	2,226	2,091	1,997	1,927	1,873	1,829	1,793	1,763	1,715	1,662	1,605	1,574	1,541	1,506	1,467	1,425	1,377
60	2,791	2,393	2,177	2,041	1,946	1,875	1,819	1,775	1,738	1,707	1,657	1,603	1,543	1,511	1,476	1,437	1,395	1,348	1,291
120	2,748	2,347	2,130	1,992	1,896	1,824	1,767	1,722	1,684	1,652	1,601	1,545	1,482	1,447	1,409	1,368	1,320	1,265	1,193
1000+	2,706	2,303	2,084	1,945	1,847	1,774	1,717	1,670	1,632	1,599	1,546	1,487	1,421	1,383	1,342	1,295	1,240	1,169	1,000

Table 7-10

Critical F values, F _crit95 , versus N – 1 and N _{ref – 1} at 95 per cent confidence

N – 1	1	2	3	4	5	6	7	8	9	10	12	15	20	24	30	40	60	120	1000+
N _ref – 1
1	161,4	199,5	215,7	224,5	230,1	233,9	236,7	238,8	240,5	241,8	243,9	245,9	248,0	249,0	250,1	251,1	252,2	253,2	254,3
2	18,51	19,00	19,16	19,24	19,29	19,33	19,35	19,37	19,38	19,39	19,41	19,42	19,44	19,45	19,46	19,47	19,47	19,48	19,49
3	10,12	9,552	9,277	9,117	9,014	8,941	8,887	8,845	8,812	8,786	8,745	8,703	8,660	8,639	8,617	8,594	8,572	8,549	8,526
4	7,709	6,944	6,591	6,388	6,256	6,163	6,094	6,041	5,999	5,964	5,912	5,858	5,803	5,774	5,746	5,717	5,688	5,658	5,628
5	6,608	5,786	5,410	5,192	5,050	4,950	4,876	4,818	4,773	4,735	4,678	4,619	4,558	4,527	4,496	4,464	4,431	4,399	4,365
6	5,987	5,143	4,757	4,534	4,387	4,284	4,207	4,147	4,099	4,060	4,000	3,938	3,874	3,842	3,808	3,774	3,740	3,705	3,669
7	5,591	4,737	4,347	4,120	3,972	3,866	3,787	3,726	3,677	3,637	3,575	3,511	3,445	3,411	3,376	3,340	3,304	3,267	3,230
8	5,318	4,459	4,066	3,838	3,688	3,581	3,501	3,438	3,388	3,347	3,284	3,218	3,150	3,115	3,079	3,043	3,005	2,967	2,928
9	5,117	4,257	3,863	3,633	3,482	3,374	3,293	3,230	3,179	3,137	3,073	3,006	2,937	2,901	2,864	2,826	2,787	2,748	2,707
10	4,965	4,103	3,708	3,478	3,326	3,217	3,136	3,072	3,020	2,978	2,913	2,845	2,774	2,737	2,700	2,661	2,621	2,580	2,538
11	4,844	3,982	3,587	3,357	3,204	3,095	3,012	2,948	2,896	2,854	2,788	2,719	2,646	2,609	2,571	2,531	2,490	2,448	2,405
12	4,747	3,885	3,490	3,259	3,106	2,996	2,913	2,849	2,796	2,753	2,687	2,617	2,544	2,506	2,466	2,426	2,384	2,341	2,296
13	4,667	3,806	3,411	3,179	3,025	2,915	2,832	2,767	2,714	2,671	2,604	2,533	2,459	2,420	2,380	2,339	2,297	2,252	2,206
14	4,600	3,739	3,344	3,112	2,958	2,848	2,764	2,699	2,646	2,602	2,534	2,463	2,388	2,349	2,308	2,266	2,223	2,178	2,131
15	4,543	3,682	3,287	3,056	2,901	2,791	2,707	2,641	2,588	2,544	2,475	2,403	2,328	2,288	2,247	2,204	2,160	2,114	2,066
16	4,494	3,634	3,239	3,007	2,852	2,741	2,657	2,591	2,538	2,494	2,425	2,352	2,276	2,235	2,194	2,151	2,106	2,059	2,010
17	4,451	3,592	3,197	2,965	2,810	2,699	2,614	2,548	2,494	2,450	2,381	2,308	2,230	2,190	2,148	2,104	2,058	2,011	1,960
18	4,414	3,555	3,160	2,928	2,773	2,661	2,577	2,510	2,456	2,412	2,342	2,269	2,191	2,150	2,107	2,063	2,017	1,968	1,917
19	4,381	3,522	3,127	2,895	2,740	2,628	2,544	2,477	2,423	2,378	2,308	2,234	2,156	2,114	2,071	2,026	1,980	1,930	1,878
20	4,351	3,493	3,098	2,866	2,711	2,599	2,514	2,447	2,393	2,348	2,278	2,203	2,124	2,083	2,039	1,994	1,946	1,896	1,843
21	4,325	3,467	3,073	2,840	2,685	2,573	2,488	2,421	2,366	2,321	2,250	2,176	2,096	2,054	2,010	1,965	1,917	1,866	1,812
22	4,301	3,443	3,049	2,817	2,661	2,549	2,464	2,397	2,342	2,297	2,226	2,151	2,071	2,028	1,984	1,938	1,889	1,838	1,783
23	4,279	3,422	3,028	2,796	2,640	2,528	2,442	2,375	2,320	2,275	2,204	2,128	2,048	2,005	1,961	1,914	1,865	1,813	1,757
24	4,260	3,403	3,009	2,776	2,621	2,508	2,423	2,355	2,300	2,255	2,183	2,108	2,027	1,984	1,939	1,892	1,842	1,790	1,733
25	4,242	3,385	2,991	2,759	2,603	2,490	2,405	2,337	2,282	2,237	2,165	2,089	2,008	1,964	1,919	1,872	1,822	1,768	1,711
26	4,225	3,369	2,975	2,743	2,587	2,474	2,388	2,321	2,266	2,220	2,148	2,072	1,990	1,946	1,901	1,853	1,803	1,749	1,691
27	4,210	3,354	2,960	2,728	2,572	2,459	2,373	2,305	2,250	2,204	2,132	2,056	1,974	1,930	1,884	1,836	1,785	1,731	1,672
28	4,196	3,340	2,947	2,714	2,558	2,445	2,359	2,291	2,236	2,190	2,118	2,041	1,959	1,915	1,869	1,820	1,769	1,714	1,654
29	4,183	3,328	2,934	2,701	2,545	2,432	2,346	2,278	2,223	2,177	2,105	2,028	1,945	1,901	1,854	1,806	1,754	1,698	1,638
30	4,171	3,316	2,922	2,690	2,534	2,421	2,334	2,266	2,211	2,165	2,092	2,015	1,932	1,887	1,841	1,792	1,740	1,684	1,622
40	4,085	3,232	2,839	2,606	2,450	2,336	2,249	2,180	2,124	2,077	2,004	1,925	1,839	1,793	1,744	1,693	1,637	1,577	1,509
60	4,001	3,150	2,758	2,525	2,368	2,254	2,167	2,097	2,040	1,993	1,917	1,836	1,748	1,700	1,649	1,594	1,534	1,467	1,389
120	3,920	3,072	2,680	2,447	2,290	2,175	2,087	2,016	1,959	1,911	1,834	1,751	1,659	1,608	1,554	1,495	1,429	1,352	1,254
1000+	3,842	2,996	2,605	2,372	2,214	2,099	2,010	1,938	1,880	1,831	1,752	1,666	1,571	1,517	1,459	1,394	1,318	1,221	1,000

The least-squares regression slope, a _1y, shall be calculated by means of equation (7-162):

a_{1 y} = \frac{\sum_{i = 1}^{N} (y_{i} - \bar{y}) \times (y_{ref i} - {\bar{y}}_{ref})}{\sum_{i = 1}^{N} {(y_{ref i} - {\bar{y}}_{ref})}^{2}}

(7-162)

The least-squares regression intercept, a _0y, shall be calculated by means of equation (7-163):

a_{0 y} = \bar{y} - (a_{1 y} \times {\bar{y}}_{ref})

(7-163)

The standard estimate of error, SEE, shall be calculated by means of equation (7-164):

{SEE}_{y} = \sqrt{\frac{\sum_{i = 1}^{N} {[y_{i} - a_{0 y} - (a_{1 y} \times y_{ref i})]}^{2}}{N - 2}}

(7-164)

The coefficient of determination, r ², shall be calculated by means of equation (7-165):

r_{2}^{y} = 1 - \frac{\sum_{i = 1}^{N} {[y_{i} - a_{0 y} - (a_{1 y} \times y_{ref i})]}^{2}}{\sum_{i = 1}^{N} {[y_{i} - \bar{y}]}^{2}}

(7-165)

Annotations

Textual amendments