Statistical treatment of data 
The Statistical Treatment of Indeterminate Errors
Statistics is the mathematical science that deals with chance variations. We must emphasize at the outset that statistics only reveals information that is already present in a data set. That is, no new information is created by statistics. Statistical treatment of a data set does, however, allow us to make objective judgements concerning the validity of results that are otherwise difficult to make.
Population Mean (µ) and the Sample Mean ()
When N is small
When N ®¥
When N is small, sample mean does not represent population mean because a small sample of data does not exactly represent its population.
In the absence of determinate error, the population mean µ is the true value of a measured quantity.
Sample Standard Deviation (s) and the Population Standard Deviation (s )
When N is small
When N ®¥
(this is true standard deviation)
When N®¥ , ® µ, and s ®s . When N is greater than about 20, s and s can be assumed to be identical for all practical purposes.
Uncertainty in the calculated value of s decreases as N increases as shown inn the following figure:
Therefore do as many repetitions as possible as the time, equipment and sample resources allow.
Properties of Normal Error Curve
Error Curves can be drawn in two ways. One is the frequency of occurrence Vs (x  µ) (part a in the following figure) and the frequency of occurrence versus which turns out to be 1s, 2s, 3s…
The normal error curve, which is a Gaussian curve, has several general properties.
It can be shown that 68.3% of the area beneath any normal curve lies within one standard deviation (±1s ) of the mean µ, meaning 68.3% of the data making up the population lie within these bounds. Furthermore,
95.5 % m ± 2s
99.7 % m ± 3s
So one can say that the chances are 68.3 in 100 that the indeterminate uncertainty of any single measurement in a normal distribution is no more than ±1s. Similarly, the chances are 95.5 in 100 that the error is less than ±2s, and so forth.
The Uses of Statistics
Experimentalists use statistical calculations to sharpen their judgement concerning the effects of indeterminate errors. The most common applications are:
Confidence Limits
The exact value of the mean µ for a population of data can never be determined exactly because such a determination requires an infinite number of measurements. Statistical theory allows us to set limits around an experimentally determined mean, , however, and the true mean µ lies within these limits with a given degree of probability. These limits are called confidence limits, and the interval they define is known as the confidence interval. In other words, confidence limits define an interval around that probably contains µ.
Confidence Interval when s is close to s
Consider the following figure:
So for the first plot, the confidence level is 50% and the confidence interval is ±0.67s meaning that the probability of existence of the true value in between ( 0.67s) to ( 0.67s) is 50%. So, the confidence interval is a function of confidence level which in turn is a function of z:
So, the general expression for the confidence limits (CL) of a single measurement can easily be shown to be:
CL for µ = x ± zs
Values of z is fixed for differet confidence levels and listed in the following table:
CONFIDENCE
LEVELS FOR VARIOUS VALUES OF z 

Confidence Levels, % z  
50 68 80 90 95 96 99 99.7 99.9 
0.67 
1.00 

1.29 

1.64 

1.96 

2.00 

2.58 

3.00 

3.29 
Example: Calculate the 95% confidence limits of the measurement (11.24 cm) for a set of measurements where s ® s = 0.07 cm.
95% CL for m = 11.24 ± 1.96 x 0.07 = 11.24 ± 0.14
For the confidence limits of the mean (average) value, use the following equation:
CL for
Example: How many replicate measurements of specimen in the previous example should be done to decrease the 95% confidence interval to 0.01s
Using above equation,
Confidence interval = = 0.01 =
N is, then, calculated as 4.
Confidence Limits When s is unknown:
If s is known but s is unknown, use the equation:
CL for
Use the following table for the values of t (t is often called student's t)
VALUES OF t FOR VARIOUS LEVELS OF PROBABILITY  
Number of Observations  Factor for Confidence Interval  
80%  90%  95%  99%  99.9%  
1 
3.08  6.31  12.7  63.7  637 
2 
1.89  2.92  4.30  9.92  31.6 
3 
1.64  2.35  3.18  5.84  12.9 
4 
1.53  2.13  2.78  4.60  8.60 
5 
1.48  2.02  2.57  4.03  6.86 
6 
1.44  1.94  2.45  3.71  5.96 
7 
1.42  1.90  2.36  3.50  5.40 
8 
1.40  1.86  2.31  3.36  5.04 
9 
1.38  1.83  2.26  3.25  4.78 
10 
1.37  1.81  2.23  3.17  4.59 
11 
1.36  1.80  2.20  3.11  4.44 
12 
1.36  1.78  2.18  3.06  4.32 
13 
1.35  1.77  2.16  3.01  4.22 
14 
1.34  1.76  2.14  2.98  4.14 
¥ 
1.29  1.64  1.96  2.58  3.29 
Please note that as N ® ¥ , t ® z.
Rejection of outliers
When a set of data contains an outlying result (possibly out of a gross error) that appears to differ excessively from the average, a decision must be made whether to retain or reject the result. It is not easy and there is no universal rule that can be invoked to settle the question of retention or rejection.
Statistical Tests.
All tests assume that the distribution of the population data is normal, or gaussian. Unfortunately, this condition cannot be proven, or disproved for samples consisting of many fewer than 50 results. Therefore, statistical rules should be used with extreme caution when applied to samples containing only a few data. Thus, statistical tests for rejection should be used only as aids to common sense when small samples are involved.
The Q Test
It is the simplest test.
In this test, the absolute value of the difference between the questionable result x_{q} and its nearest neighbour x_{i} is divided by the spread (range) of the entire set to give the quantity Q_{exp}:
Q_{exp} = ú x_{q}  x_{n}ú / w
This ratio is then compared with rejection values Q_{crit} found in the following table:
CRITICAL VALUES FOR REJECTION QUOTIENT Q  
Q_{crit }(Reject if Q_{exp} > Q_{crit})  
Number of Observations 
90% Confidence 
95% Confidence 
99% Confidence 

3 
0.941 
0.970 
0.994 

4 
0.765 
0.829 
0.926 

5 
0.642 
0.710 
0.821 

6 
0.560 
0.625 
0.740 

7 
0.507 
0.568 
0.680 

8 
0.468 
0.526 
0.634 

9 
0.437 
0.493 
0.598 

10 
0.412 
0.466 
0.568 
If Q_{exp }is greater than Q_{crit}, the questionable result can be rejected with the indicated degree of confidence.
Example:The measurement of the density of a certain mineral yielded 3.456, 3.451, 3.475, and 3.452 g/cm^{3}. 3.477 g/cm^{3} appears to be anomalous; should be retained or rejected at the 90% confidence level?
Q_{exp} = ú 3.475  3.456ú / (3.475  3.451) = 0.792
At 90% confidence level, with 4 measurements Q_{crit }= 0.765 which is smaller than Q_{exp}= 0.792. Therefore the questionable data 3.475 can be rejected at 90% confidence level. But it can be retained at 95% confidence level since Q_{exp} (0.765)< Q_{crit }(0.829)
The T_{n} Test
In the Amerikan Society for Testing Materials (ASTM) T_{n} serves as the rejection criterion, where
Here, x_{q} is the questionable result, and and s are the mean and standard deviations of the entire set including the questionable result. Rejection is indicated if the calculated T_{n} is greater than the critical values found in the following table:
CRITICAL VALUES FOR REJECTION QUOTIENT T_{n}  
Number
of Observations 
T_{nc} 

95% Confidence  97.5% Confid.  95% Confidence  
3 
1.15 
1.15 
1.15 

4 
1.46 
1.48 
1.49 

5 
1.67 
1.71 
1.75 

6 
1.82 
1.89 
1.94 

7 
1.94 
2.02 
2.10 

8 
2.03 
2.13 
2.22 

9 
2.11 
2.21 
2.52 

10 
2.18 
2.29 
2.41 
Example: The analysis of a calcite sample yielded CaO percentages of 55.95, 56.00, 56.04, 56.08, and 56.23. Should the last data retained or rejected at 95% confidence level?
The standard deviation s can be calculated as s = 0.107. The mean value, = 56.06.
Thus = 1.59
Since T_{n} (1.59) is smaller than T_{nc} (1.67), 56.23 is retained as an acceptable data.
Recommendations for the Treatment of Outliers
Recommendations for the treatment of a small set of results that contains a suspect value follow:
The blind application of statistical tests to retain or reject a suspect measurement in a small set of data is likely to be much more fruitful than an arbitrary decision. The application of good judgement based on broad experience with an analytical method is usually a sounder approach. In the end, the only valid reason for rejecting a result from a small set of data is the sure knowledge that a mistake was made in the measurement process. Without this knowledge, a cautious approach to rejection of an outlier is wise.