Counting Statistics (radioactive contaminated land sample size) widget

Counting statistics (radioactive contaminated land sample size) widget

Dr Chris Robbins, Grallator

Is the mean activity of an area of land above a given threshold?

Suppose you have an area of land that has a possibility of being contaminated at some level by radionuclides. A regulatory threshold limit, \(T_r\), will be applied to the mean value of the activity over the given area. If the mean value is below the threshold no remedial action is required. If it is above the threshold the land will need remediation such as removal of material to a suitable repository site. There are two questions to ask:

• How big a survey sample size is required to be reasonably sure the mean site activity is below the threshold?
   
• Where should the measurements that make up the sample be taken?

Some notes:

• Sample is used here in the statistical sense of a set of values in a single survey. Physically, at each measurement location a soil sample will be taken for analysis. Each soil sample provides a measurement in the survey sample. Do not confuse the two uses of the word sample!
   
• The phase above "reasonably sure" highlights the fact that when you take a finite number of measurements the mean value you obtain is an estimate for the actual mean and will be subject to some uncertainty. It would also be fair to assume that more measurements should mean a better estimate.
   
• The values measured at different parts of a site will generally form a population of values which will have an overall mean (the site mean we're trying to determine) and a spread of values due to contamination heterogeneity in the land and also to some extent uncertainty on the lab measurement methods used. This spread of values is captured by the standard deviation of the population denoted \(\sigma\).

To answer the above questions requires a little bit of background in statistics, starting with the central limit theorem.

Estimating the mean and the central limit theorem

The central limit theorem (CLT) is a very important cornerstone of statistics and probability theory. Suppose we have a probability density function, such as a normal distribution with mean zero (\(\mu = 0\)) and standard deviation 1 (\(\sigma = 1\)), \(N\left( 0,1 \right)\), as shown below.

The above distribution represents perfect knowledge; it is the reality of a situation. However, as a mere mortal the knowledge that the mean is zero (and the standard deviation is one) is not known to me, and must be estimated.

To estimate the mean of the unkonwn distribution, a sample of size \(n\) is drawn with individual values, \(x_1, x_2, \cdots, x_n\). The mean of the sample is simply given by \[ \overline{x}=\frac{x_1 + x_2 + \cdots + x_n}{n} \] Would such a sample give an estimated mean of zero with, say a sample size of 50? Probably not! An example case might produce, for example, \(\overline{x} = -0.149\). A different sample of size 50 might produce \(\overline{x} = 0.06\). These two values are an estimate for the real mean of zero, and they are obviously in error, but is there a way to estimate the error on the mean generated by the sample?

The above has given two sample mean values, what if more samples are taken? The figure below shows the distribution of sample means when 1,000 samples of size \(n=50\) are taken.

The sample means form a distribution centred near the actual mean. Further, it can be shown that the distribution of sample means follows a normal distribution and that the standard deviation of the distribution of sample means, denoted the standard error of the mean, \(SEM\) or \(\sigma_\overline{x}\) is given by \[ SEM = \sigma_\overline{x} = \frac{\sigma}{\sqrt{n}} \] where \(\sigma\) is the standard deviation of the distribution of the population from which values are drawn (\(\sigma = 1\) in this example). This result is a demonstration of the central limit theorem.

Notice that the standard error of the mean scales as the reciprocal of the square root of the sample size. A sample with \(n=100\) will have a standard error that is about 3.16 times smaller than a sample with \(n=10\) even though the sample has ten times the number of measurements. (This relationship was also shown when performing Monte Carlo calculations.)

The remarkable thing about the central limit theorem is that it's true even if the population being sampled isn't a normal distribution and isn't symmetrical about the mean\(^*\). For example, the probability distribution below is right (positive) skewed with a mode of 2 and a mean of \(\mu=4\). The standard deviation of this distribution is \(\sigma = 2 \sqrt{2}=2.828\)

Taking 1000 samples each of size \(n=50\) and plotting a histogram of sample means also gives a normal distribution centred near the actual mean and has a standard deviation of \(SEM = \sigma_\overline{x} = \frac{\sigma}{\sqrt{n}}\), even though the population distribution is skewed.

\(^*\)Caveat

This is true with enough samples each with a large enough n. When populations are skewed and few samples/low values of \(n\) are used the distribution of the sample means can be skewed. If you look at the figure immediately above you could convince yourself that the right tail is a bit longer than the left, but certainly nowhere near the extent as the population from which samples are drawn. In such cases it often helps to transform the values so that the transformed distribution is more symmetrical. A typical transformation is to take the log of the measured values. \[x^*_i = \log x_i\] Treat this transformation with care and respect; the arithmetic mean of the log transformed values when transformed back gives the geometric mean of the original data not the arithmetic mean! Showing this is left as an exercise for the reader...

Is the estimated mean above a threshold?

The above has shown that a sample of data can produce an estimate of the mean of a population and that the estimate has an uncertainty that is distributed normally about the mean estimate and with a standard deviation given by \( \sigma_\overline{x} = \frac{\sigma}{\sqrt{n}}\). The introduction used the phrase "reasonably sure the mean site activity is below the threshold", and we are now in a position to quantify what "reasonably sure" means. For a normal distribution 95% of the area under the curve is to the left of the mean plus 1.645 standard deviations, i.e. 95% probability that the actual value of the mean is less than \(\overline{x}+1.645 \sigma_\overline{x} \).

The complement of the last statement above is that there is a 5% probability that the actual mean is above \(\overline{x}+1.645 \sigma_\overline{x} \). This is our measure of "reasonably sure". If there is a 95% probability that the mean, taking account of the uncertainty on the value, is below the threshold, \(T_r\), then accept that the mean land activity is below the threshold and remedial action is not required. Otherwise accept that the mean land activity is above the threshold and that remediation is required, i.e.

• Below threshold if \(\overline{x}+1.645 \sigma_\overline{x} < T_r\)
   
• Above threshold if \(\overline{x}+1.645 \sigma_\overline{x} \ge T_r\)

Note to clarify, if \(\overline{x}+1.645 \sigma_\overline{x}\) is an infinitesimal amount less than \(T_r\) the land activity is denoted as below the threshold value. There is a 95% probability that this is the right decision. However, there is also a 5% probability that this is the wrong decision and the estimated mean is above the threshold value. To mitigate such situations two changes can be made:

• Use a higher confidence can be used. For example if instead of having 95% confidence of the mean being below a threshold you want a 99% confidence then the test value becomes \(\overline{x}+2.326 \sigma_\overline{x}\).
   
• Increase the sample size \(n\). This works by reducing the size of the uncertainty through \( \sigma_\overline{x} = \frac{\sigma}{\sqrt{n}}\).

Using a higher confidence value may result in more remediation work on land that does not require it, incurring unnecessary cost. Using a larger sample size will require more time and sampling/evaluation costs. The general implementation is often to fix the probability of the limiting actual mean being above the threshold based on a sample at 5% and then work out what sample size is required to achieve this.

The values 1.645 and 2.236 can be found in standard statisitical tables and are usually denoted \(z\) for the normal distribution. To make the test a little more readable, the test limit value will be written as the general form \(\overline{x} + z \sigma_\overline{x}\).

How to determine the sample size?

In the limiting case the threshold is coincident with the value of \(\overline{x} + z \sigma_\overline{x}\).

The value \(z \sigma_\overline{x}\) is sometimes referred to as the "margin of error", i.e. it is a measure for how wide an error on your estimate are you willing to tolerate. Denoting this by \(E\) so that \(E = z \cdot \sigma_\overline{x}\) and using the expression \( \sigma_\overline{x} = \frac{\sigma}{\sqrt{n}}\) gives \[ E = z \cdot \sigma_\overline{x} = \frac{z \cdot \sigma}{\sqrt{n}} \] Rearranging for \(n\) and noting that the sample size has to be integer gives \[ n = \left\lceil \left( \frac{z \cdot \sigma}{E} \right)^2 \right\rceil \] where \(\lceil \cdot \rceil \) means take the next integer value above the enclosed value. Some publications use \(1 + \left[ \cdot \right] \) where \( \left[ \cdot \right] \) means take the integer part of the enclosed value to give an equivalent result.

The next question to ask is what value should be taken for \(E\)? One way would be to scale it as a fraction of the threshold value being used. This has the advantage that the same relative margin error will be used whatever the monitoring situation and threshold value is. \[E=f \cdot T_r\] giving \[ n = \left\lceil \left( \frac{z \cdot \sigma}{f \cdot T_r} \right)^2 \right\rceil \]

Notes

• Increasing the confidence level (increase the value \(z\)) results in a larger sample size.
   
• Decreasing the margin of error (decrease the value \(f\)) results in a larger sample size.

The following table shows how many samples are required at different values of \(f\) for the case when \[ \begin{align} z &= 1.645 \\ \sigma &= 0.1 \\ T_r &= 0.4 \text{ Bq/gm} \end{align} \]

The table above sometimes gives small values for the sample size. In practice a minimum sample size would be specified, for example a minimum sample size of 4 may be sepcified so that the first three results in the table are ignored.

What about the standard deviation of the population?

The above has quietly used the population standard deviation \(\sigma\) under the assumption it is a known value for the site of interest. However, the value probably isn't known. In this case we need to estimate the value and some possible approaches to do this are:

• Use prior or historical data. There may be existing surveys or detailed studies of comparable sites which have a standard deviation characterised.
   
• Use the "range rule of thumb", \(\sigma \approx \frac{\text{max measured value - min measured value}}{4}\). This is derived from the fact that in a normal distribution approximately 95% of the values will be within (approximately) \(\pm 2\) standard deviations of the mean.
   
• Use sample values.

The last point above estimates the population standard deviation from the sample standard deviation as: \[ \sigma \approx s = \sqrt{ \frac{\sum_{i=1}^n {\left( \overline{x} - x_i\right)}^2}{n - 1} } \] The obvious problem point here is that we're using \(s\) to estimate the sample size through \[n = \left\lceil \left( \frac{z \cdot s}{f \cdot T_r} \right)^2 \right\rceil \tag{1}\label{1} \] when s itself depends on the value of \(n\) we're trying to determine!

To avoid the circular logic of the above, it is often the case that the site population standard deviation, \(s\), is estimated using an initial survey sample with a limited sample size. This value can be used in \(\eqref{1}\) to determine what sample size would be predicted with the value of \(s\). If the value is smaller than the investigation sample size it probably indicates that the investigation sample is sufficient for purposes. However, in many cases expression \(\eqref{1}\) will determine that a larger sample size is required.

Consequences of estimating the standard deviation

When the standard deviation is estimated it is more appropriate to use Student's t-distribution rather than the normal distribution to determine the sample size. In this case \(\eqref{1}\) becomes \[ n = \left\lceil \left( \frac{t \cdot s}{f \cdot T_r} \right)^2 \right\rceil \tag{2}\label{2} \] where a \(t\) value is used rather than a \(z\) value. The \(t\) value depends on the confidence level required, e.g. 95%, 99% etc., and also on the so-called degrees of freedom, \(\text{df}\) which, in this example is the sample size minus 1. Note, this is the sample size of the survey sample which will be denoted \(n_I\) to differentiate it from \(n\) the full investigation sample size. Using this notation \(\text{df} = n_I -1\). Note, when a full investigation sample is taken, and assuming the investigation sample size is larger than the survey sample size, the standard deviation can be refined by using the values from the investigation sample. In this case \(\text{df} = n -1\).

The table below shows one-sided \(t\) values for various values of \(\text{df}\) and confidence levels. Note that as the number of degrees of freedom increases the t value approaches the \(z\) value.

The \(t\) values are larger than the equivalent \(z\) values which is a reflection of the fact that knowledge of the population is imperfect.

Some calculations

A calculator/simulator (Ionactive widget) has been produced to help demonstrate some of the features and pitfalls of investigating an area of land for contamination. Note, as with the text above, this is simplified to a representative problem which is intended to show general principles rather than something that can give a full authorative basis for an actual site characterisation. The basic set out is as shown below.

The sliders at the top allow the user to change the threshold activity limit \(T_r\) and the fraction of the limit to be used as to calculate the "margin of error" used when estimating an investigation sample size. In the initial set up the limit is \(T_r=0.2\) and a fraction of \(f=0.01\), i.e. the margin of error is 1% of \(T_r\).

The box on the right allows the user to perform a survey to estimate the site standard devaition from which the sample size for investigation is estimated using the equations above. This box also gives the survey mean for information.

The box on the lower left will show how the sample test locations are chosen - more on this later.

The actual dstribution from which samples values are taken can be seen and modified by clicking on IonActive man or his speech bubble. For illustrative purposes set the mean activity to be 0.39 Bq/gm and the standard deviation to be 0.04. While the window is open, drag the threshold activity slider to be 0.4 Bq/gm, i.e. just above the actual mean value.

Click the cross in the upper right of the distribution window to closes it and drag the survey sample size slider to be 10. The area at the lower left will indicate a sample pattern which is either based on a rectangular pattern or a herringbone pattern.

The following points are noted:

• Points may be selected by using human judgement. However, this is generally avoided as it can introduce bias into the sample.
   
• Points may be selected at random. However, purely random locations may result in uneven sampling densities, for example, there may be lots of points in the upper left region with few or non in the lower right.
   
• To give a uniform sampling densities points are chosen on a regular grid. To avoid bias, the starting point for constructing the grid is chosen at random.

The random starting point of the last item can be seen by repeatedly pressing either the "rectangular" or "herringbone" button. Each time it is pressed the pattern will move around the area.

It is also noted that some points are coloured red while others are not. The reason for this is that the simulator attempts to fill the area with a regular grid that gives a uniform sampling density in both directions. Where this cannot be done with the sample size specified the points that are "left out" are highlighted. In this case you should think about changing the spacing or changing the number of points. Methods for the former are not covered here but see, for example, centroidal Voronoi tessellation (CVT).

Note, the pattern shown is for illustration only - there is no underlaying spatial distribution in the simulation.

Doing the survey

The survey box simulates the situation where the standard deviation of the contamination distribution is not known and a survey sample is used to estimate the value. Move, the margin of error fraction to 0.05 (i.e. the margin of error will be 5% of \(T_r\)) and click the "Do survey" button. A typical result may look as below.

The above has estimated from the survey sample that the distribution mean is 0.398 and the standard deviation is 0.050, which may be compared with the actual values, set above, of 0.390 and 0.040 respectively. Using the estimated standard deviation yields a suggested investigation sample size of 18. Note, this value uses a 95% z value. Clicking the "Use t value" button changes the estimate to a sample size of 22, indicating the more conservative nature of using a t-distribution.

Go back to using the z value for the time being and repeat the "Do survey". You will invariably get a different estimate for the investigation sample size.

The image on the left of the slider shows that when the sample happens to have 10 values that [randomly] happen to be fairly close together, the standard deviation is underestimated which results in a lower investigation sample size. In this case a value of 8 is given and, as this is smaller than the survey sample size, a warning is issued - why do another sample of size 8 when you've already got one of size 10?

The image on the right of the slider shows that the sample values happen to be well spread and so it overestimates standard deviation and hence predicts a larger investigation sample size is required.

Knowing the actual standard deviation of \(\sigma = 0.04\) gives the "right" answer for the investigation sample size as \[ n = \left\lceil \left( \frac{z \cdot s}{f \cdot T_r} \right)^2 \right\rceil = \left\lceil \left( \frac{1.645 \cdot 0.04}{0.05 \cdot 0.4} \right)^2 \right\rceil = 11 \]

The above exercise highlights the first consideration when taking a survey which results in values being randomly drawn from a distribution. The result you get depends on the values you happen to measure. By changing the survay sample size, and experimenting with multiple samples at each value of the sample size, the following should become evident:

• When the sample size is small, that variability of the standard deviation and hence suggested investigation sample size is (generally) larger.
   
• When the sample size is large, that variability of the standard deviation and hence suggested investigation sample size is (generally) smaller.

The lesson from this is that a larger survey sample size gives a more accurate estimate of the investigation sample size. However, depending on circumstances, the suggested sample size may be smaller than the survey sample size. In this case use the survey as the investigation, but you may find out after the survey that you could have used a smaller sample! The word "generally" is added to the above as any random process can (and does) occasionally throw up a bad set of values.

Changing the margin of error fraction

The margin of error fraction is used to specify how narrowly you want to be able to estimate the mean so that it can be compared with the threshold limit (more on this below). Recall the margin of error is related to the sample size through \(E = z \cdot \sigma_\overline{x} = \frac{z \cdot \sigma}{\sqrt{n}} \). A large margin of error allows for a smaller sample size, while a small margin of error requires a larger sample size, as shown for \(f = 0.1\) and \(f=0.01\) below (where a survey sample size of 30 is used to reduce the variability of the suggested investigation sample size due to random variation in the sample values). Note, the sample size is under a square root in the margin of error expression which means that changes in the margin of error square the outcome; 0.01 is \(\frac{1}{10}\) the value of 0.1 which means the corresponding value of \(n\) is approximataly 100 times larger.

We will return to the margin of error later.

Doing the investigation

After the survey we move on to the investigation to determine whether the land should be considered to have a mean activity above the given threshold or not. The starting point assumed is the following survey results.

In this case, the suggested investigation sample size is larger than the survey sample size (admittedly not by a lot, but this is only for demonstration purposes!). For the case where the suggested investigation sample size is equal to or smaller than the survey sample, the survey sample is used.

Click the "Investigate mode" button to switch to analysing the mean activity determination. Note, the values for the mean and standard deviation will change as a new sample of the appropriate size will be assumed (this does not affect the generality when the survey sample is used as the investigation sample).

The sample mean and sample standard deviation are given, along with the constructed standard error of the mean, \( \sigma_\overline{x} = \frac{\sigma}{\sqrt{n}}\) and the limiting upper value for determing whether the land should be considered as contaminated above the threshold, \(\overline{x}+1.645 \sigma_\overline{x} \). Note, that if t values are used, the 1.645 will be substituted by the appropriate t value.

In the figure above the limiting upper value is above the threshold limit and the land is deemed to be contaminated. 

To get a clearer picture of the assessment click the "Show mean plot" button to give, for example:

This plot shows the estimated mean along with the [normal] error distribution associated with the estimate. The green shaded area represents 95% of the distribution and the real mean should be within this region 95% of the time. The upper limiting upper value, shown as 0.4044 is the point at which the actual mean has a 5% chance of exceeding. In this case this value is above the assessment threshold of \(T_r = 0.4\) and the land is deemed to be contaminated.

Recall, from the previous set up we know the actual mean value is 0.39 and so the land shouldn't actually be marked as contaminated (but also note the actual mean is very close to the threshold value). This is a type 1 error (a false positive). This is partly a consequence of using the upper limit on the estimate of the mean which is designed to reduce the probability of a type 2 error (a false negative) under the assumption that it is better to err on the side of caution.

Click the "Do investigation" button several times and observe the outcome. More often than not the result will be that the land is deemed to be contaminated with the occasional result of it being [correctly] deemed to be uncontaminated.

The reason why the outcome is invariably contaminated is that the study has been set up with a margin of error fraction that is 5% of \(T_r = 0.02\). While this reduces the sample size it does mean that in situations where the actual mean is close to the threshold, you cannot resolve the difference. To change things, click on "Survey mode" set the margin of error fraction to 0.01 and click the "Do survey" button to get a new investigation sample size.

Now return to "Investigate mode" and show the mean plot

In this case the plot shows that the threshold value is off to the right of the screen and not visible. In the previous case the standard error of the mean was 0.011 while in the above the standard error of the mean is 0.002. This narrows the uncertainty on the estimate of the mean so that the study can now more clearly resolve the difference in the estimated mean and the threshold value.

Repeatedly pressing "Do investigation" will show that the majority of the time the mean will be correctly identified as being below the limit. But note, this increase in resolving power comes at the expense of a large sample size.

False negatives

Return to "Survey mode" and set the margin of error fraction to 0.05.

Click IonActive man and change the actual distribution mean to be 0.41 and 0.050.

Close the distribution and click "Do survey" to get a new estimate of an investigation sample size, in the following example 14 is calculated.

Go back in to "Investigate mode" and show the mean plot. As the actual mean is slightly above the threshold it is expected that the upper limit for the estimated mean will be above the threshold and that the land will be deemed to be contaminated, as shown in the example below.

Click the "Do investigation" button to change the sample. Most of the time the land will be deemed contaminated. However on rare occasions the land will be deemed to be uncontaminated, i.e. a false negative as shown below.

To minimise the risk of false positives you can go back to "Survey mode" and reduce the margin of error fraction which will result in a larger sample size being required for the investigation.

In the example above, the margin of error fraction has been reduced to 0.01 and the survey sample size set to 20 to yield a suggested investigation sample size of 320 (based on the values drawn in that particular survey). In "investigation mode" it will be seen that the \(T_r\) marker lies to the left of the upper limiting value of the estimated mean virtually all the time. (Press "Do investigate" enough times and you may find a false positive, but it may take a lot of clicking!)