| I'm going to answer your second question first about the proper
"average" variance to use.
Brief answer: use the sum of the variances divided by the n. You can
take the square root of that and get an "average" standard deviation,
if you wish.
Explanation: If you have a bunch of ordinary variables x1, x2, ..., xn
if you want to take the "average" you add them together and then you
"rescale" them to the same units as the original by dividing by n.
What you end up with is something that is a kind of rough stand in
for the seperate values, which we will call x*.
Now imagine that you do the same thing, over and over again. Each time
you take the average. Each set of variables xi is a sample. There
will be a distribution of values that each of the xi will take (perhaps
each of the xi will have the same distribution of values, perhaps not).
In other words, we are now talking about a set of *random* variables
which we will call X1, X2, ..., XN. The x* (the average) produced with
each sample, also will have a distribution associated with it (which
can, in principle at least be derived from the distributions of the
Xi), and is therefore "really" a random variable X*. The mean (pretend
that doesn't mean the same thing as "average") and the
variance/standard deviation are descriptors of the distributions (one
being an indicator of the "location" of the distribution and the others
being an indicator of the "scale" of the distribution). We want to
know what the mean and the variance of X* are.
Let's add one more random variable, X+, which is the sum of the Xi's.
X* is then X+/n. What is the mean of X*? Well the mean is a "linear
operator". That has two important consequences:
1) The mean of the sum of random variables is the sum of their
means. ( E(A+B) = E(A) + E(B) ).
2) The mean of a random variable divided by a constant is the
same as dividing the mean of the random variable by the
constant. ( E(A/c) = E(A)/c )
So the mean of X+ is the sum of the means of the Xi, and the mean of
X* (= X+/n) is the mean-of-X+ divided by n. This is what justifies
the procedure of finding the "average" mean by taking the average of
the means.
What about the *variance* of our average value (X*)? Well, it turns out
that the variance is *also* linear (actually not surprising when you
remember that the variance is a mean: its the mean of the squared
deviations from the overall mean), so the same reasoning applies:
the variance of the average is the average of the variances.
The standard deviation is *not* linear, however. Since it does
represent a "scale" you can multiply it by some scalar and get something
meaningful, but you should never add standard deviations to each other,
it doesn't mean anything.
Note that since the "average" is the same thing as the "mean", the above
says that you will get the same result however you divide up the
samples into subsamples and take the averages of the averages. You
will get the mean and the variance of the whole thing.
I'll try to address your other problem as soon as I have a moment.
Topher
|
| RE: .4
Different purposes. The number concerned represents the number of
"false positives" you are willing to tolerate.
In conventional hypothesis testing the false positive is false
rejection of the null hypothesis. That is, you will tentatively
conclude -- (tentatively because if it is important enough there will
presumably be at least one replication, and if it is less important
it will still probably lead to later contradictions which will cause
it to be challenged -- only if it is completely unimportant will it
not be re-examined) -- that something "interesting" is going on.
The cost of a false positive is relatively low -- when it is not,
stricter criteria are used (e.g., the second standard of .01). A high
alpha means fewer false negatives -- you are less likely to miss
something interesting.
In SPC if you get a "positive" then you throw away the batch, shut
down the line for inspection, send everyone to the bomb shelters, order
additional, costly tests or whatever. A false positive is expensive
and you cannot tolerate as high a level as you can in research.
Therefore you set it low -- and
I'm not sure just where that particular standard value came from. Its
not clear that a standard value is appropriate at all. In research the
costs and the rewards are quite intangible. It is, therefore hard to
do detailed cost/benefit analysis to determine the proper alpha. A
rule of thumb is therefore appropriate (though whether the one we have
ended up with is the one that we *should* be using is an interesting
question). In SPC, generally, the costs and benefits of various
choices are much easier to quantify and so cost/benefit analysis would
seem to be the way to go.
(Probably argued in the SPC literature which I am almost totally
ignorant of).
Topher
|
| I think the previous reply (.5) summed it up quite nicely. I would only
add the following:
The .0037 is with only 1 `rule' applied. Very commonly, the Western
Electric rules and/or Nelson rules are applied which increase alpha to
a level ~.01 or greater. Examples are: 2 of 3 points beyond the 2-sigma
limit, 6 consecutive points increasing or decreasing in the same
direction. So, the alpha risk isn't really that low. The advantage of
applying more rules (weighed against the increased alpha) is the
greater `statistical power' of the test: the ability to reject Ho when
H1 is true.
Topher is right in that the economics of the situation should drive the
choices. There is a chapter in a book called ``Introduction to
STATISTICAL QUALITY CONTROL'' by Montgomery (publ.=Wiley) that is
entitled "Economic Design of Control Charts". You can guess what this
chapter is about!
Peter
|