q-values

출처: http://www.nonlinear.com/support/progenesis/comet/faq/v2.0/pq-values.aspx

q-values

Q-values are the name given to the adjusted p-values found using an optimised FDR approach. 

The FDR approach is optimised by using characteristics of the p-value distribution to produce a list of q-values. 

In what follows, I will tie up some ideas and hopefully this will help clarify what we have been saying about p and q values.

It is usual to test many hundreds or thousands of compound variables in a metabolomics experiment. 

Each of these tests will produce a p-value. 

The p-values take on a value between 0 and 1 and 

we can create a histogram to get an idea of how the p-values are distributed between 0 and 1. 

Some typical p-value distributions are shown below. 

On the x-axis, we have histogram bars representing p-values. 

Each bar has a width of 0.05 and so in the first bar (red or green) we have those p-values that are between 0 and 0.05. 

Similarly, the last bar represents those p-values between 0.95 and 1.0, and so on. 

The height of each bar gives an indication of how many values are in the bar. 

This is called a density distribution because the area of all the bars always adds up to 1. 

Although the two distributions appear quite different, you will notice that they flatten off towards the right of the histogram. 

The red (or green) bar represents the significant values, if you set a p-value threshold of 0.05.

If there are no significant changes in the experiment, you will expect to see a distribution more like that on the left above 

while an experiment with significant changes will look more like that on the right. 

So, even if there are no significant changes in the experiment, we still expect, by chance, to get p-values < 0.05. 

These are false positives, and shown in red. 

Even in an experiment with significant changes (in green), we are still unsure if a p-value < 0.05 represents a true discovery or a false positive. 

Now, the q-value approach tries to find the height where the p-value distribution flattens out 

and incorporates this height value into the calculation of FDR adjusted p-values. 

We can see this in the histogram below. 

This approach helps to establish just how many of the significant values are actually false positives (the red portion of the green bar).

Now, the q-values are simply a set of values that will lie between 0 and 1. 

Also, if you order the p-values used to calculate the q-values, then the q-values will also be ordered. 

This can be seen in the following screen shot from Progenesis CoMet; notice that q-values can be repeated:

To interpret the q-values, you need to look at the ordered list of q-values. 

There are 3516 compounds in this experiment. 

If we take unknown compound 1723 as an example, we see that it has a p-value of 0.0101 and a q-value of 0.0172. 

Recall that a p-value of 0.0101 implies a 1.01% chance of false positives, 

On the other hand, the q-value is 0.0172, 

which means we should expect 1.72% of all the compounds with q-value less than this to be false positives. This is a much better situation. 

We know that 800 compounds have a q-value of 0.0172 or less 

and so we should expect 800 × 0.0172 = 13.76 false positives rather than the predicted 36. 

Just to reiterate, false positives according to p-values take all 3516 values into account 

when determining how many false positives we should expect to see while q-values take into account only those tests 

with q-values less than the threshold we choose. 

Of course, it is not always the case that q-values will result in less false positives, 

but what we can say is that they give a far more accurate indication of the level of false positives for a given cut-off value.

When doing lots of tests, as in a metabolomics experiment, it is more intuitive to interpret p and q values 

by looking at the entire list of values in this way rather that looking at each one independently. 

In this way, a threshold of 0.05 has meaning across the entire experiment. 

When deciding on a cut-off or threshold value, you should do this from the point of view of how many false positives will this result in, 

rather than just randomly picking a p- or q-value of 0.05 and saying that every compound with p-value less this is significant.