Significance of overlapping gene lists

Wen Fury and Wentian Li
http://www.nslij-genetics.org/wli/pub/ieee-embs06.pdf

To identify significance of overlap for two differentially expressed gene sets n1 and n2 (e.g. d1-n1 and d2-n1) use either hypergeometric or Fisher's exact test p-value.

Given integers n, n1, n2, m (max(n1,n2) <= n and m <= min (n1,n2)), the hypergeometric distribution is defined as

P(m) = [C(n1, m) * C (n - n1, n2 -m)]/ C (n, n2)

where C(n,m) is the number of possibilities of choosing m objects out of n objects : C (n,m) = n!/[m! (n -m)!]

It is usually more interesting to calculate the sum of P(m) for m's equal or larger than the observed value (i.e. p-value) :


p-value = Sigma [k= m to min (n1,n2)] p(k)
= Sigma [k = 0 to min (n1,n2)] p(k) - Sigma [k = 0 to m - 1] p(k)

For calculating it in R use :

if m = 0, p-value = 1

phyper (m, n1, n - n1, n2):
p-value = phyper(min(n1,n2), n1, n-n1, n2) - phyper(m-1, n1, n-n1, n2) if m > 0

One can also use Fisher's exact test on the following 2-by-2 table:

col1 col2 total
row1 m n1-m n1
row2 n2-m n-n1-n2+m n - n1
total n2 n-n2 n

They produce identical results.

Test about a single proportion for categorical data

Ho: Pi = Pio vs. HA: Pi NA Pio

Wald Test: Z = Pi-hat - Pio/ sqrt (Pi-hat (1 - Pi-hat)/n)

Score Test : Z = Pi-hat - Pio/ sqrt (Pio (1 - Pio)/n)

Score test is slightly more powerful in distinguising evidence against Ho.


R-code for score test : prop.test (27, 922, p=0.02, correct =FALSE)

Here is p is the population probability. and 27 is the observed instance of some phenomena in population. 27/922 will give Pi-hat - the binomial estimate in sample.
Its a Chi-square test on 1 degree of freedom.

Here the p-value is 0.0440 - where the Ho is rejected and 27/922 = 0.029 is NA to 0.02.

Obtaining Confidence Intervals
===============================

The Wald Confidence Interval for single proportion can be obtained by inverting the Wald test statistic but it falls into problem if n is small or Pi is small.

A Score CI can be used in this case.

The R-code is

Wald

library(Hmisc)

binconf(27,922,method="asymptotic")

PointEst Lower Upper
0.02928416 0.01840125 0.04016708


Score (Wilson):

binconf(27,922,method="wilson")

PointEst Lower Upper
0.02928416 0.02020271 0.04227177

prop.test also give the score CI.

========================

In case observed events are small in number and normality assumption is not valid.
One can use binomial exact test to calculate the p-value and CI.

binom.test(3, 58, p = 0.02,
+ alternative = "two.sided",
+ conf.level = 0.95)
Exact binomial test
data: 3 and 58
number of successes = 3, number of trials = 58,
p-value = 0.1101
alternative hypothesis: true probability of success
is not equal to 0.02
95 percent confidence interval:
0.01079648 0.14380463
sample estimates:
probability of success
0.05172414