Как выполнить тест независимости хи-квадрат в Python
Хи -квадрат тест независимостииспользуется, чтобы определить, существует ли значительная связь между двумя категориальными переменными.
В этом руководстве объясняется, как выполнить тест независимости хи-квадрат в Python.
Пример: критерий независимости хи-квадрат в Python
Предположим, мы хотим знать, связан ли пол с предпочтениями политической партии. Мы берем простую случайную выборку из 500 избирателей и опрашиваем их об их предпочтениях в отношении политических партий. В следующей таблице представлены результаты опроса:
| | | | | | | — | — | — | — | — | | | республиканец | демократ | Независимый | Общий | | Мужской | 120 | 90 | 40 | 250 | | женский | 110 | 95 | 45 | 250 | | Общий | 230 | 185 | 85 | 500 |
Используйте следующие шаги, чтобы выполнить тест независимости хи-квадрат в Python, чтобы определить, связан ли пол с предпочтениями политической партии.
Шаг 1: Создайте данные.
Сначала мы создадим таблицу для хранения наших данных:
data = [[120, 90, 40], [110, 95, 45]] Шаг 2: Выполните тест независимости хи-квадрат.
Затем мы можем выполнить критерий независимости хи-квадрат, используя функцию chi2_contingency из библиотеки SciPy, которая использует следующий синтаксис:
chi2_contingency (наблюдается)
Следующий код показывает, как использовать эту функцию в нашем конкретном примере:
import scipy.stats as stats #perform the Chi-Square Test of Independence stats.chi2_contingency(data) (0.864, 0.649, 2, array([[115. , 92.5, 42.5], [115. , 92.5, 42.5]])) Способ интерпретации вывода следующий:
- Статистика теста хи-квадрат: 0,864
- р-значение: 0,649
- Степени свободы: 2 (рассчитывается как #rows-1 * #columns-1)
- Массив: последний массив отображает ожидаемые значения для каждой ячейки в таблице непредвиденных обстоятельств.
Напомним, чтокритерий независимости хи-квадрат использует следующие нулевые и альтернативные гипотезы:
- H 0 : (нулевая гипотеза) Две переменные независимы.
- H 1 : (альтернативная гипотеза) Две переменные не являются независимыми.
Поскольку p-значение (0,649) теста не меньше 0,05, мы не можем отвергнуть нулевую гипотезу. Это означает, что у нас нет достаточных доказательств, чтобы сказать, что существует связь между полом и предпочтениями политических партий.
Другими словами, предпочтения пола и политической партии не зависят друг от друга.
scipy.stats.chi2_contingency#
Chi-square test of independence of variables in a contingency table.
This function computes the chi-square statistic and p-value for the hypothesis test of independence of the observed frequencies in the contingency table [1] observed. The expected frequencies are computed based on the marginal sums under the assumption of independence; see scipy.stats.contingency.expected_freq . The number of degrees of freedom is (expressed using numpy functions and attributes):
dof = observed.size - sum(observed.shape) + observed.ndim - 1
The contingency table. The table contains the observed frequencies (i.e. number of occurrences) in each category. In the two-dimensional case, the table is often described as an “R x C table”.
correction bool, optional
If True, and the degrees of freedom is 1, apply Yates’ correction for continuity. The effect of the correction is to adjust each observed value by 0.5 towards the corresponding expected value.
lambda_ float or str, optional
By default, the statistic computed in this test is Pearson’s chi-squared statistic [2]. lambda_ allows a statistic from the Cressie-Read power divergence family [3] to be used instead. See scipy.stats.power_divergence for details.
Returns : res Chi2ContingencyResult
An object containing attributes:
expected_freq ndarray, same shape as observed
The expected frequencies, based on the marginal sums of the table.
An often quoted guideline for the validity of this calculation is that the test should be used only if the observed and expected frequencies in each cell are at least 5.
This is a test for the independence of different categories of a population. The test is only meaningful when the dimension of observed is two or more. Applying the test to a one-dimensional table will always result in expected equal to observed and a chi-square statistic equal to 0.
This function does not handle masked arrays, because the calculation does not make sense with missing values.
Like scipy.stats.chisquare , this function computes a chi-square statistic; the convenience this function provides is to figure out the expected frequencies and degrees of freedom from the given contingency table. If these were already known, and if the Yates’ correction was not required, one could use scipy.stats.chisquare . That is, if one calls:
res = chi2_contingency(obs, correction=False)
then the following is true:
(res.statistic, res.pvalue) == stats.chisquare(obs.ravel(), f_exp=ex.ravel(), ddof=obs.size - 1 - dof)
The lambda_ argument was added in version 0.13.0 of scipy.
Cressie, N. and Read, T. R. C., “Multinomial Goodness-of-Fit Tests”, J. Royal Stat. Soc. Series B, Vol. 46, No. 3 (1984), pp. 440-464.
Berger, Jeffrey S. et al. “Aspirin for the Primary Prevention of Cardiovascular Events in Women and Men: A Sex-Specific Meta-analysis of Randomized Controlled Trials.” JAMA, 295(3):306-313, DOI:10.1001/jama.295.3.306, 2006.
In [4], the use of aspirin to prevent cardiovascular events in women and men was investigated. The study notably concluded:
…aspirin therapy reduced the risk of a composite of cardiovascular events due to its effect on reducing the risk of ischemic stroke in women […]
The article lists studies of various cardiovascular events. Let’s focus on the ischemic stoke in women.
The following table summarizes the results of the experiment in which participants took aspirin or a placebo on a regular basis for several years. Cases of ischemic stroke were recorded:
Aspirin Control/Placebo Ischemic stroke 176 230 No stroke 21035 21018
Is there evidence that the aspirin reduces the risk of ischemic stroke? We begin by formulating a null hypothesis \(H_0\) :
The effect of aspirin is equivalent to that of placebo.
Let’s assess the plausibility of this hypothesis with a chi-square test.
>>> import numpy as np >>> from scipy.stats import chi2_contingency >>> table = np.array([[176, 230], [21035, 21018]]) >>> res = chi2_contingency(table) >>> res.statistic 6.892569132546561 >>> res.pvalue 0.008655478161175739
Using a significance level of 5%, we would reject the null hypothesis in favor of the alternative hypothesis: “the effect of aspirin is not equivalent to the effect of placebo”. Because scipy.stats.contingency.chi2_contingency performs a two-sided test, the alternative hypothesis does not indicate the direction of the effect. We can use stats.contingency.odds_ratio to support the conclusion that aspirin reduces the risk of ischemic stroke.
Below are further examples showing how larger contingency tables can be tested.
>>> obs = np.array([[10, 10, 20], [20, 20, 20]]) >>> res = chi2_contingency(obs) >>> res.statistic 2.7777777777777777 >>> res.pvalue 0.24935220877729619 >>> res.dof 2 >>> res.expected_freq array([[ 12., 12., 16.], [ 18., 18., 24.]])
Perform the test using the log-likelihood ratio (i.e. the “G-test”) instead of Pearson’s chi-squared statistic.
>>> res = chi2_contingency(obs, lambda_="log-likelihood") >>> res.statistic 2.7688587616781319 >>> res.pvalue 0.25046668010954165
A four-way example (2 x 2 x 2 x 2):
>>> obs = np.array( . [[[[12, 17], . [11, 16]], . [[11, 12], . [15, 16]]], . [[[23, 15], . [30, 22]], . [[14, 17], . [15, 16]]]]) >>> res = chi2_contingency(obs) >>> res.statistic 8.7584514426741897 >>> res.pvalue 0.64417725029295503
scipy.stats.chi2_contingency#
Chi-square test of independence of variables in a contingency table.
This function computes the chi-square statistic and p-value for the hypothesis test of independence of the observed frequencies in the contingency table [1] observed. The expected frequencies are computed based on the marginal sums under the assumption of independence; see scipy.stats.contingency.expected_freq . The number of degrees of freedom is (expressed using numpy functions and attributes):
dof = observed.size - sum(observed.shape) + observed.ndim - 1
The contingency table. The table contains the observed frequencies (i.e. number of occurrences) in each category. In the two-dimensional case, the table is often described as an “R x C table”.
correction bool, optional
If True, and the degrees of freedom is 1, apply Yates’ correction for continuity. The effect of the correction is to adjust each observed value by 0.5 towards the corresponding expected value.
lambda_ float or str, optional
By default, the statistic computed in this test is Pearson’s chi-squared statistic [2]. lambda_ allows a statistic from the Cressie-Read power divergence family [3] to be used instead. See scipy.stats.power_divergence for details.
Returns : res Chi2ContingencyResult
An object containing attributes:
expected_freq ndarray, same shape as observed
The expected frequencies, based on the marginal sums of the table.
An often quoted guideline for the validity of this calculation is that the test should be used only if the observed and expected frequencies in each cell are at least 5.
This is a test for the independence of different categories of a population. The test is only meaningful when the dimension of observed is two or more. Applying the test to a one-dimensional table will always result in expected equal to observed and a chi-square statistic equal to 0.
This function does not handle masked arrays, because the calculation does not make sense with missing values.
Like scipy.stats.chisquare , this function computes a chi-square statistic; the convenience this function provides is to figure out the expected frequencies and degrees of freedom from the given contingency table. If these were already known, and if the Yates’ correction was not required, one could use scipy.stats.chisquare . That is, if one calls:
res = chi2_contingency(obs, correction=False)
then the following is true:
(res.statistic, res.pvalue) == stats.chisquare(obs.ravel(), f_exp=ex.ravel(), ddof=obs.size - 1 - dof)
The lambda_ argument was added in version 0.13.0 of scipy.
Cressie, N. and Read, T. R. C., “Multinomial Goodness-of-Fit Tests”, J. Royal Stat. Soc. Series B, Vol. 46, No. 3 (1984), pp. 440-464.
Berger, Jeffrey S. et al. “Aspirin for the Primary Prevention of Cardiovascular Events in Women and Men: A Sex-Specific Meta-analysis of Randomized Controlled Trials.” JAMA, 295(3):306-313, DOI:10.1001/jama.295.3.306, 2006.
In [4], the use of aspirin to prevent cardiovascular events in women and men was investigated. The study notably concluded:
…aspirin therapy reduced the risk of a composite of cardiovascular events due to its effect on reducing the risk of ischemic stroke in women […]
The article lists studies of various cardiovascular events. Let’s focus on the ischemic stoke in women.
The following table summarizes the results of the experiment in which participants took aspirin or a placebo on a regular basis for several years. Cases of ischemic stroke were recorded:
Aspirin Control/Placebo Ischemic stroke 176 230 No stroke 21035 21018
Is there evidence that the aspirin reduces the risk of ischemic stroke? We begin by formulating a null hypothesis \(H_0\) :
The effect of aspirin is equivalent to that of placebo.
Let’s assess the plausibility of this hypothesis with a chi-square test.
>>> import numpy as np >>> from scipy.stats import chi2_contingency >>> table = np.array([[176, 230], [21035, 21018]]) >>> res = chi2_contingency(table) >>> res.statistic 6.892569132546561 >>> res.pvalue 0.008655478161175739
Using a significance level of 5%, we would reject the null hypothesis in favor of the alternative hypothesis: “the effect of aspirin is not equivalent to the effect of placebo”. Because scipy.stats.contingency.chi2_contingency performs a two-sided test, the alternative hypothesis does not indicate the direction of the effect. We can use stats.contingency.odds_ratio to support the conclusion that aspirin reduces the risk of ischemic stroke.
Below are further examples showing how larger contingency tables can be tested.
>>> obs = np.array([[10, 10, 20], [20, 20, 20]]) >>> res = chi2_contingency(obs) >>> res.statistic 2.7777777777777777 >>> res.pvalue 0.24935220877729619 >>> res.dof 2 >>> res.expected_freq array([[ 12., 12., 16.], [ 18., 18., 24.]])
Perform the test using the log-likelihood ratio (i.e. the “G-test”) instead of Pearson’s chi-squared statistic.
>>> res = chi2_contingency(obs, lambda_="log-likelihood") >>> res.statistic 2.7688587616781319 >>> res.pvalue 0.25046668010954165
A four-way example (2 x 2 x 2 x 2):
>>> obs = np.array( . [[[[12, 17], . [11, 16]], . [[11, 12], . [15, 16]]], . [[[23, 15], . [30, 22]], . [[14, 17], . [15, 16]]]]) >>> res = chi2_contingency(obs) >>> res.statistic 8.7584514426741897 >>> res.pvalue 0.64417725029295503