To calculate the standard error for confidence intervals or for a hypothesis test using the correct formula.
To check Validity Conditions for Two Sample z-procedures
To apply Theory-Based methods for Inference and Confidence Intervals for Two Proportions.
Draw appropriate conclusions from Theory-based techniques for Two Proportions
As usual, we start by loading our two packages: mosaic
and ggformula
. To load a package, you use the
library()
function, wrapped around the name of a package.
I’ve put the code to load one package into the chunk below. Add the
other package you need.
We’ll load the example data, GSS_clean.csv
from this
Url: https://raw.githubusercontent.com/IJohnson-math/Math138/main/GSS_clean.csv
and use the read.csv()
function.
#load data
GSS22 <- read.csv("https://raw.githubusercontent.com/IJohnson-math/Math138/main/GSS22clean.csv")
We also need to do a little data cleaning to ensure this will work properly for the lab,
Our research question is whether there is a difference in the
proportion of people who said marijuana should be made legal in the two
groups of people that are self employed or work for somebody else. The
works_for
variable is the explanatory variable and
should_marijuana_be_legal
is the response variable.
Our null hypothesis is the proportion of people that believe marijuana should be made legal is the same in the self employed group as it is in the work for someone else group. In other words, there is no association between thinking marijuana should be legal and whether a person works for someone else or is self employed.
Let \(\pi_{selfEmp}\) be the proportion of self-employed people that think marijuana should be legal and \(\pi_{someoneElse}\) be the proportion of people that work for someone else that think marijuana should be legal.
Our null and alternative hypotheses are
\[H_0 : \pi_{selfEmp} - \pi_{someoneElse} = 0\] \[H_a : \pi_{SelfEmp} - \pi_{SomeoneElse} \neq 0\]
Let’s start by creating a bar chart to visualize the data. We want to graph the two groups, self employed or work for somebody else, and see in each bar those that believe marijuana should be legal and those that don’t.
Here is the most basic bar chart of counts.
gf_bar( ~works_for, fill= ~ should_marijuana_be_legal, data=GSS22, xlab="Employment", title="Employment and views on marijuana legalization")
Here is a bar chart of counts that doesn’t have the counts stacked and instead has them positioned side-by-side.
Here is a segmented bar graph. Notice that the
command has changed to gf_props
instead of
gf_bar
.
Here is a mosaic plot. Caution! In the
mosaicplot( )
function, make sure to list the explanatory
variable first.
mosaicplot( ~ works_for + should_marijuana_be_legal, data=GSS22, main="Employment and Marjiuana Legalization", ylab=" ", xlab=" ", las=1, color=c("salmon", "turquoise"))
We create a 2-way table with the command tally
to
determine the proportion of self employed people that believe marijuana
should be made legal and the proportion of people that work for someone
else that believe marijuana should be made legal.
Important note: the order of the variables matters!!
It should be tally( response_var ~ explanatory_var)
. To
check your work make sure the explanatory variable is displayed
horizontally. Be careful or your proportions will be incorrect!
The first code chunk is a table of counts and the second is a table of proportions.
## works_for
## should_marijuana_be_legal self-employed someone else
## should be legal 90 674
## should not be legal 33 282
We can calculate the sample size for each group: \(n_1\) is the number of people that are self-employed and \(n_2\) the number of people that work for someone else.
## [1] 123
## [1] 956
## [1] 1079
## works_for
## should_marijuana_be_legal self-employed someone else
## should be legal 0.7317073 0.7050209
## should not be legal 0.2682927 0.2949791
Validity Conditions: The theory-based test and interval for the difference in two proportions (called a two-sample z-test or interval) work well when there are at least 10 observations in each of the four cells of the 2 × 2 table.
If we look at the tally
of counts, we see that the
values in the 2 x 2 table are 90, 674, 33, 282, all of which are greater
than 10. So our validity conditions are definitely satisfied.
Let’s start by finding our observed statistic, \(p_{diff}\).
p1 <- 0.7317073
p2 <- 0.7050209
#(p1 for the self-employed group) - (p2 for the works for someone else group)
p_diff <- p1 - p2
p_diff
## [1] 0.0266864
For two proportions, in a hypothesis test the standard error of the null distribution is given by
\[ SE=\sqrt{\frac{\hat{p}(1-\hat{p})}{n_1}+\frac{\hat{p}(1-\hat{p})}{n_2}} \] where \(\hat{p}\) is the pooled proportion.
Using R as a calculator the pooled proportion is
## [1] 0.708063
The Standard error is
## [1] 0.04355216
Next, we can calculate the standardized statistic using the formula
\[ z = \frac{\hat{p}_1 - \hat{p}_2 - 0}{\sqrt{\frac{\hat{p}(1-\hat{p})}{n_1}+\frac{\hat{p}(1-\hat{p})}{n_2}}} = \frac{\hat{p}_{diff} - 0}{\sqrt{\frac{\hat{p}(1-\hat{p})}{n_1}+\frac{\hat{p}(1-\hat{p})}{n_2}}}\]
## [1] 0.6127457
What evidence if any does this standardized statistic provide regarding our hypothesis test?
The standardized statistic is not greater than 2 or less than -2, so we don’t have enough evidence to reject the null hypothesis. It looks like the proportion of people that believe that marijuana should be legal in the group of people that work for someone else is similar to the proportion for those that are self-employed. Thus the difference between those proportions could plausibly be zero.
Next we calculate the theory based \(p\)-value using prop.test
.
Note: In the code below we will omit the default continuity correction
(using the option correct= FALSE
because the counts in
all four cells of the two-way table are large. The continuity
correction becomes important if one of the cell counts is smaller than
10, especially if a count is less than or equal to 5.
#inference for two proportions
prop.test(should_marijuana_be_legal ~ works_for, data = GSS22, success = "should be legal", alternative = "two.sided", conf.level = 0.95, correct=FALSE)
##
## 2-sample test for equality of proportions without continuity correction
##
## data: tally(should_marijuana_be_legal ~ works_for)
## X-squared = 0.37546, df = 1, p-value = 0.54
## alternative hypothesis: two.sided
## 95 percent confidence interval:
## -0.05678065 0.11015344
## sample estimates:
## prop 1 prop 2
## 0.7317073 0.7050209
Here is another way to input data for two proportion inference. This assumes we only have data from a two-way table.
## works_for
## should_marijuana_be_legal self-employed someone else
## should be legal 90 674
## should not be legal 33 282
#USE THIS COMMAND for inference when you only have the counts and not the data
# c(90, 674) are the success counts for the two groups: self employed or works for someone else
# c(123, 956) are the sample size counts for the two groups
# be consistent with the order of the numbers! I'm consistently putting the self-employed group first.
# always use alternative = "two.sided" when calculating confidence intervals!
prop.test(c(90, 674), c(123, 956), alternative = "two.sided", conf.level = 0.99, correct=FALSE)
##
## 2-sample test for equality of proportions without continuity correction
##
## data: c out of c90 out of 123674 out of 956
## X-squared = 0.37546, df = 1, p-value = 0.54
## alternative hypothesis: two.sided
## 99 percent confidence interval:
## -0.0830079 0.1363807
## sample estimates:
## prop 1 prop 2
## 0.7317073 0.7050209
Does the \(p\)-value from
prop.test
support the conclusion made with the standardized
statistic?
The p-value is 0.54 which is much larger that 0.01. We do not have evidence that supports rejecting the null hypothesis. The null hypothesis is plausible. Contextually, this means that the group of self-employed people and the group people that work for someone else support marijuana legalization at similar percentages.
To do find confidence intervals for a difference of proportions, we start by computing the standard error. Recall that the formula for standard error depends on whether we’re doing a confidence interval or a hypothesis test. The reason for the two formulas stems from the fact that when we do a hypothesis test we have a hypothesized value for the unknown parameter, namely\(\pi_{diff}=0\), but when determining a confidence interval we have no preferenced value for the parameter.
For two proportions, the standard error for a confidence interval is given by \[ SE = \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}\]
Notice that this formula uses our observed proportions \(\hat{p}_1\) and \(\hat{p}_2\) instead of the pooled proportion \(\hat{p}\).
Our margin of error (MOE) for a 2SD interval is given by
## [1] 0.07870749
The interval is centered at \(p_{diff}\) with endpoints of our 2SD confidence interval are
## [1] -0.05202109
## [1] 0.1053939
Does this align with a 95% confidence interval calculated using
prop.test
? Yes, they are nearly identical.
We are 95% confident that the difference in proportions, \(\pi_{diff}\), is between -0.055 and 0.109.
Interpret the confidence interval: We are 95% confident that the difference in proportion of people that believe that marijuana should be legal between those that are self-employed and those that work for someone else is between -0.055 and 0.109. Since this confidence interval contains 0, it is plausible that these two proportions are not different at all.
Notice: the standardized statistic, the p-value and the confidence interval all lead to the same conclusion!