This chapter is a continuation of the previous chapter on inferential statistics.

The importance of statistical inference is grounded in several assumptions. First, it addresses a particular type of uncertainty, namely that caused by having data from random samples rather than having complete knowledge of entire populations, processes, or distributions. Second, we cannot build statistical inference without first building an appreciation of sample versus population, of description versus inference, and of characteristics of samples giving estimates of characteristics of populations. Third, any conceptual approach to statistical inference must flow from some essential understanding of the nature and behavior of sampling variation. Finally, statistical inference is viewed as both an outcome and a reasoned process of creating or testing probabilistic generalizations from data. The prior four chapters introduced you in varying degrees to these points.

In this chapter, you will see how sampling distributions are used to test hypotheses and construct confidence intervals. Topics discussed in this chapter are:

• Implicit Assumptions in Making Statistical Inferences
• Significance of Statistical Inference Methods
  • Confidence Intervals
  • Tests of Significance

 5.1. Implicit Assumptions in Making Statistical Inferences

One of the statistician’s most important roles is the upfront contribution of planning a study, which includes identifying the implicit assumptions when making statistical inferences. It is important that these assumptions are met, and if they are not, the statistical inferences can result in seriously flawed conclusions in the study. According to Hahn & Meeker (1993)^[1], the first assumption is that the target population has been explicitly and precisely defined. The second assumption is that a specific listing (or another enumeration) of the population from which the samples have been selected has been made. Third, the data are assumed to be a random sample of the population. The assumption of random sampling is critical. When the assumption of random sampling (or randomization) is not met, inferences to the population become difficult. In this case, statisticians should describe the sample and population in sufficient detail to justify that the sample was at least representative of the intended population.

 5.2. The Significance of Statistical Inference Methods

You have now acquired a clear understanding of the difference between a sample (the observed) and the population (the unobserved). The extent to which credence should be placed in a given sample statistic as a description of the population parameter is the problem of inferring from the part to the whole. This is sometimes called the problem of inductive inference, or the problem of generalization.

Using the field of Econometrics^[2] as an example, Keuzenkamp & Magnus (1995)^[3] describes the four types of statistical testing:

1. Theory Testing: This method consists of formulating hypotheses from which predictions of novel facts are deduced (the consequences).
2. Validity Testing: Performed in order to find out whether the statistical assumptions underlying some models are credible. In order to pursue a theory test, one first must be sure of the validity of the statistical assumptions that are made.
3. Simplification Testing: Simplicity matters. Simple models are typically preferred to complex ones. Rather than testing from general to simple, statisticians perform iterative simplification searches. In the study of statistics, we focus on mathematical distributions for the sake of simplicity and relevance to the real world. Understanding these distributions enables us to visualize the data more easily and build models more quickly.
4. Decision-Making: Based on statistical acceptance rules, this method can be important for process quality control and can be extended to appraising theories.

The type of statistical test one uses depends on the type of study design, number of groups of comparison, and type of data (i.e., continuous, dichotomous, and categorical) (Parab & Bhalerao 2010).^[4]

Subsequent sections will discuss statistical inference methods that use the language of probability to estimate the value of a population parameter. The two most common methods are confidence intervals and tests of significance.

 5.2(a). Confidence Intervals (CI)

A confidence interval (CI), in statistics, refers to the probability that a population parameter falls between two set values. The sample is used to estimate the interval of probable values of the parameters of the population. It is a matter of convention to use the standard 95% confidence interval having the probability that there are 95 chances out of 100 of being right. There are five chances out of 100 of being wrong. Even when using the 95%^[5] confidence interval, we must remember that the sample mean could be one of those five sample means that fall outside the established interval. A statistician never knows for sure.

According to Zhang, Hanik & Chaney (2008)^[6], the CI has four noteworthy characteristics:

1. For a given sample size, at a given level of confidence, and using probability sampling, there can be infinitely many CIs for a particular population parameter. The point estimates and endpoints of these CIs vary due to sampling errors that occur each time a different sample is drawn.
2. The CI reported in a certain research study is just one of these infinitely many CIs.
3. The percentage of these CIs that contain the population parameter is the same as the level of confidence.
4. Whether a certain CI reported by a research study contains the population parameter is unknown. In other words, the level of confidence is applied to the infinitely many CIs, rather than a single CI reported by a single study.

Any confidence interval has two parts: an interval calculated from the data and a confidence level C. The confidence interval often has the form:

estimate [latex]\pm[/latex] margin of error

The confidence level is the success rate of the method that produces the interval, that is, the probability that the method will give a correct answer. The statistician chooses the confidence level, and the margin of error follows from this choice. When the data and the sample size remain the same, higher confidence results in a larger margin of error. There is a tradeoff between the confidence level and the margin of error. To obtain a smaller margin of error, the statistician must be able to accept a lower confidence. As the sample size increases, the margin of error gets smaller.

A level C confidence interval for the mean µ of a Normal population with known standard deviation σ, based on a simple random sample of size n, is given by:

[latex]\bar{X}\pm z*\frac{\sigma}{\sqrt{n}}[/latex]

The critical value z*  is chosen so that the standard Normal curve has an area C between – z* and z*. Below are the entries for the most common confidence levels:

We know from the Central Limit Theorem (chapter 4) that when n ≥ 30, the sampling distribution of sample means approximates a normal distribution. The level of confidence C is the area under the standard normal curve between the critical values, -z_c and z_c. Critical values are values that separate sample statistics that are probable from sample statistics that are improbable, or unusual. For instance, if C =95%, then 2.5% of the area lies to the left of -z_c = -1.960 and 2.5% lies to the right of z_c = 1.960, as shown in the Table below:

An example of the meaning of the 95% confidence interval is as follows:

Suppose a medical researcher is interested in the prenatal care received by pregnant women in the inner city of Philadelphia. She breaks down the various sections of North Philly and computes the average number of gynecological check ups per pregnancy for all possible samples of size 20 and constructs the 95% confidence intervals using these sample means. Then 95% of these intervals would contain µ [population parameter]and 5% would not. Note that we cannot say that the probability is .95 that the interval from 2.6 to 3.4 gynecological checkups, for example, contains µ. Either the interval contains µ or it does not.

The 95% confidence interval of 2.6 to 3.4 was calculated based on the average number of gynecological check ups per pregnancy being 3, with a standard deviation of 1.

[latex]\bar{X}\pm z*\frac{\sigma }{n}[/latex]

[latex]=3\pm 1.96\frac{1}{\sqrt{20}}[/latex]

[latex]=2.6[/latex] to [latex]3.4[/latex]

Important! After constructing a confidence interval, the results must be interpreted correctly. Consider the 95% confidence interval constructed in the above example. Because µ is a fixed value predetermined by the population, it is in the interval or not. It is not correct to say, “There is a 95% probability that the actual mean will be in the interval (25.22, 26.78).” This statement is wrong because it suggests that the value µ can vary, which is not true. The correct way to interpret this confidence interval is to say, “With 95% confidence, the mean is in the interval (25.22, 26.78).” This means that when a large number of samples are collected and a confidence interval is created for each sample, approximately 95% of these intervals will contain µ.

Now Try It Yourself

5.2(b). Tests of Significance

The second type of statistical inference is tests of significance. The tests of significance aid the statistician in making inferences from the observed sample to the unobserved population.

The test of significance provides a relevant and useful way of assessing the relative likelihood that a real difference exists and is worthy of interpretive attention, as opposed to the hypothesis that the set of data could be an arrangement lacking any type of pattern. The basic idea of statistical tests is like that of confidence intervals: what would happen if we repeated the sample many times?

A statistical test starts with a careful statement of the claims a statistician wants to compare. Because the reasoning of tests looks for evidence against a claim, we start with the claim we seek evidence against, such as “no effect or “no difference.”

 5.2(b)(1). The Null Hypothesis: No Difference Between Means

A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently supports a particular hypothesis.^[8] In terms of selecting a statistical test, the most important question is, “What is the main hypothesis for the study?” In some cases, there is no hypothesis; the statistician just wants to “see what is there.”

On the other hand, if a scientific question is to be examined by comparing two or more groups, one can perform a statistical test. For this, initially, a null hypothesis needs to be formulated, which states that there is no difference between the two groups. It is expected that at the end of the study, the null hypothesis is either rejected or not rejected (Parab & Bhalerao 2010).

The claim tested by a statistical test is called the null hypothesis. The test is designed to assess the strength of the evidence against the null hypothesis. The claim about the population that we are trying to find evidence for is the alternative hypothesis. The alternative hypothesis is one-sided if it states that the parameter is larger than or smaller than the null hypothesis value. It is two-sided if it states that the parameter is different (larger or smaller) from the null value (Moore, Notz & Fligner 2013).

The null hypothesis is abbreviated as H_o, and the alternative hypothesis as H_a. H_o is a statistical hypothesis that contains a statement of equality, such as ≤, =, or ≥. H_a is the complement of the null hypothesis. It is a statement that must be true if H_o is false. It contains a statement of strict inequality, such as >, ≠, or <. H_o is read as “H naught.” H_a is read as “H sub-a.”

You always begin a hypothesis test by assuming that the equality condition in the null hypothesis is true. When you perform a hypothesis test, you make one of two decisions:

1. Reject the null hypothesis or
2. Fail to reject the null hypothesis.

Know that because a decision is based on a sample rather than the entire population, there is the possibility of making the wrong decision. The only way to be absolutely certain of whether Ho is true or false is to test the entire population, which, in reality, may not be feasible. So, the statistician must accept the fact that the decision might be incorrect. As shown in Figure 2 below, there are two types of errors that can be made:

A Type I Error occurs if the null hypothesis is rejected when it is true.

A Type II Error occurs if the null hypothesis is not rejected when it is false.

In a hypothesis test, the level of significance is the maximum allowable probability of making a Type I error. It is denoted by α, the lowercase Greek letter alpha. The probability of a Type II error is denoted by β, the lowercase Greek letter beta. By setting the level of significance at a small value, the probability of rejecting a true null hypothesis will be small. The commonly used levels of significance are: α = 0.01; α = 0.05; and α = 0.10. When α is decreasing (the maximum probability of making a Type I Error), it is likely that β is increasing. The value 1 – β is called the power of the test. It represents the probability of rejecting the null hypothesis when it is false.

Hypotheses always refer to the population, not to a particular outcome. They are stated in terms of population parameters. Thus, for no difference between means, the null hypothesis can be symbolized as

µ₁  = µ₂

where µ₁ = mean of the first population

µ₂ = mean of the second population

Now Try It Yourself^[9]

A test is based on a test statistic that measures how far the sample outcome is from the value stated by H_o. The P-value of a test is the probability that the test statistic will take a value at least as extreme as that actually observed. Small P-values indicate strong evidence to reject the null hypothesis provided by the data. However, a very low P-value does not constitute proof that the null hypothesis is false, only that it is probably false. Large P-values fail to give evidence against H_o. The most important task is to understand what a P-value says.

If the P-value is as small or smaller than a specified value α (alpha), the data are statistically significant at significance level α.

The P-value of a test depends on the nature of the test. As shown in Figure 3 below, there are three types of hypothesis tests—left-tailed, right-tailed, and two-tailed (where evidence that would support the alternative hypothesis could lie in either tail of the sampling distribution).

In a left-tailed test,  the alternative hypothesis H_a contains the less-than inequality symbol (<):

H_o: µ ≥ k

H_a: µ < k

In a right-tailed test, the alternative hypothesis H_a contains the greater-than-inequality symbol (>):

H_o: µ ≤ k

H_a: µ > k

In a two-tailed test, the alternative hypothesis H_a contains the not-equal-to symbol (≠):

H_o: µ = k

H_a: µ ≠ k

Student’s t Test^[12]: A t-test may be used to evaluate whether a single group differs from a known value (a one-sample t-test), whether two groups differ from each other (an independent two-sample t-test), or whether there is a significant difference in paired measurements (a paired, or dependent samples t-test). t-tests rely on an assumed “null hypothesis.” You have to decide whether this is a one-tail (Is it greater or less than?) or a two-tail test (Is there a difference?) and choose the level of significance (Greek letter alpha, α). An alpha of .05 results in 95% confidence intervals and determines the cutoff for when P-values are considered statistically significant.

The assumptions of a t-test are:

• One variable of interest.
• Numeric data.
• Two groups or less.
• Random sample.
• Normally distributed.

Chi-Square Test (χ2): a statistical test commonly used to determine if there is a significant association between two variables.

For example^[13], many Black and Latinx organizations receive relatively small program grants. To reverse this trend and to engender sustainability, foundations could create grantee cohorts in traditionally underserved communities and neighborhoods. This neighborhood cohort approach has been used successfully by foundations for decades for a variety of reasons, such as achieving efficiency and promoting collaboration. Unbound Philanthropy’s Good Neighbor Committee, a staff initiative dedicated to grantmaking in the New York City metro, sought to reduce gang violence in central Long Island by supporting several organizations that were working on the problem from different angles. These included a youth development organization and a parent advocacy organization. The Chi-Square Test for Independence tests two hypotheses:

Null Hypothesis: There is not a significant association between variables, the variables are independent of each other. Any association between variables is likely due to chance and sampling error. For example, there is no significant association between Organization A (a youth development organization) and Organization B (a parent advocacy organization). Each organization’s ability to reduce gang violence has nothing to do with the other.

Alternative Hypothesis: There is a significant association (positive or negative) between variables, the variables are independent of each other. Any association between variables is not likely due to chance and sampling error. For example, there is a significant association between Organization A (a youth development organization) and Organization B (a parent advocacy organization). Each organization’s ability to reduce gang violence, to some degree, impacts the other.

1. Find the P-value.
2. Use this decision rule: If P-value is less than or equal to the level of significance, then reject the null hypothesis. If P-value is greater than the level of significance, then fail to reject the null hypothesis.
3. Write a statement to interpret the decision in the context of the original claim.

Now Try It Yourself

Below are a sample of hypotheses found in the research literature on race and ethnicity and their intersection with other variables, such as age, socioeconomic status, and sexual preference:

• Contact Hypothesis: Interracial contact is associated with more positive racial attitudes, especially among Whites and some effects are appreciable.
• Cumulative Disadvantage Hypothesis: Predicts that initial advantages and disadvantages compound and produce diverging health trajectories as individuals age. Rationale: Given the structural disadvantages that people of color face across multiple domains of the life course, the cumulative disadvantage hypothesis predicts that racial-ethnic health disparities increase with age. (Brown, O’Rand & Adkins (2012)^[15].
• Ethnicity Hypothesis: Ethnic minorities engage more in social activities than Whites of comparable socioeconomic status. Rationale: The relatively smaller, more cohesive ethnic group is able to exert pressure on the individual member to conform to the norms of the respective ethnic affiliation. (Antunes and Gaitz 1975)^[16].
• LGBT-POC (People of Color) Hypothesis: These individuals may experience unique stressors associated with their dual minority status, including simultaneously being subjected to multiple forms of microaggressions (brief, daily assaults which can be social or environmental, verbal or nonverbal, intentional or unintentional). Within LGBT communities, LGBT-POC may experience racism in dating relationships and social networks. Rationale: Racial/ethnic minority individuals have reported exclusion from LGBT community events and spaces. For example, certain gay bars have been noted for refusing entry of African Americans and providing poorer service to Black patrons. (Balsam et al. 2011)^[17].
• Persistent Inequality Hypothesis: Predicts that racial-ethnic inequalities in health remain stable with age. Rationale: Socio-economic conditions and race-ethnicity are considered “fundamental causes” of disease and illness because of their persistent association with health over time regardless of changing intermediate mechanisms. (Brown, O’Rand & Atkins (2012).
• Statistical Discrimination or Profiling Hypothesis: In this situation, an individual or firm uses overall beliefs about a group to make decisions about an individual from that group. The perceived group characteristics are assumed to apply to the individual. Thus, statistical discrimination may result in an individual member of the disadvantaged group being treated in a way that does not focus on his or her own capabilities. (National Academies Press 2004)^[18].
• Weathering Hypothesis: Chronic exposure to social and economic disadvantage leads to accelerated decline in physical health outcomes and could partially explain racial disparities in a wide array of health conditions. (Forde, Crookes, Suglia & Demmer 2019)^[19].

Author’s Comment: The above reminds us that the journey of “equity-mindedness” is really about doing anti-racism, anti-oppression, and anti-colonialism work. Thus, statistics must be socially just in its methods as well as its intentions.

Writing this OER book has reinforced the importance of upholding dharma (ethical and moral righteousness) when planning and implementing a study. Not only do we have to be careful about “bias creep,” but be careful about the unintended deleterious effects of conclusions reached.

Thank you all for staying committed to this journey! I take this opportunity to acknowledge the group Sweet Honey in the Rock, which celebrates its 50th Anniversary this year of singing social justice songs. Their debut began in 1973 at a workshop at Howard University, a historically Black college in Washington, DC. It was their creative songs that motivated me each day to write this OER book.