Confidence Intervals

class: center, middle, inverse, title-slide

# Confidence Intervals
### AJ Smit
### 2020/06/30 (updated: 2022-04-19)

---

## Confidence Intervals

For more details about a Confidence Intervals, please visit
<https://rcompanion.org/handbook/C_03.html>

---
## What are Confidence Intervals for?

Confidence intervals (CIs) determined *for a sample* tell us the range within which we may be certain to find *the true population mean* from which the sample had been drawn.

If we were to repeatedly sample the same population over and over and calculate a mean every time, the 95% CI indicates the range that 95% of those means would fall into.

It is used to indicate how accurate a calculated statistic is likely to be. It can be calculated for many statistics: the mean, median, slope of a linear regression, etc. The wider the CIs, the less certainty we can attach to estimate of the statistic in question.

When we draw only a small sample size from the population, the CIs will be wide. But the more samples we take, the narrower the CI band becomes.

There are many ways in which CIs can be calculated. We have already seen CIs a couple of times in the course. Let's start there.

---
## Revisiting *t*-tests

```r
data(iris)
head(iris)
```

```
R>   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
R> 1          5.1         3.5          1.4         0.2  setosa
R> 2          4.9         3.0          1.4         0.2  setosa
R> 3          4.7         3.2          1.3         0.2  setosa
R> 4          4.6         3.1          1.5         0.2  setosa
R> 5          5.0         3.6          1.4         0.2  setosa
R> 6          5.4         3.9          1.7         0.4  setosa
```

---
What is the mean `Sepal.Length` for *Iris setosa*? Is this value significantly different from zero? To do this, we do a one-sample, two-sided *t*-test. The H0 is, "The mean sepal width for *Iris setosa* is 0" (because it is a two-sided test, the mean can be less than 0, or larger than 0).

```r
setosa <- iris %>%
  filter(Species == "setosa")
setosa_t_test <- t.test(setosa$Sepal.Length)
setosa_t_test
```

```
R> 
R> 	One Sample t-test
R> 
R> data:  setosa$Sepal.Length
R> t = 100.42, df = 49, p-value < 2.2e-16
R> alternative hypothesis: true mean is not equal to 0
R> 95 percent confidence interval:
R>  4.905824 5.106176
R> sample estimates:
R> mean of x 
R>     5.006
```

---
We can extract the CI if we want to use it for plots, etc.:

```r
setosa_t_test$conf.int # pull out the CIs only
```

```
R> [1] 4.905824 5.106176
R> attr(,"conf.level")
R> [1] 0.95
```

Same with the mean:

```r
setosa_t_test$estimate
```

```
R> mean of x 
R>     5.006
```

---
## How does the width of the CI vary by sample size?

Let's assume the population is all the data in the `iris` dataset (across all three species). Let's again look at `Sepal.Length`. From the population, let's draw at random 3, 6, 10, 50, and lastly 100 samples. Let's then see how the width of the CI changes:

```r
n <- c(3, 6, 10, 50, 100)
ci_function <- function(x) {
  out <- data.frame(mean = t.test(sample(iris$Sepal.Length, x))$estimate,
                    lower = t.test(sample(iris$Sepal.Length, x))$conf.int[1],
                    upper = t.test(sample(iris$Sepal.Length, x))$conf.int[2])
  return(out)
}
ci_data <- plyr::ldply(n, ci_function)
ggplot(data = ci_data, aes(x = n)) +
  geom_segment(aes(x = n, xend = n, y = lower, yend = upper), size = 1) +
  geom_point(aes(y = mean), colour = "red3", size = 2, shape = 2) + ylab("Mean ± 95% CI")
```

---
## Other ways to calculate CIs for the mean

**Example 1:** Normal data not belonging to any groups. We apply the 'traditional' method (i.e. not bootstrapped). Let's use the `iris` data again. The `~ 1` indicates that we ignore the grouping arrangement of the data:

```r
library(rcompanion)
groupwiseMean(Sepal.Length ~ 1, data = iris, conf = 0.95, digits = 3)
```

```
R>    .id   n Mean Conf.level Trad.lower Trad.upper
R> 1 <NA> 150 5.84       0.95       5.71       5.98
```

**Example 2:** Still staying with normal data, we may also want to consider the `iris` data's grouping (`Species`):

```r
(iris_grps <- groupwiseMean(Sepal.Length ~ Species, data = iris, conf = 0.95, digits = 3))
```

```
R>      Species  n Mean Conf.level Trad.lower Trad.upper
R> 1     setosa 50 5.01       0.95       4.91       5.11
R> 2 versicolor 50 5.94       0.95       5.79       6.08
R> 3  virginica 50 6.59       0.95       6.41       6.77
```

---
Continuing with the grouped data example, we may also use the 95% CIs to see if there are significant differences in `Sepal.Length` between the three `Species`. You can interpret the table, above, directly, or we can make a graph:

```r
ggplot(data = iris_grps, aes(x = Species)) +
  geom_segment(aes(x = Species, xend = Species, y = Trad.lower, yend = Trad.upper), size = 1) +
  geom_point(aes(y = Mean), colour = "red3", size = 2, shape = 2) + ylab("Mean ± 95% CI")
```

We may interpret the non-overlapping 95% CIs as showing that there are statistically significant differences in Sepal Length between the three *Iris* species.

Looking at the help file (`?groupwiseMean`) we see that we may also apply this function for two-way (two grouping variables) data, as which you would require for a 2-way ANOVA, for example.

---
**Example 3:** For skewed data (non-normal), confidence intervals by bootstrapping may be more reliable. We can achieve this by setting various options in the `groupwiseMean()` function:

```r
groupwiseMean(Sepal.Length ~ Species, data = iris, conf = 0.95, digits = 3,
              R = 10000, boot = TRUE, traditional = FALSE, normal = FALSE,
              basic = FALSE, percentile = FALSE, bca = TRUE)
```

```
R>      Species  n Mean Boot.mean Conf.level Bca.lower Bca.upper
R> 1     setosa 50 5.01      5.01       0.95      4.91      5.10
R> 2 versicolor 50 5.94      5.94       0.95      5.79      6.08
R> 3  virginica 50 6.59      6.59       0.95      6.41      6.76
```

---
.left-column[
# Questions
]

.right-column[
1. Using the Lung Capacity data, please calculate the 95% CIs for the `LungCap` variable as a function of:

a. `Gender`
    
    b. `Smoke`
    
    c. `Caesarean`
    
2. Create a graph of the mean ± 95% CIs (as I have done in the above example), and determine if there are statistical differences in `LungCap` between the levels of `Gender`, `Smoke`, and `Caesarean`. Do the same using a *t*-test. Are your findings the same using these two approaches?

3. Produce all the associated tests for assumptions -- i.e. the assumptions to be met when deciding on which *t*-test to use, or whether you should apply the 'traditional' or bootstrapping approach in the calculation of CIs.

]