Confidence intervals are frequently misunderstood and misinterpreted. Even professional scientists can get the intepretation wrong. A 95% confidence interval does NOT mean that there is a 95% chance that the true value of the parameter lies inside the interval. The parameter's value is fixed, so once a confidence interval is calculated, it either contains the true value or it doesn't. It is not a matter of probability.
The correct interpretation of a 95% confidence interval is as follows: If we repeated this experiment many times, the intervals we calculate would contain the true parameter value 95% of the time.
This app is designed to demonstrate this fact. It generates random samples from a Normal distribution. You can select the true mean and standard deviation parameters, as well as the number of samples and the size of each sample. The app will then create confidence intervals for each sample. As you change the confidence level, notice the relationship between the confidence level and the number of CIs that contain the true mean.
In "Number of Samples", select the number of times to repeat the experiment. In "Sample Size", select the number of values to take in each sample. "Mean" and "Standard Deviation" allow you to set the mean and standard deviation, respectively, of the population distribution. "Confidence level" is the value of 1- α used in the CI calculations.
The plot shows all of the confidence intervals. Blue lines are intervals which do contain the true mean, and orange lines are ones that do not. We would expect about α % of the intervals to be orange, although due to randomness this will not always exactly be the case. Hover over each sample for more information about it.
Click on the "Visualization" tab above to go to the app. If you found it useful, please share it on Facebook or Twitter using the buttons in the top right!