Visualizing An Ad's Confidence Interval with Python and Statistics
Have you ever wondered how to analyze confidence intervals for an ad's CTR or Conversion Rate? In this video, I'll show you how in Python.
To start our ad statistical analysis, we'll calculate a basic proportion of successes and trials.
Then we'll determine our level of confidence for our confidence interval. Since this is an interval, that means there will be a lower bound and an upper bound. The z score is what determines how many standard deviations the confidence interval is from the proportion's mean.
The standard error is a calculated.
Finally, we calculate +- zscore*standard_error from the mean. This is our interval. An interval is a range of values that we expect 95% of values to fall within.
The steeper the interval and the more narrow the base, the more confident we are about an ad's confidence interval. That usually means we have accumulated a lot of data (impressions, clicks, conversions) on that ad.
The wider the interval's base is, the less certain we are about that ad's true mean. Remember: we are trying to estimate the interval where the true mean likely exists.