Statistical science develops and applies methods for collecting, analysing and interpreting data. 1. Design: planning how to gather relevant data. 2. Description: summarising the data. 3. Inference: making evaluations, such as estimations and predictions, based on the data.

The population is the total set of subjects of interest. A sample is the set of subjects from the population for which data are available.

Example: what percentage of voters approve of the French President? There are over 40 million voters, but randomly sampling 1,000 of them gives an approval percentage that estimates the percentage among all voters.

Population: the entire group (all French voters). Parameter: the characteristic of the population we want (approval among all voters). Sample: the subset we collect data from (1,000 voters). Statistic (estimate): the measure derived from the sample (approval among the sampled voters). Key point: even a relatively small sample (100 or 1,000) can closely approximate the parameter of a population of 40 million.

Ideally the sample is chosen by simple random, stratified, systematic or cluster sampling, not convenience sampling (s20).

Designs are fundamentally distinguished as nonexperimental or experimental.

Nonexperimental design: no treatments are administered. Descriptive when little is known about a phenomenon; explorative or explanatory when relationships between factors are explored or explained.

Experimental design: treatments are administered. Predictive when the study tests predictive questions to address effectiveness or possible cause-and-effect.

Examples: descriptive, a university surveys every engineering student about daily screen time, just to report the average. Explorative, researchers check existing records to see whether sleep hours seem related to GPA. Explanatory, analysts use hospital records to examine whether smoking history helps explain lung-cancer rates. Experimental, students are randomly assigned to spaced repetition vs cramming and their scores compared.

Response variable (dependent, outcome): the variable whose value we are trying to predict or explain. Explanatory variable (independent, predictor): a variable used to explain or predict changes in the response. It is a role, not a fixed label: on a scatterplot the explanatory variable sits on the x-axis, the response on the y-axis, but the same variable can be a response in one study and explanatory in another.

Independent variable: deliberately changed or manipulated by the researcher to observe its effect (the study method, in the spaced-repetition experiment). Spot it by asking "what is the researcher changing or assigning?" That control over assignment makes cause-and-effect conclusions possible.

Dependent variable: the outcome measured to see whether it responds to the independent variable (the exam score). Ask "what result am I measuring?"

Control variable: held constant across all groups so it cannot distort the relationship (same exam, same study time, same room and time of day). An uncontrolled variable can become a confounding variable, an alternative explanation: if the cramming group got an easier exam, we could not tell whether the score difference came from the method or the exam.

Worked example (s10): a survey records exercise frequency, sleep, stress, diet, age and academic performance. Way 1: "what affects academic performance?" Response: performance; explanatory: the other five. Way 2: "what influences stress?" Response: stress; explanatory: the others, now including academic performance.

Association: any relationship (positive, negative or none) between two variables, regardless of type. Measured by chi-square, Cramér's V or odds ratios for categorical data, by correlation, regression or scatterplots for quantitative data. Examples: smoking and lung cancer; car colour and being stopped by police; ice-cream sales and temperature.

Correlation: a numerical measure (usually Pearson's \(r\)) of the strength and direction of a linear relationship between two quantitative variables. Range −1 to +1: +1 perfect positive, −1 perfect negative, 0 no linear correlation. Height and weight about +0.8; study hours and exam score about +0.6; temperature and hot-chocolate sales about −0.7.

\[ r = \frac{\sum (x_i - \bar x)(y_i - \bar y)}{\sqrt{\sum (x_i - \bar x)^2 \cdot \sum (y_i - \bar y)^2}} \]

The numerator captures how \(x\) and \(y\) move together; the denominator rescales it to fall between −1 and +1.

All correlations are associations, but not all associations are correlations.

Putting it all together (s20): design → description → inference; a parameter describes the population and a statistic estimates it from a sample; nonexperimental studies observe, experimental studies assign; association is any relationship, correlation the specific linear measure; \(\bar x\) estimates \(\mu\) and \(s\) (divided by \(n - 1\)) estimates \(\sigma\).

Frequentist definition: the probability of an outcome is the proportion of times it would occur in an indefinitely long sequence of like observations under the same conditions. A forecast of 0.20 means rain occurs on 20% of days with identical atmospheric conditions.

Subjective definition: the degree of belief that the outcome will occur, based on all the available information. Useful when no long run of identical trials exists (intelligent life elsewhere, a new business succeeding). This is the foundation of Bayesian statistics. With enough data the two usually converge.

A fair coin: \(P(\text{Heads}) = 0.5\), \(P(\text{Tails}) = 0.5\). Three heads in a row does not mean the coin always comes up heads. Kerrich, interned in Denmark in the 1940s, got 5,067 heads in 10,000 flips.

Why statisticians need probability: descriptive statistics summarise data we have; inferential statistics draw conclusions about a population from a sample, and every such conclusion carries uncertainty. Probability is the language for that uncertainty: it estimates how far a statistic is likely to fall from the true value, quantifies risk, and is the machinery behind hypothesis tests, confidence intervals and regression.

The sample space \(S\) is the set of all possible outcomes of a random experiment. An event \(A\) is any subset of the sample space.

One fair coin: \(S = \{H, T\}\), event "heads" \(= \{H\}\). One die: \(S = \{1, 2, 3, 4, 5, 6\}\), event "even" \(= \{2, 4, 6\}\).

Ask three people whether they support a government action (yes/no): \(S\) has 8 outcomes, from (yes, yes, yes) to (no, no, no). Event \(A\) = "all three give the same response" \(= \{(\text{yes,yes,yes}), (\text{no,no,no})\}\). Event \(B\) = "the first person says yes" has 4 outcomes.

The complement \(A^c\) (also \(\neg A\)) is every point of \(S\) not in \(A\). The intersection \(A \cap B\) (also \(AB\)) is the points in both; here \(A \cap B = \{(\text{yes,yes,yes})\}\). The union \(A \cup B\) is the points in \(A\) or in \(B\) or both; here 5 of the 8 outcomes.

\[ P(A \cup B) = \frac{2}{8} + \frac{4}{8} - \frac{1}{8} = \frac{5}{8} \]

Subtract the overlap, or YYY gets counted twice.

Rule for equally likely outcomes: if a chance experiment has \(n\) equally likely outcomes,

\[ P(A) = \frac{\text{number of outcomes in } A}{n} \]

Coin: \(P(H) = 1/2\). Die: \(P(\text{even}) = 3/6 = 0.50\). This only works when outcomes are equally likely: 48.8% of newborns being girls is not an equally-likely situation, so broader rules are needed.

The three axioms (Kolmogorov, 1933). Every rule of probability follows from these:

\[ \text{1. Non-negativity:}\quad P(A) \ge 0 \] \[ \text{2. Normalisation:}\quad P(S) = 1 \] \[ \text{3. Additivity for disjoint events:}\quad P(A \cup B) = P(A) + P(B) \text{ if } A \cap B = \varnothing \]

Sanity check: every probability lies in \([0, 1]\), and a complete distribution sums to exactly 1.

\[ P(A^c) = 1 - P(A) \]

Since \(P(S) = 1\) and \(A\), \(A^c\) split \(S\) with no overlap, their probabilities add to 1. In 2020 about 50,000 babies were born in Qatar and 48.8% were girls:

\[ P(\text{boy}) = 1 - 0.488 = 0.512 \]

Loaded coins: a coin that always comes up heads has \(P(H) = 1\), \(P(T) = 0\). With \(P(H) = 0.75\), \(P(T) = 1 - 0.75 = 0.25\). Probability law for coins: \(P(H) + P(T) = 1\).

Construction workers (s68): of 12,942 injured, Houston 219, Miami 112, Denver 140. Probability of a city other than those three:

\[ 1 - \frac{219 + 112 + 140}{12{,}942} = 1 - 0.0364 = 0.9636 \]

"At least one": find \(P(\text{none})\) first, then subtract from 1 (review catch-up).

Two events are mutually exclusive if they cannot occur at the same time: they share no outcomes. Rolling a die, \(A = \{1, 6\}\) and \(B = \{2, 3\}\) can never both happen on one roll.

Addition rule, special case (valid only when mutually exclusive):

\[ P(A \text{ or } B) = P(A) + P(B) \] \[ P(1 \text{ or } 6) = \tfrac{1}{6} + \tfrac{1}{6} = \tfrac{1}{3} \]

General addition rule (any two events):

\[ P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B) \]

Adding \(P(A)\) and \(P(B)\) counts the overlap twice, so subtract it once. One card from 52: \(A\) = King (4 cards), \(B\) = Heart (13 cards), the King of Hearts is in both:

\[ P(\text{King or Heart}) = \tfrac{4}{52} + \tfrac{13}{52} - \tfrac{1}{52} = \tfrac{16}{52} = 0.308 \]

Two events are independent if knowing one occurs does not change the probability of the other. Multiplication rule (valid only when independent):

\[ P(A \text{ and } B) = P(A) \times P(B) \] \[ P(HH) = \tfrac{1}{2} \times \tfrac{1}{2} = \tfrac{1}{4} \]

Two dice: \(P(\text{both even}) = \tfrac{1}{2} \times \tfrac{1}{2} = \tfrac{1}{4}\).

Truth tables: two fair flips give HH, HT, TH, TT each with probability 0.25. A loaded coin with \(P(H) = 0.6\): HH 0.36, HT 0.24, TH 0.24, TT 0.16. All outcomes always add up to 1, so you can check your work.

Exactly one head in two fair flips: HT or TH, \(0.25 + 0.25 = 0.5\). Exactly one head in three fair flips: 8 cases each \(1/8\); only HTT, THT, TTH qualify, so \(3 \times 0.125 = 0.375\).

Across multiple flips, the probability of \(n\) heads is \(P(H)^n\), because the events are independent. For dependent events (rain tomorrow given rain today) these rules do not apply.

Mutually exclusive vs independent (s39): mutually exclusive is a question about outcomes (can \(A\) and \(B\) happen on the same trial? then \(P(A \cap B) = 0\)). Independent is a question about information (does knowing \(A\) change the chance of \(B\)? then \(P(A \cap B) = P(A)P(B)\)).

Drawing two balls from 5 red and 5 blue (L4 s26): with replacement the draws are independent, \(0.5 \times 0.5 = 0.25\); without replacement the second draw depends on the first, \(0.5 \times \tfrac{4}{9} = 0.22\).

A factorial is the product of every whole number from 1 to the number before the exclamation mark: \(2! = 2\), \(3! = 6\), \(4! = 24\), \(5! = 5 \cdot 4 \cdot 3 \cdot 2 \cdot 1 = 120\).

\[ n! = n(n-1)(n-2)\cdots 1 \]

Five flips, exactly 1 head: HTTTT, THTTT, TTHTT, TTTHT, TTTTH, 5 outcomes. Exactly 2 heads: the first head can be in 5 positions and the second in 4, but that double counts each pair, so \(5 \cdot 4 / 2 = 10\). Exactly 3 heads: \(5 \cdot 4 \cdot 3 = 60\) placements, divided by the \(3 \cdot 2 \cdot 1 = 6\) duplicate orderings, gives 10.

Ten flips, 4 heads: \(10 \cdot 9 \cdot 8 \cdot 7 = 5{,}040\), divided by \(4 \cdot 3 \cdot 2 \cdot 1 = 24\), gives 210. So 210 of the \(2^{10} = 1{,}024\) outcomes have exactly 4 heads and 6 tails.

Number of ways to get \(k\) heads in \(n\) flips:

\[ \binom{n}{k} = \frac{n!}{(n-k)!\,k!} \]

Probability of one particular sequence with \(k\) heads and \(n - k\) tails:

\[ p^k (1-p)^{n-k} \]

Truth-table size for \(n\) flips is \(2^n\). Probability of exactly 1 head in 5 fair flips: \(5/2^5 = 0.15625\); exactly 3 heads: \(10/32 = 0.3125\). Loaded coin \(P(H) = 0.8\), three flips, exactly 1 head: each qualifying sequence has probability \(0.8 \times 0.2 \times 0.2 = 0.032\), and there are 3, so \(0.096\).

In real life events often depend on each other: the outcome of the first event affects the second (an early bird running at 5 am vs a night owl). A probability of an event \(A\), given that an event \(B\) occurred, is a conditional probability, denoted \(P(A \mid B)\):

\[ P(A \mid B) = \frac{P(A \cap B)}{P(B)},\qquad P(B) \gt 0 \]

Placing \(P(B)\) in the denominator corresponds to conditioning on (restricting to) the sample points in \(B\); the numerator is the fraction of \(B\) that is also in \(A\). The denominator is always the "given" event.

General multiplication rule (L4 s26): the second factor is always conditional; independence just makes it equal \(P(B)\):

\[ P(A \text{ and } B) = P(A) \times P(B \mid A) \]

Test for independence: \(A\) and \(B\) are independent exactly when \(P(B \mid A) = P(B)\).

Sports-car buyers (s79): of 100 buyers, 40 bought an alarm, 30 bucket seats, 20 both. Given an alarm, probability of bucket seats:

\[ P(B \mid A) = \frac{0.2}{0.4} = 0.5 \]

Contingency table (s80): P(male and car) = 0.41, P(car) = 0.86:

\[ P(M \mid CO) = \frac{0.41}{0.86} = 0.477 \]

Language models (L5 s2–4) do the same thing: \(P(\text{next token} \mid \text{previous tokens})\). The context narrows the plausible outcomes, the model picks a token, adds it to the context, and repeats.

Cancer screening (L3 s73–76, L4 s27–32): \(P(C) = 0.1\) (10% of the population has cancer, \(P_0\)); sensitivity \(P(\text{Pos} \mid C) = 0.9\) (\(P_1\)); specificity \(P(\text{Neg} \mid \neg C) = 0.8\) (\(P_2\)). Hence \(P(\neg C) = 0.9\), \(P(\text{Neg} \mid C) = 0.1\), \(P(\text{Pos} \mid \neg C) = 0.2\).

\[ P(\text{Pos}) = P(\text{Pos} \mid C)\,P(C) + P(\text{Pos} \mid \neg C)\,P(\neg C) \] \[ P(\text{Pos}) = 0.9 \times 0.1 + 0.2 \times 0.9 = 0.09 + 0.18 = 0.27 \] \[ P(\text{Neg}) = 0.1 \times 0.1 + 0.8 \times 0.9 = 0.01 + 0.72 = 0.73 \]

Check: \(P(\text{Pos}) + P(\text{Neg}) = 1\). Truth table of the four cases: cancer and positive 0.09, cancer and negative 0.01, no cancer and positive 0.18, no cancer and negative 0.72.

Counting people: start with 1,000. 100 have cancer, of whom 90 test positive and 10 negative. 900 do not, of whom 180 test positive and 720 negative. Positives: \(90 + 180 = 270\). Of those 270, only 90 really have cancer: \(90/270 = 33\%\). The test catches 90% of cancers, so a positive result feels like 90%; it is not, because the 900 healthy people produce 180 false positives, twice the 90 true ones. When a disease is rare, false positives outnumber true ones.

\[ P(H \mid E) = \frac{P(E \mid H)\,P(H)}{P(E)} \]

\(P(H \mid E)\): posterior, the updated probability of the hypothesis after the evidence; the answer you want. \(P(E \mid H)\): likelihood, how likely the evidence is if the hypothesis is true; usually given. \(P(H)\): prior, what you believed before, often the base rate in the population. \(P(E)\): marginal likelihood or evidence, the total probability of the evidence across all hypotheses; the total on the bottom.

Cancer with 1% prevalence, 90% sensitive, 90% specific (L4 s35, s39–40):

\[ P(+) = 0.9 \times 0.01 + 0.1 \times 0.99 = 0.108 \] \[ P(C \mid +) = \frac{0.9 \times 0.01}{0.108} = 0.083 \]

Follow 1,000 people: 10 have cancer, 9 test positive; 990 are healthy, 99 test positive; \(9/108 = 8\%\). A positive result multiplies the odds by about eight, but eight times a very small number is still small. Screening programmes retest rather than diagnose on one result.

Swapping the condition changes the answer completely (L4 s41): \(P(\text{positive} \mid \text{cancer}) = 90\%\) counts inside the 10 people with cancer; \(P(\text{cancer} \mid \text{positive}) = 8\%\) counts inside the 108 who tested positive. Same 9 people in the numerator, different denominator. Courtrooms slip the same way: "the chance of this DNA match if he were innocent is one in a million" is \(P(\text{match} \mid \text{innocent})\), not \(P(\text{innocent} \mid \text{match})\).

Spam (L4 s42–43): \(P(\text{spam}) = 0.2\), \(P(\text{"win"} \mid \text{spam}) = 0.7\), \(P(\text{"win"}) = 0.18\):

\[ P(\text{spam} \mid \text{"win"}) = \frac{0.7 \times 0.2}{0.18} = 0.78 \]

Medical diagnostics (L3 s87): 1% have the disease, 95% of them test positive, overall 5% test positive: posterior \(0.95 \times 0.01 / 0.05 = 0.19\). Fraud (L4 s44): \(P(F) = 0.01\), \(P(\text{unusual} \mid F) = 0.8\), \(P(\text{unusual}) = 0.05\): \(0.8 \times 0.01 / 0.05 = 0.16\).

Three steps whether diagnosing, filtering email or reading an A/B test: prior (what you believed), likelihood (how likely the evidence would be if true), posterior (what you believe now).

What percentage of students have cheated in an exam? Students may be too embarrassed to answer truthfully. Each student tosses a coin twice. Tails on the first toss: answer Q1 "have you ever cheated on an exam in college?" Heads: answer Q2 "did you get tails on the second toss?" A "yes" can come from a cheater answering Q1 or from an honest student whose second toss was tails; the lecturer cannot tell which, so there is no reason to lie.

With \(\pi\) the proportion of cheaters:

\[ P(\text{yes}) = \tfrac{1}{2}\pi + \tfrac{1}{4} \] \[ \pi = 2\,P(\text{yes}) - \tfrac{1}{2} \]

In one survey 27 said yes and 30 said no:

\[ \hat\pi = 2 \times \frac{27}{57} - 0.5 = 2(0.474) - 0.5 = 0.45 \]

About 45% of the class had cheated, a number no direct question would have produced. Adding known randomness to each answer costs nothing at the group level, because you know exactly how much noise you added.