Bayes theorem explained with real examples

#Why it matters

Bayes theorem is how you turn "how often does the test fire on fraud?" into the question the business actually asks: "this transaction was flagged, how likely is it to be fraud?" Fraud teams at Indian fintechs, spam filters, medical screening and every alert system live or die on that difference. It is also one of the most asked probability questions in Data Scientist interviews (statistics appears in 82% of the Data Scientist job descriptions we read), usually in the form of the disease-test puzzle below, and most candidates get it wrong by a factor of ten.

#The concept

#The formula and its three parts

• $H$ is a hypothesis ("this transaction is fraud"), $E$ is the evidence ("the detector flagged it").

• Prior $P(H)$: how likely H was before the evidence. For fraud, the base rate.

• Likelihood $P(E\mid H)$: how likely this evidence is if H is true. For a detector, its sensitivity.

• Posterior $P(H\mid E)$: how likely H is after seeing the evidence. What you want.

• The denominator $P(E)$ is the total chance of seeing the evidence, from both true and false causes: $P(E)=P(E\mid H)P(H)+P(E\mid \text{not }H)P(\text{not }H)$.

Bayes is just the definition of conditional probability from the previous lesson, Probability basics: rules and conditional probability, applied in both directions: $P(H\text{ and }E)$ written once as $P(E\mid H)P(H)$ and once as $P(H\mid E)P(E)$.

#Sensitivity, specificity and positive predictive value

For any yes/no detector (a medical test, a fraud rule, a spam model):

Sensitivity and specificity are properties of the test. PPV depends on the test and on the base rate. The same detector has a high PPV in a population where the condition is common and a terrible one where it is rare.

#The base rate fallacy

Ignoring the prior and treating sensitivity as if it were PPV is the base rate fallacy. When the thing you are looking for is rare, even a small false positive rate applied to the huge negative population produces more false alarms than there are true cases. That is why rare-event detectors (fraud, rare disease, intrusion detection, defaulting loans) produce mostly false alarms unless their false positive rate is far below the base rate.

#Natural frequencies: how to explain it to anyone

Probabilities confuse people; counts do not. Rewrite the problem as "out of 100,000 transactions":

• 100 are fraud (0.1%). The detector catches 99 of them.

• 99,900 are genuine. 1% of them, 999, are flagged anyway.

• So 99 + 999 = 1,098 flags, of which 99 are fraud: 9%.

Gigerenzer's research showed doctors answer this kind of question far better in natural frequencies than in percentages. Use them in every stakeholder deck.

#Updating step by step

Today's posterior is tomorrow's prior. If a flagged transaction then fails a second, independent check, you run Bayes again starting from 9% instead of 0.1%. Each piece of evidence multiplies the odds by its likelihood ratio, $P(E\mid H)/P(E\mid \text{not }H)$:

The odds form is the quickest way to do Bayes in your head. Odds are $p/(1-p)$; a likelihood ratio of 99 (99% sensitivity over 1% false positive rate) turns 1:999 into 99:999, which is 9%.

#Naive Bayes in one paragraph

A naive Bayes classifier applies this odds update once per feature, multiplying likelihood ratios as if the features were independent given the class. The "naive" assumption is almost always false, so its probabilities are badly calibrated, but its ranking of cases is often good enough for spam filtering and text classification. Training and tuning such models belongs to Machine learning; below you will see exactly how the independence assumption over-states a probability on real data.

#Bayesian and frequentist, in plain language

• A frequentist treats probability as long-run frequency. Parameters (the true conversion rate) are fixed; data is random. Tools: p-values, confidence intervals.

• A Bayesian treats probability as a degree of belief. Parameters get a prior distribution, updated with data into a posterior. Tools: posterior distributions, credible intervals.

Bayes theorem is accepted by both; it is plain probability. The disagreement is about whether you may put a prior on an unknown parameter. Most of this skill uses frequentist tools because that is what A/B testing platforms and interviewers mostly expect, but several experimentation platforms report Bayesian "probability to beat control" numbers, so recognise both.

#Worked example: the 99% fraud detector

#Step 1: compute PPV with Bayes

A detector catches 99% of fraud (sensitivity 0.99) and flags 1% of genuine transactions (false positive rate 0.01). Fraud is 0.1% of transactions.

import numpy as np
import pandas as pd

def posterior(prior, sensitivity, false_positive_rate):
    """P(positive | flagged) by Bayes theorem."""
    p_flag = sensitivity * prior + false_positive_rate * (1 - prior)
    return sensitivity * prior / p_flag

ppv = posterior(prior=0.001, sensitivity=0.99, false_positive_rate=0.01)
print(f"P(fraud | flagged) = {ppv:.4f}")

for prior in [0.0001, 0.001, 0.01, 0.05, 0.2, 0.5]:
    print(f"base rate {prior:>7.2%}  ->  PPV {posterior(prior, 0.99, 0.01):.1%}")

P(fraud | flagged) = 0.0902
base rate   0.01%  ->  PPV 1.0%
base rate   0.10%  ->  PPV 9.0%
base rate   1.00%  ->  PPV 50.0%
base rate   5.00%  ->  PPV 83.9%
base rate  20.00%  ->  PPV 96.1%
base rate  50.00%  ->  PPV 99.0%

Only 9% of flags are fraud. The same detector is 98% right when it fires in a population where half the cases are fraud, and under 1% right at a base rate of 1 in 10,000. When someone quotes you an accuracy, ask "at what base rate?"

#Step 2: verify by simulating a million transactions

rng = np.random.default_rng(2024)
n = 1_000_000
is_fraud = rng.random(n) < 0.001
flagged = np.where(is_fraud, rng.random(n) < 0.99, rng.random(n) < 0.01)

print(f"transactions: {n:,}")
print(f"fraud: {is_fraud.sum():,}   flagged: {flagged.sum():,}")
print(f"flagged and fraud: {(flagged & is_fraud).sum():,}   flagged but genuine: {(flagged & ~is_fraud).sum():,}")
print(f"simulated P(fraud | flagged) = {(flagged & is_fraud).sum() / flagged.sum():.4f}")
print(f"accuracy of the detector     = {(flagged == is_fraud).mean():.4f}")

transactions: 1,000,000
fraud: 1,025   flagged: 10,998
flagged and fraud: 1,013   flagged but genuine: 9,985
simulated P(fraud | flagged) = 0.0921
accuracy of the detector     = 0.9900

The detector is "99% accurate" by the accuracy metric, and nine out of ten of its alerts are false. If each flag blocks a customer's UPI payment, you are annoying about ten genuine customers for every fraud you stop. That is a product decision, and it depends entirely on the base rate.

#Step 3: update with a second, independent check

Suppose every flagged transaction goes to a second check (a call to the customer) with sensitivity 0.95 and false positive rate 0.02, independent of the first.

step1 = posterior(0.001, 0.99, 0.01)
step2 = posterior(step1, 0.95, 0.02)
print(f"prior 0.10%  -> after flag {step1:.1%}  -> after second check {step2:.1%}")

def odds(p):
    return p / (1 - p)

lr1, lr2 = 0.99 / 0.01, 0.95 / 0.02
post_odds = odds(0.001) * lr1 * lr2
print(f"odds form: prior odds {odds(0.001):.5f} x LR {lr1:.0f} x LR {lr2:.1f} = {post_odds:.3f} -> p = {post_odds / (1 + post_odds):.1%}")

prior 0.10%  -> after flag 9.0%  -> after second check 82.5%
odds form: prior odds 0.00100 x LR 99 x LR 47.5 = 4.707 -> p = 82.5%

Two independent pieces of evidence take you from 0.1% to 82%. This is why real fraud systems stack weak signals and why a second-step verification is cheaper than blocking on the first flag. The odds form gives the same answer with two multiplications.

#Step 4: Bayes on real data, UCI Bank Marketing

The same logic works on any table. In the UCI Bank Marketing calls (CC BY 4.0, introduced in Probability basics: rules and conditional probability), the base rate of subscription is the prior. Evidence: the client said yes to the bank's previous campaign.

url = "https://raw.githubusercontent.com/selva86/datasets/master/bank-additional-full.csv"
bank = pd.read_csv(url, sep=";")
yes = bank["y"] == "yes"
prev_success = bank["poutcome"] == "success"
cellular = bank["contact"] == "cellular"

prior = yes.mean()
sens = prev_success[yes].mean()          # P(previous success | subscribed)
fpr = prev_success[~yes].mean()          # P(previous success | not subscribed)
print(f"prior P(subscribe)              = {prior:.4f}")
print(f"P(prev success | subscribed)    = {sens:.4f}")
print(f"P(prev success | not subscribed) = {fpr:.4f}")
print(f"Bayes: P(subscribe | prev success) = {posterior(prior, sens, fpr):.4f}")
print(f"direct count                       = {yes[prev_success].mean():.4f}")

prior P(subscribe)              = 0.1127
P(prev success | subscribed)    = 0.1927
P(prev success | not subscribed) = 0.0131
Bayes: P(subscribe | prev success) = 0.6511
direct count                       = 0.6511

Bayes reproduces the direct count exactly (it has to: it is the same arithmetic). The value of the formula is that you can reason with rates reported separately, as you often get them: "19% of subscribers had a previous success, against 1.3% of non-subscribers, and overall conversion is 11%."

#Step 5: what the naive independence assumption does

Now add a second piece of evidence, the cellular contact, and update the odds naively, as a naive Bayes classifier would:

lr_prev = sens / fpr
lr_cell = cellular[yes].mean() / cellular[~yes].mean()
naive_odds = odds(prior) * lr_prev * lr_cell
print(f"LR previous success = {lr_prev:.2f}, LR cellular = {lr_cell:.2f}")
print(f"naive Bayes P(subscribe | prev success, cellular) = {naive_odds / (1 + naive_odds):.4f}")
both = prev_success & cellular
print(f"direct count (n = {both.sum():,})                    = {yes[both].mean():.4f}")

LR previous success = 14.70, LR cellular = 1.36
naive Bayes P(subscribe | prev success, cellular) = 0.7176
direct count (n = 1,270)                    = 0.6520

Naive multiplication says 72%; the data says 65%. Clients with a previous success were almost all (1,270 of 1,373) called on mobiles already, so "cellular" adds little information once you know about the previous success. The two pieces of evidence are not independent, and multiplying their likelihood ratios double counts. Remember this whenever a model's probabilities look too confident.

#Common mistakes

• Treating sensitivity as PPV. "The test catches 99% of cases" says nothing about how many alerts are real. You need the base rate.

• Quoting accuracy for rare events. A detector that never fires is 99.9% accurate at a 0.1% base rate. Report precision (PPV) and recall (sensitivity) instead.

• Using a base rate from the wrong population. The prior for a transaction already flagged by another rule is not the overall fraud rate. Ask which population this case came from.

• Multiplying likelihood ratios of correlated evidence. Step 5 over-stated the probability by 7 percentage points.

• Explaining with percentages to non-technical people. Use natural frequencies: "of 100,000 transactions, 1,098 get flagged and 99 of those are fraud."

#Interview angle

#Practice task

1. A spam filter catches 95% of spam and wrongly flags 2% of genuine email. Compute the PPV at spam rates of 10%, 50% and 80%. Write each answer as a natural frequency sentence.

2. Simulate step 1 with 500,000 emails at the 10% rate and confirm your PPV.

3. In the Bank Marketing data, use Bayes to compute P(subscribe | job = "student") from the three rates (prior, P(student | yes), P(student | no)) and check it against the direct count.

4. Compute the naive Bayes estimate of P(subscribe | student and cellular) and compare it with the direct count. Is the independence assumption reasonable here?

Done when each probability matches between the formula and the simulation or count, and you have one sentence explaining any gap.

for p in [0.10, 0.50, 0.80]:
    ppv = posterior(p, 0.95, 0.02)
    flagged_spam, flagged_ham = 1000 * p * 0.95, 1000 * (1 - p) * 0.02
    print(f"spam {p:.0%}: PPV {ppv:.1%}  ({flagged_spam:.0f} of {flagged_spam + flagged_ham:.0f} flagged per 1,000 emails are spam)")

student = bank["job"] == "student"
print("Bayes :", round(posterior(yes.mean(), student[yes].mean(), student[~yes].mean()), 4))
print("direct:", round(yes[student].mean(), 4))

lr_s = student[yes].mean() / student[~yes].mean()
o = odds(yes.mean()) * lr_s * lr_cell
print("naive:", round(o / (1 + o), 4), " direct:", round(yes[student & cellular].mean(), 4))

#Further reading

• Gigerenzer and Hoffrage, How to improve Bayesian reasoning without instruction (Psychological Review, 1995): the natural frequencies research.

• Blitzstein and Hwang, Introduction to Probability, chapter 2: conditional probability and Bayes with many worked problems.

• scikit-learn naive Bayes guide: how the independence assumption is used in practice.

• Google Machine Learning Crash Course: precision and recall: the same quantities as sensitivity and PPV, in ML language.