R on Marco Guerra

Can a Baby Learn? Beliefs and Certainty

marcoaurelioguerrap@gmail.com (Marco Aurélio Guerra Pereira) — Thu, 03 Sep 2026 10:00:00 -0300

Part II: Bayesian

Continuing from the last part, let’s now model the kid’s behaviour using a Bayesian approach. Instead of a Q-value updated by prediction error, she now keeps a whole distribution over her belief: not just what she thinks, but how sure she is. She also holds a belief about how much to trust the evidence that shifts that belief.

Here is the same problem: a tempting chair adventure, where a father wants to curb the kid’s willingness. Her belief is about her father: what percentage of the time does he actually say “No” when she tries? She places a Beta prior on $\theta = P(\text{says No})$, started with a small prior strength $K$ (initial stubbornness, prior_strength) and a starting optimism: she believes the father never intervenes, so her desire to climb opens at the full fun value $v_{fun} = 10$. Same starting point as before ($Q_0 = 10$).

$$\theta \sim \text{Beta}(a, b), \qquad a_0 = \text{initial\_belief} \cdot K, \quad b_0 = (1 - \text{initial\_belief}) \cdot K,$$

with initial_belief = 0 in the climbing task (and a neutral $50/50$ when a new task starts in Section 4, mirroring the companion post’s $Q_0 = 0$ there). Read literally, $\text{Beta}(0, K)$ is not a proper distribution. It is a point belief (“he never says No”) serving as a starting position; the first updates move her off the boundary at once.

Importantly, the child does not take all evidence at face value. The credibility weight $z = \frac{n}{n + K}$ determines how much of an event she takes in. $K$ represents the stubbornness factor; a large $K$ means the kid learns little from the evidence: with $K = 3$, $z$ is still only $0.985$ after two hundred attempts. What saturates quickly is the desire curve, not the weight: her belief is most sensitive when she knows the least, so desire moves fastest at the first trials and flattens asymptotically, close to the curves of the previous part:

$$a \leftarrow a + z \cdot [\text{No}], \qquad b \leftarrow b + z \cdot [\text{Yes}],$$

where $[\text{No}]$ is $1$ when the father intervenes on that attempt and $[\text{Yes}]$ is $1$ when he lets her climb. Note that $n$ counts the attempts before the current one, so the very first event arrives at $z = 0$: she needs a moment of history before anything registers.

The desire to climb is the expected payoff under the posterior, with the same targets as before ($v_{fun} = 10$, $c_{penalty} = 20$):

$$\text{desire} = v_{fun} - c_{penalty} \cdot \mathbb{E}[\theta]$$

(Section 2 will add a severity factor that multiplies the cost when harshness enters the picture; with no harshness expected it is $1$, so this is the form that matters in this section.)

And because the belief is a distribution, the posterior also carries a second number everywhere: the posterior standard deviation, i.e. how sure the child is. Where the Q-learning child had a step size, this child has a belief and its precision.

library(ggplot2)
library(dplyr)
library(tidyr)

simulate_bayesian_credibility <- function(
    trials = 200,
    phase1 = 200,
    new_task = FALSE,
    prior_strength = 3,
    initial_belief = 0,
    new_task_belief = 0.5,
    new_task_prior_strength = 10,
    harsh_weight = 7,
    no_prob = 1,
    harsh_prob = 0,
    harsh_severity = 3,
    stubbornness_growth = 50,
    v_fun = 10,
    c_penalty = 20,
    kappa_a0 = 0.1,
    kappa_b0 = 9.9
) {
  K <- prior_strength
  harsh_seen <- 0
  history <- data.frame()
  for (t in 1:(if (new_task) 2 * trials else trials)) {
    if (t == 1) {
      a <- initial_belief * prior_strength
      b <- (1 - initial_belief) * prior_strength
      ca <- kappa_a0
      cb <- kappa_b0
    }
    if (new_task && t == phase1 + 1) {
      a <- new_task_belief * new_task_prior_strength
      b <- (1 - new_task_belief) * new_task_prior_strength
      ca <- kappa_a0
      cb <- kappa_b0
    }
    if (t <= phase1) {
      intervene <- ifelse(runif(1) < no_prob, 1, 0)
      harsh_event <- ifelse(intervene == 1 & runif(1) < harsh_prob, 1, 0)
      z <- (t - 1) / (t - 1 + K)
      if (harsh_event == 1) {
        a <- a + z
        ca <- ca + harsh_weight * z * z
        harsh_seen <- harsh_seen + 1
      } else if (intervene == 1) {
        a <- a + z
        cb <- cb + z
      } else {
        b <- b + z
      }
      K <- prior_strength + stubbornness_growth * harsh_seen
      theta <- a / (a + b)
      kappa <- ca / (ca + cb)
      sd <- sqrt(a * b / ((a + b)^2 * (a + b + 1)))
      desire <- v_fun - c_penalty * theta * (1 + (harsh_severity - 1) * kappa)
      z2 <- NA
    } else if (new_task) {
      z2 <- (t - phase1 - 1) / (t - phase1 - 1 + K)
      b <- b + z2
      theta <- a / (a + b)
      kappa <- NA
      sd <- sqrt(max(a, 1) * b / ((a + b)^2 * (a + b + 1)))
      desire <- v_fun - c_penalty * theta
    }
    history <- rbind(history, data.frame(
      trial = t, theta = theta, sd = sd, desire = desire,
      kappa = kappa, harsh = harsh_seen,
      z = ifelse(is.na(z2), (t - 1) / (t - 1 + K), z2)
    ))
  }
  return(history)
}

1. Consistent vs. Inconsistent Father

Same as in part one, we have two fathers: one that says “No” every time and another that says “No” $70\%$ of the time. The kids also have 200 attempts each.

set.seed(101)
consistent <- simulate_bayesian_credibility(trials = 200, phase1 = 200, no_prob = 1, harsh_prob = 0)
inconsistent <- simulate_bayesian_credibility(trials = 200, phase1 = 200, no_prob = 0.7, harsh_prob = 0)

belief_data <- bind_rows(
  consistent %>% mutate(regime = "consistent (100% No)"),
  inconsistent %>% mutate(regime = "inconsistent (70% No)")
) %>%
  mutate(
    desire_lo = 10 - 20 * (theta + 1.96 * sd),
    desire_hi = 10 - 20 * (theta - 1.96 * sd)
  )

ggplot(belief_data, aes(x = trial, y = desire, color = regime)) +
  geom_ribbon(aes(ymin = desire_lo, ymax = desire_hi, fill = regime, group = regime), alpha = 0.12, color = NA) +
  geom_line(linewidth = 1) +
  geom_hline(yintercept = -10, linetype = "dashed", color = "grey60") +
  geom_hline(yintercept = 0, linetype = "dotted", color = "grey60") +
  annotate("text", x = 2, y = -10.9, label = "healthy result: -10", hjust = 0, color = "grey35", size = 3) +
  annotate("text", x = 190, y = 0.5, label = "prone above / not prone below", hjust = 1, color = "grey35", size = 3) +
  geom_text(data = belief_data %>% group_by(regime) %>% slice_tail(n = 1),
            aes(x = 203, label = round(desire, 1)), hjust = 0, size = 3.2, show.legend = FALSE) +
  scale_x_continuous(limits = c(0, 218)) +
  scale_y_continuous(limits = c(-13, 19)) +
  scale_color_manual(values = c(
    "consistent (100% No)" = "#2E86AB",
    "inconsistent (70% No)" = "#A23B72"
  )) +
  scale_fill_manual(values = c(
    "consistent (100% No)" = "#2E86AB",
    "inconsistent (70% No)" = "#A23B72"
  )) +
  labs(
    title = "Consistent vs. Inconsistent Feedback: Belief and Credible Band",
    subtitle = "Lines are the expected desire under the posterior; bands are +/- 1.96 posterior SDs.",
    x = "Number of Attempts",
    y = "Desire to Climb (Expected Payoff)",
    color = "Father",
    fill = "Father"
  ) +
  theme_minimal()

The labels at the end of each line show the settled desire.

Both children learn the right point: the consistent kid’s belief converges towards the always-“No” father. This means the desire converges to -9.7, slightly off the “healthy no” line of $-10$. This happens because of the finite credibility weight; the belief approaches certainty in the limit, never quite arriving (after 200 tries her $\theta$ sits at 0.984). The inconsistent kid’s belief approaches the true $70\%$ (her $\theta$ sits at 0.683), and her desire lands on -3.7, finishing close to $-4$, same as the previous text.

Note the band. The consistent child’s posterior shrinks to ±0.02 (sd = 0.00903); the inconsistent child’s band settles about 3.7 times wider, ±0.07 (sd = 0.0337), so she never gets a clean read on her father. The Q-learning child in the companion post wobbled because the signal was noise. The Bayesian child’s line is smoother, but her uncertainty tells the same story: an inconsistent father keeps the kid from fully internalizing the lesson.

2. Adding Harsh Punishment

Now the father always intervenes, but rarely, with probability $\varepsilon = 0.01$, the “No” is harsh, at $s = 3$ times the cost. The child holds a second small belief: what fraction of “No"s are the harsh kind, $\kappa \sim \text{Beta}(c_a, c_b)$, starting near zero. Every plain “No” adds a small count to $c_b$; a harsh “No” is counted harsh_weight = 7 times harder than a plain one, but it arrives through the same credibility filter, so its effective weight is harsh_weight · z². (A harsh “No” also counts as a regular “No” for $\theta$.) The harsh event must pass two gates: it registers only at the credibility volume $z$, and its severity is counted through the same trust: the $7\times$ attention bonus amplifies admitted evidence, not raw events:

$$c_a \leftarrow c_a + \text{harsh\_weight} \cdot z^2 \cdot [\text{harsh No}], \qquad c_b \leftarrow c_b + z \cdot [\text{plain No}].$$

The expected payoff is then

$$\text{desire} = v_{fun} - c_{penalty} \cdot \mathbb{E}[\theta] \cdot \big(1 + (s - 1)\, \mathbb{E}[\kappa]\big).$$

Why $1 + (s-1)\,\kappa$?

The child holds two beliefs: $\theta = P(\text{the father intervenes})$ and $\kappa = P(\text{harsh} \mid \text{intervenes})$. The expected payoff is:

Outcome Probability (P(Outcome)) Payoff (r)

(a) No intervention $1 - \theta$ $v_{fun}$

(b) Plain “No” $\theta\,(1 - \kappa)$ $v_{fun} - c_{penalty}$

(c) Harsh “No” $\theta\,\kappa$ $v_{fun} - s\,c_{penalty}\ \ (s = 3)$

Expected payoff:
$$\mathbb{E}[r] = P(a) \cdot r_a + P(b) \cdot r_b + P(c) \cdot r_c$$$$\mathbb{E}[r] = (1-\theta)\,v_f + \theta(1-\kappa)(v_f - c) + \theta\kappa\,(v_f - s\,c)$$
The $v_f$ terms:
$$(1-\theta)v_f + \theta(1-\kappa)v_f + \theta\kappa v_f = v_f$$
What’s left is the expected cost, and the key is factoring it:
$$\text{cost} = \theta(1-\kappa)\,c + \theta\kappa\,s\,c = \theta\,c\,[(1-\kappa) + s\kappa] = \theta\,c\,[1 + (s-1)\kappa]$$
So the formula is just:
$$\mathbb{E}[r] = v_f - c\,\theta\,\big(1 + (s-1)\,\kappa\big)$$
Also, note that $\mathbb{E}[\theta] \cdot \big(1 + (s - 1)\, \mathbb{E}[\kappa]\big)$ is the exact form under one assumption: we treat the two posterior beliefs as independent, because $\theta$ and $\kappa$ are updated in separate parameter pairs ($a$/$b$ vs $c_a$/$c_b$), so $\mathbb{E}[\theta \cdot \kappa] = \mathbb{E}[\theta] \cdot \mathbb{E}[\kappa]$. Strictly speaking, the two beliefs share the same evidence stream (a harsh “No” moves both), so the independence is a simplification, and a harmless one at these effect sizes.

A modeling note: the $z^2$ filter is a choice. It encodes “stops listening”: when trust collapses, the severity evidence collapses with it. The alternative, a single gate with $7 \cdot z$ and no second trust discount, keeps a hypervigilant channel: the shocked child keeps registering harshness even when she stops trusting, and in the chronic scenario her desire ends deep in the fear zone, about -23.1. We chose the two gates; a hypervigilant child is a different model.

Outcome	Probability (P(Outcome))	Payoff (r)
(a) No intervention	$1 - \theta$	$v_{fun}$
(b) Plain “No”	$\theta\,(1 - \kappa)$	$v_{fun} - c_{penalty}$
(c) Harsh “No”	$\theta\,\kappa$	$v_{fun} - s\,c_{penalty}\ \ (s = 3)$

set.seed(101)
harsh_data <- bind_rows(
  simulate_bayesian_credibility(trials = 200, phase1 = 200, harsh_prob = 0.00) %>% mutate(regime = "never harsh"),
  simulate_bayesian_credibility(trials = 200, phase1 = 200, harsh_prob = 0.01) %>% mutate(regime = "occasional harsh")
) %>%
  mutate(
    desire_lo = 10 - 20 * (theta + 1.96 * sd) * (1 + 2 * kappa),
    desire_hi = 10 - 20 * (theta - 1.96 * sd) * (1 + 2 * kappa)
  )

occ_desire <- harsh_data$desire[harsh_data$regime == "occasional harsh"]
occ_shocks <- which(diff(c(0, harsh_data$z[harsh_data$regime == "occasional harsh"])) < -0.05)
occ_steps <- round(occ_desire[occ_shocks] - occ_desire[occ_shocks - 1], 1)

ggplot(harsh_data, aes(x = trial, y = desire, color = regime)) +
  geom_ribbon(aes(ymin = desire_lo, ymax = desire_hi, fill = regime, group = regime), alpha = 0.12, color = NA) +
  geom_line(linewidth = 1) +
  geom_hline(yintercept = -10, linetype = "dashed", color = "grey60") +
  geom_hline(yintercept = 0, linetype = "dotted", color = "grey60") +
  annotate("text", x = 2, y = -10.9, label = "healthy result: -10", hjust = 0, color = "grey35", size = 3) +
  annotate("text", x = 190, y = 0.5, label = "prone above / not prone below", hjust = 1, color = "grey35", size = 3) +
  geom_text(data = harsh_data %>% group_by(regime) %>% slice_tail(n = 1),
            aes(x = 203, label = round(desire, 1)), hjust = 0, size = 3.2, show.legend = FALSE) +
  scale_x_continuous(limits = c(0, 218)) +
  scale_y_continuous(limits = c(-16, 19)) +
  scale_color_manual(values = c(
    "never harsh" = "#7F8C8D",
    "occasional harsh" = "#BBB111"
  )) +
  scale_fill_manual(values = c(
    "never harsh" = "#7F8C8D",
    "occasional harsh" = "#BBB111"
  )) +
  labs(
    title = "A Harsh No Is Priced Forever",
    subtitle = "Rare harsh events step the desire down permanently: she updates her belief about severity, and never hears the correction.",
    x = "Attempts",
    y = "Desire to Climb (Expected Payoff)",
    color = "Regime",
    fill = "Regime"
  ) +
  theme_minimal()

The labels at the end of each line show the settled desire.

The rare harsh event is surprising evidence: a harsh “No” is heard at harsh_weight = 7 times the weight of a plain one, through the same credibility filter (harsh_weight · z²), so it moves the severity belief hard. This is the Bayesian fingerprint: the step it causes never recovers. The first shock alone costs -3.6 points, about half of Part I’s early shock, where one harsh “No” dropped the Q-value by roughly $4$ points in a single update; the second costs -1.1. By the end the occasional-harsh child sits at -12.1 while the never-harsh child sits at -9.7. One harsh “No” in a hundred changes the whole expected payoff: harshness is cheap to have, and hard to un-have.

3. Chronic Harshness: The Child Who Stops Trusting

The dose increases. The state-space view of the companion post hid a harshness state that suppressed the step size. Here harshness attacks the other lever: every harsh event makes the child more stubborn, and the credibility weight’s $K$ grows with each shock (stubbornness_growth = 50), so the evidence arrives at a smaller and smaller volume:

$$K = K_0 + \text{stubbornness\_growth} \cdot \#\text{harsh events}, \qquad z = \frac{n}{n + K}.$$

set.seed(101)
bayesian_all <- bind_rows(
  simulate_bayesian_credibility(harsh_prob = 0.00) %>% mutate(regime = "never harsh"),
  simulate_bayesian_credibility(harsh_prob = 0.01) %>% mutate(regime = "occasional harsh"),
  simulate_bayesian_credibility(harsh_prob = 0.60) %>% mutate(regime = "chronic harsh")
) %>%
  mutate(
    desire_lo = 10 - 20 * (theta + 1.96 * sd) * (1 + 2 * kappa),
    desire_hi = 10 - 20 * (theta - 1.96 * sd) * (1 + 2 * kappa)
  )

bayesian_long <- bayesian_all %>%
  pivot_longer(c(desire, sd), names_to = "metric", values_to = "value") %>%
  mutate(metric = factor(metric, levels = c("desire", "sd")))

bayesian_final <- bayesian_all %>%
  group_by(regime) %>%
  summarise(
    desire_end = round(tail(desire, 1), 1),
    sd_end = signif(tail(sd, 1), 3),
    z_end = signif(tail(z, 1), 3),
    kappa_end = signif(tail(kappa, 1), 3),
    theta_end = round(tail(theta, 1), 3),
    shocks = max(harsh)
  )

desire_panel <- bayesian_long %>% filter(metric == "desire")
sd_panel <- bayesian_long %>% filter(metric == "sd")

ggplot(bayesian_long, aes(x = trial, y = value, color = regime)) +
  geom_ribbon(aes(ymin = desire_lo, ymax = desire_hi, fill = regime, group = regime),
              alpha = 0.12, color = NA, data = desire_panel) +
  geom_line(linewidth = 1) +
  facet_wrap(~metric, scales = "free_y", labeller = as_labeller(c(desire = "desire to climb", sd = "posterior uncertainty (SD of theta)"))) +
  geom_hline(aes(yintercept = -10), linetype = "dashed", color = "grey60", data = desire_panel) +
  geom_hline(aes(yintercept = 0), linetype = "dotted", color = "grey60", data = desire_panel) +
  geom_text(data = desire_panel, aes(x = 2, y = -10.9, label = "healthy result: -10"), hjust = 0, color = "grey35", size = 3) +
  geom_text(data = bayesian_long %>% group_by(metric, regime) %>% slice_tail(n = 1),
            aes(x = 203, label = ifelse(metric == "desire", round(value, 1), signif(value, 3))),
            hjust = 0, size = 3, show.legend = FALSE) +
  scale_x_continuous(limits = c(0, 218)) +
  scale_color_manual(values = c(
    "never harsh" = "#7F8C8D",
    "occasional harsh" = "#BBB111",
    "chronic harsh" = "#E74C3C"
  )) +
  scale_fill_manual(values = c(
    "never harsh" = "#7F8C8D",
    "occasional harsh" = "#BBB111",
    "chronic harsh" = "#E74C3C"
  )) +
  labs(
    title = "Chronic Harshness: Dose and the Credibility Weight",
    subtitle = "Each harsh event grows the stubbornness K, so the evidence arrives at a smaller volume. Left: expected desire with 95% credible band. Right: posterior SD.",
    x = "Attempts",
    y = NULL,
    color = "Regime",
    fill = "Regime"
  ) +
  theme_minimal()

The labels at the end of each line show the settled desire (left) and the final posterior SD (right).

The never-harsh child ends at -9.7, the occasional-harsh child at -12.1. The chronic-harsh child ends at -5.7, partway to the healthy -10. Harshness arrives through the same credibility filter as everything else (harsh_weight · z²), so once her trust collapsed the severity evidence stopped landing too: her severity belief sits at 0.0698, hardly higher than the occasional child’s, and her $\theta$ is stuck at 0.69. She carries the widest uncertainty of the three: posterior SD 0.142 (the never-harsh child’s is 0.00903), a 95% band wider than the space between the two reference lines. Her credibility weight collapsed to 0.0349: after 110 shocks she hears roughly three percent of the evidence that reaches other children. The child who has been shocked the most is the child who believes the world the least: she half-learned the rule, and she knows she doesn’t know it. The Q-learning child froze because her step size collapsed; the Bayesian child freezes because her trust did.

4. A New Task: Only the Trust Carries Over

Now the same question as before: what happens to the next thing the child tries to learn? The companion post’s answer was that only the learning rate carried over. The Bayesian answer is symmetric: the belief itself resets, and a new context (“be nice to your friend”) means a fresh $50/50$ prior, though a stronger one than she was born with (new_task_prior_strength = 10 against the original $3$), so the desire starts at $0$, exactly where the companion post’s child started her new task. Because the old task taught the child nothing about being kind, what carries over is the trust: the stubbornness $K$ accumulated in the old task. The kind acts pay $v_{fun} = 10$, the father is not harsh here, and the three children see exactly the same two hundred kind acts; nothing is random in this task, so the evidence is identical for all three.

set.seed(101)
transfer_all <- bind_rows(
  simulate_bayesian_credibility(harsh_prob = 0.00, new_task = TRUE) %>% mutate(regime = "never harsh"),
  simulate_bayesian_credibility(harsh_prob = 0.01, new_task = TRUE) %>% mutate(regime = "occasional harsh"),
  simulate_bayesian_credibility(harsh_prob = 0.60, new_task = TRUE) %>% mutate(regime = "chronic harsh")
) %>%
  mutate(
    desire_lo = ifelse(trial <= 200,
                       10 - 20 * (theta + 1.96 * sd) * (1 + 2 * kappa),
                       10 - 20 * (theta + 1.96 * sd)),
    desire_hi = ifelse(trial <= 200,
                       10 - 20 * (theta - 1.96 * sd) * (1 + 2 * kappa),
                       10 - 20 * (theta - 1.96 * sd))
  )

transfer_long <- transfer_all %>%
  pivot_longer(c(desire, sd), names_to = "metric", values_to = "value") %>%
  mutate(
    metric = factor(metric, levels = c("desire", "sd")),
    task = factor(ifelse(trial <= 200, "old task", "new task"), levels = c("old task", "new task")),
    trial_in_task = trial - 200 * (trial > 200)
  )

transfer_final <- transfer_all %>%
  group_by(regime) %>%
  summarise(
    desire_new_end = round(tail(desire, 1), 1),
    sd_new_end = signif(tail(sd, 1), 3),
    theta_new_end = round(tail(theta, 1), 3),
    z_new = signif(tail(z, 1), 3),
    shocks = max(harsh),
    time_to_7 = suppressWarnings(min(which(desire[201:400] >= 7)))
  )

desire_panel2 <- transfer_long %>% filter(metric == "desire")

ggplot(transfer_long, aes(x = trial_in_task, y = value, color = regime)) +
  geom_ribbon(aes(ymin = desire_lo, ymax = desire_hi, fill = regime, group = regime),
              alpha = 0.12, color = NA, data = desire_panel2) +
  geom_line(linewidth = 1) +
  facet_grid(metric ~ task, scales = "free_y", labeller = labeller(
    metric = c(desire = "desire (expected payoff)", sd = "posterior uncertainty (SD)"),
    task = c("old task" = "old task: climbing is forbidden",
             "new task" = "new task: be nice to your friend")
  )) +
  geom_hline(aes(yintercept = 0), linetype = "dotted", color = "grey60", data = desire_panel2) +
  geom_hline(aes(yintercept = -10), linetype = "dashed", color = "grey60", data = desire_panel2 %>% filter(task == "old task")) +
  geom_text(data = desire_panel2 %>% filter(task == "old task"), aes(x = 2, y = -11.2, label = "healthy result: -10"), hjust = 0, color = "grey35", size = 2.6) +
  geom_text(data = transfer_long %>% group_by(metric, task, regime) %>% slice_tail(n = 1),
            aes(x = 203, label = ifelse(metric == "desire", round(value, 1), signif(value, 3))),
            hjust = 0, size = 3, show.legend = FALSE) +
  scale_x_continuous(limits = c(0, 218)) +
  scale_color_manual(values = c(
    "never harsh" = "#7F8C8D",
    "occasional harsh" = "#BBB111",
    "chronic harsh" = "#E74C3C"
  )) +
  scale_fill_manual(values = c(
    "never harsh" = "#7F8C8D",
    "occasional harsh" = "#BBB111",
    "chronic harsh" = "#E74C3C"
  )) +
  labs(
    title = "A New Task: Only the Trust Carries Over",
    subtitle = "The belief resets to 50/50 in the new task; the stubbornness K does not. Ribbons are 95% posterior bands.",
    x = "Attempts in this task",
    y = NULL,
    color = "Regime",
    fill = "Regime"
  ) +
  theme_minimal()

The four facets split the same 400 attempts into the two tasks; labels show the final desire and posterior SD per task.

The never-harsh child enters with $K = 3$ and hears the kind acts at volume 0.985: she is past a desire of $+7$ by her 32-th attempt and ends at 9.5. The occasional-harsh child carries $K$ from her two shocks (volume 0.659) and ends at 9, with a posterior SD of 0.022 instead of 0.0112: she learns, but always a step behind, and never as sure.

The chronic-harsh child carries $K$ from 110 shocks into the new task and hears the kindness at volume 0.0349, about three percent. After two hundred kind acts her belief about being punished has moved from $0.5$ to 0.37 (the never-harsh child’s fell to 0.025), and her desire ends at 2.6, not far from where every child started. Her uncertainty, 0.127, remains the widest of the three, because she could not let the evidence in.

That is the Bayesian version of the same punchline: treatment harsh enough to break trust does not just fail to teach the old rule, it makes the next rule unteachable. The two children agree on the outcome, and disagree on the mechanism. The Q-learning child stops because her updates are too small; the Bayesian child stops because her beliefs are too stubborn. One stops updating; the other stops listening.

Conclusion

Part I promised this same exercise in a Bayesian framework, and the outcome is unchanged: consistency teaches, a rare harsh “No” is priced forever, and chronic harshness breaks the learner. What the Bayesian lens adds is where the damage lands. A shock that never repeats still moves the severity belief, and nothing in a consistent future will ever correct it. A shock that keeps repeating moves something deeper: the credibility weight itself, and with it the child’s ability to hear the next lesson at all. The Q-learning child’s step size collapsed; the Bayesian child’s trust did.

As in Part I, this is a toy model: no child runs Beta updates in her head. But when two very different learning rules land on the same three lessons about consistency and harshness, I trust the lessons more than either model.

Can a Baby Learn? A Toy Model of Discipline and Desire

marcoaurelioguerrap@gmail.com (Marco Aurélio Guerra Pereira) — Fri, 21 Aug 2026 10:00:00 -0300

Part I: Q-learning

I’m a father of two, and raising my kids gives me a lot of joy. I love teaching them and seeing them learn, whether from me or on their own. Lately, I have been contemplating how important consistency and patience are when teaching my kids.

More often than not, kids will not listen to you. This is just how it is. For a two-year-old, that colourful remote control is just too inviting. Climbing a chair is just one adventure that one cannot refuse, and so on. The mighty “No!” is not enough.

After a while, the “No” slowly starts to work. Babies learn at their own pace, but what intrigues me is why some learn, and others don’t. Of course, there might be an innate reason, but I doubt anyone would question that the environment and the caretaker play a big role.

One big thing I’ve always noticed is consistency. It is always important for the caretakers to be consistent with the rules at home. No is No. Something else is that harsh treatment (yelling, spanking) clearly has negative effects on the kid. I keep these two ideas in mind whenever I’m working with any learning human, not just my kids.

Thinking about this, I decided to see whether I could model it in a learning agent. The idea is simple: I want to model an agent whose learning is shaped by two things: i) caretaker consistency (how stable the payoff is) and ii) harsh treatment (punishment beyond the normal “No”).

Before anything, a disclaimer. This is just a simple model. I can promise you no brain works like the models I’ll show in this and the next part. This is just an exercise.

The child

Take a little girl too young to climb a chair on her own. To her, climbing looks fun: she puts an inner fun value $v_{fun} = 10$ on it. Her father has other plans. He wants to discourage the climbing, so every time she tries, he says “No” and blocks her. The “No” isn’t free for her: it carries a cost penalty $c_{penalty} = 20$. So the reward $r_t$ she experiences on attempt $t$ is:

$$r_t = \begin{cases} v_{fun} = 10 & \text{the father lets her climb}, \\[2pt] v_{fun} - c_{penalty} = -10 & \text{the father says ``No''}. \end{cases}$$

Notice that a blocked attempt is a net negative even though climbing is fun: $10 - 20 = -10$.

She doesn’t take one experience at face value. She keeps a running estimate $Q_t$ of what climbing is worth, and after each attempt she moves that estimate a fraction $\alpha$ toward the reward she just received. This is the simplest model in reinforcement learning: a Q-value updated by a prediction error. Economists will recognise it as adaptive expectations:

$$Q_{t+1} = Q_t + \alpha\, (r_t - Q_t),$$

where $r_t - Q_t$ is her surprise: how much the experience differed from what she expected. Repeatedly, $Q_t$ settles on the value of the payoff the father actually delivers.

1. Consistent vs. Inconsistent Father

Now consider two types of father: a consistent one and an inconsistent one who says no with probability $\gamma = 70\%$.

$$r_t = \begin{cases} v_{fun} - c_{penalty} & \text{with probability } \gamma, \\[2pt] v_{fun} & \text{with probability } 1 - \gamma, \end{cases}$$

so the Q-value converges to the expected payoff $10 \cdot 0.3 - 10 \cdot 0.7 = -4$, and it keeps wobbling because the signal is noisy:

$$\mathbb{E}[r_t] = v_{fun} - \gamma\, c_{penalty} = -4$$

The dashed line marks this healthy result. The consistent child lands exactly on $-10$. Values above 0 mean the child is still prone to climb the chair.

Why is $-10$ the “healthy” line? Because it is the honest price of the chair: $+10$ of fun, blocked for $20$, so $-10$ is the desire that matches reality. This says nothing about what sits between the zones. The model only moves the desire. A child drifting only slightly off $-10$ may follow the rule while quietly questioning the father, while a child far from it (far below, from harsh punishment, or clearly positive, from inconsistency) may be closer to fear or trauma than to learning. We do not model those zones. The distance from $-10$ is left for the reader to judge.

Below is the code that generates a simulation for each type of father.

library(ggplot2)
library(dplyr)
library(tidyr)

simulate_learning <- function(
    trials = 150,
    learning_rate = 0.05,
    v_fun = 10,
    c_penalty = 20,
    gamma_inconsistent = 0.7,
    initial_q = 10
) {
  results <- data.frame(
    trial = 1:trials,
    q_consistent = numeric(trials),
    q_inconsistent = numeric(trials)
  )
  q_c <- q_i <- initial_q
  for (t in 1:trials) {
    reward_c <- v_fun - c_penalty
    q_c <- q_c + learning_rate * (reward_c - q_c)
    reward_i <- ifelse(runif(1) < gamma_inconsistent, v_fun - c_penalty, v_fun)
    q_i <- q_i + learning_rate * (reward_i - q_i)
    results$q_consistent[t] <- q_c
    results$q_inconsistent[t] <- q_i
  }
  return(results)
}

set.seed(123)
data <- simulate_learning()

data_long <- data %>%
  pivot_longer(cols = starts_with("q_"), names_to = "Father_Type", values_to = "Q_Value")

ggplot(data_long, aes(x = trial, y = Q_Value, color = Father_Type)) +
  geom_line(linewidth = 1) +
  geom_hline(yintercept = -10, linetype = "dashed", color = "grey60") +
  geom_hline(yintercept = 0, linetype = "dotted", color = "grey60") +
  annotate("text", x = 2, y = -10.7, label = "healthy result: -10", hjust = 0, color = "grey35", size = 3) +
  annotate("text", x = 140, y = 0.5, label = "prone above / not prone below", hjust = 1, color = "grey35", size = 3) +
  geom_text(data = data_long %>% group_by(Father_Type) %>% slice_tail(n = 1),
            aes(x = 153, label = round(Q_Value, 1)), hjust = 0, size = 3.2, show.legend = FALSE) +
  scale_x_continuous(limits = c(0, 168)) +
  scale_y_continuous(limits = c(-12, 11.5)) +
  scale_color_manual(
    values = c("q_consistent" = "#2E86AB", "q_inconsistent" = "#A23B72"),
    labels = c("Consistent Father (100% No)", "Inconsistent Father (70% No)")
  ) +
  labs(
    title = "Consistent vs. Inconsistent Feedback",
    x = "Number of Attempts",
    y = "Desire to Climb (Q-Value)",
    color = "Scenario"
  ) +
  theme_minimal()

2. Adding Harsh Punishment

Now consider a father who loses his temper and says “No” harshly with probability $\varepsilon$. Severe repression has a $s$ multiplier effect on the $c_{penalty}$.

$$r_t = \begin{cases} v_{fun} - s\, c_{penalty} & \text{with probability } \varepsilon \text{ (harsh ``No'')},\\[2pt] v_{fun} - c_{penalty} & \text{otherwise}. \end{cases}$$

For comparison, assume a father who always yields, one who consistently says no, one who gives a harsh “no” ($\varepsilon = 0.02$), and an inconsistent one ($\gamma = 0.7$).

simulate_four_scenarios <- function(
    trials = 200,
    learning_rate = 0.08,
    gamma_inconsistent = 0.7,
    v_fun = 10,
    c_penalty = 20,
    harsh_prob = 0.02
) {
  q <- c(consist = 10, always = 10, incons = 10, harsh = 10)
  history <- data.frame()
  for (t in 1:trials) {
    r_consist <- v_fun - c_penalty
    q["consist"] <- q["consist"] + learning_rate * (r_consist - q["consist"])
    r_always <- v_fun
    q["always"] <- q["always"] + learning_rate * (r_always - q["always"])
    r_incons <- ifelse(runif(1) < gamma_inconsistent, v_fun - c_penalty, v_fun)
    q["incons"] <- q["incons"] + learning_rate * (r_incons - q["incons"])
    harsh_shock <- ifelse(runif(1) < harsh_prob, 1, 0)
    r_harsh <- ifelse(harsh_shock == 1, v_fun - 3 * c_penalty, v_fun - c_penalty)
    q["harsh"] <- q["harsh"] + learning_rate * (r_harsh - q["harsh"])
    history <- rbind(history, data.frame(
      trial = t,
      Consistent = q["consist"],
      Always_Yields = q["always"],
      Purely_Inconsistent = q["incons"],
      Harsh_punishment = q["harsh"],
      harsh_shock = harsh_shock
    ))
  }
  return(history)
}

set.seed(789)
results <- simulate_four_scenarios()

results_long <- results %>% 
  select(-harsh_shock) %>%
  pivot_longer(-trial, names_to = "Father_Type", values_to = "Desire")

ggplot(results_long, aes(x = trial, y = Desire, color = Father_Type)) +
  geom_line(linewidth = 1) +
  geom_hline(yintercept = -10, linetype = "dashed", color = "grey60") +
  geom_hline(yintercept = 0, linetype = "dotted", color = "grey60") +
  annotate("text", x = 2, y = -10.7, label = "healthy result: -10", hjust = 0, color = "grey35", size = 3) +
  annotate("text", x = 190, y = 0.5, label = "prone above / not prone below", hjust = 1, color = "grey35", size = 3) +
  geom_text(data = results_long %>% group_by(Father_Type) %>% slice_tail(n = 1),
            aes(x = 203, label = round(Desire, 1)), hjust = 0, size = 3.2, show.legend = FALSE) +
  scale_x_continuous(limits = c(0, 218)) +
  scale_y_continuous(limits = c(-15, 11.5)) +
  scale_color_manual(values = c(
    "Always_Yields" = "#2ECC71",
    "Consistent" = "#E74C3C",
    "Purely_Inconsistent" = "#9B59B6",
    "Harsh_punishment" = "#BBB111"
  )) +
  labs(
    title = "Four Parenting Styles",
    x = "Attempts",
    y = "Expected Utility (Q-Value)",
    color = "Scenario"
  ) +
  theme_minimal()

The expected payoff shifts only slightly, $\mathbb{E}[r_t] = v_{fun} - c_{penalty} - \varepsilon\,(s - 1)\, c_{penalty} = -10.8$. And the harsh child learns at the same step size as everyone else ($\alpha = 0.08$): the learning rate is the same. An early harsh event can still speed up the initial descent. The surprise is much bigger, and one shock drops the Q-value several points at once (here it lands at attempt 8, putting her about 5 attempts ahead of the consistent child). What harshness does not change is the pace of convergence: the curve settles toward its target at the same speed, just below it. Harshness does not speed up the lesson; it bends the target: the child settles slightly below the healthy -10 line.

A side note: harshness can buy consistency. In this model, harsh treatment has no consequences. No state builds up; nothing carries over. Under that assumption, an inconsistent father can compensate: the expected payoff only depends on the effective “No” rate $\gamma + \varepsilon\,(s - 1)$. A father who lets the child climb 70% of the time ($\gamma = 0.3$) can still bring the desire to the healthy $-10$ by punishing harshly in $\varepsilon = 0.35$ of attempts, since $0.3 + 2 \cdot 0.35 = 1$. The catch is that this works only on average. The shocks keep pulling the Q-value around, and it stops working the moment harshness has consequences, as the next section shows.

3. Too Much Harsh Punishment: A State-Space View

Now harshness is no longer just a rare event. It becomes a condition the child internalises.

The learning rate is no longer a fixed constant either: it becomes state-dependent, a function of the hidden state the child carries. In machine-learning terms, this is the child learning to learn: plasticity itself becomes a function of experience.

Each harsh event $e_t$, a coin flip with probability $\varepsilon$, adds to a hidden state $h_t$: her accumulated harshness exposure. The state decays slowly, at rate $\varphi$ (phi = 0.99):

$$h_t = \varphi\, h_{t-1} + e_t.$$

The state eats into learning. The more exposure, the smaller the step size (a frightened child updates less and less), floored at $\alpha_{min}$:

$$\alpha_t = \max\!\left( \frac{\alpha_0}{1 + \beta\, h_t},\ \alpha_{min} \right),$$

with $\beta = 10$ and $\alpha_0 = 0.1$. The payoff keeps the previous form, now tied to the event:

$$Q_{t+1} = Q_t + \alpha_t\, (r_t - Q_t), \qquad r_t = \begin{cases} v_{fun} - s\, c_{penalty} & \text{if } e_t = 1,\\[2pt] v_{fun} - c_{penalty} & \text{if } e_t = 0. \end{cases}$$

The timing matters: the update on attempt $t$ uses the step size the child carried before the event, so a harsh look lands at full strength; the whole sting registers. The suppressed step size then governs everything that follows, which is why each shock bites once and then leaves the child slow to learn until the state decays back.

Below are three scenarios: no harsh events, 1% harsh events, and 60% harsh events. With no severe events, the step size stays at its baseline $\alpha_{0} = 0.1$. With 1%, the state pops up after each rare shock, and the learning rate dips until it decays back. With 60%, the state saturates, and the step size collapses.

simulate_harsh_state_space <- function(
    trials = 200,
    lr0 = 0.1,
    v_fun = 10,
    c_penalty = 20,
    initial_q = 10,
    harsh_prob = 0.01,
    harsh_severity = 3,
    phi = 0.99,
    beta = 10,
    lr_floor = 0.0005
) {
  h <- 0
  q <- initial_q
  lr <- lr0
  history <- data.frame()
  for (t in 1:trials) {
    harsh_event <- ifelse(runif(1) < harsh_prob, 1, 0)
    reward <- ifelse(harsh_event == 1, v_fun - harsh_severity * c_penalty, v_fun - c_penalty)
    q <- q + lr * (reward - q)
    h <- phi * h + harsh_event
    lr <- max(lr0 / (1 + beta * h), lr_floor)
    history <- rbind(history, data.frame(trial = t, h = h, lr = lr, q = q))
  }
  return(history)
}

set.seed(42)
never_harsh <- simulate_harsh_state_space(harsh_prob = 0.00, harsh_severity = 3)
occasional_harsh <- simulate_harsh_state_space(harsh_prob = 0.01, harsh_severity = 3)
chronic_harsh <- simulate_harsh_state_space(harsh_prob = 0.60, harsh_severity = 3)

combined <- bind_rows(
  never_harsh %>% mutate(regime = "never harsh"),
  occasional_harsh %>% mutate(regime = "occasional harsh"),
  chronic_harsh %>% mutate(regime = "chronic harsh")
)

combined_long <- combined %>%
  pivot_longer(c(q, lr), names_to = "metric", values_to = "value")

q_panel <- combined_long %>% filter(metric == "q")

ggplot(combined_long, aes(x = trial, y = value, color = regime)) +
  geom_line(linewidth = 1) +
  facet_wrap(~metric, scales = "free_y", labeller = as_labeller(c(q = "Q-value", lr = "learning rate"))) +
  geom_hline(aes(yintercept = -10), linetype = "dashed", color = "grey60", data = q_panel) +
  geom_hline(aes(yintercept = 0), linetype = "dotted", color = "grey60", data = q_panel) +
  geom_text(data = q_panel %>% slice(1), aes(x = 2, y = -10.5, label = "healthy result: -10"), hjust = 0, color = "grey35", size = 3) +
  geom_text(data = q_panel %>% slice(1), aes(x = 243, y = 0.5, label = "prone above / not prone below"), hjust = 1, color = "grey35", size = 3) +
  geom_text(data = combined_long %>% group_by(metric, regime) %>% slice_tail(n = 1),
            aes(x = 203, label = ifelse(metric == "q", round(value, 1), signif(value, 3))),
            hjust = 0, size = 3, show.legend = FALSE) +
  scale_x_continuous(limits = c(0, 255)) +
  scale_color_manual(values = c(
    "never harsh" = "#7F8C8D",
    "occasional harsh" = "#BBB111",
    "chronic harsh" = "#E74C3C"
  )) +
  labs(
    title = "Dose and the State-Driven Learning Rate",
    x = "Attempts",
    y = NULL,
    color = "Regime"
  ) +
  theme_minimal()

The labels at the end of each line show the settled Q-value (left panel) and the final learning rate (right panel).

Same “kid”, three different “fathers”. The baseline: the patient father’s child lands at $-10$, exactly on the healthy line. The occasional-harsh father’s child tracks the baseline until the first harsh “No”. Each harsh “No” dips the Q-value deep into unhealthy territory, and the suppressed learning rate slows the recovery toward the healthy line; with several shocks spread across the run, she ends the task still below it, her learning rate at a fraction of the baseline. With the chronically harsh father, the frequency of the harsh treatment is so high that the learning rate collapses to its floor and never recovers: the rule is barely learned at all. So, besides the initial big shock, subsequent shocks get diminished, and the kids learn more slowly than in the previous situation. What is interesting here is not only that the state space tracks the learning rate, but that insensitivity to harsh punishment is endogenous to the model.

A new task: harshness carries over

We could test another scenario with the state-space model. Let’s assume the learning rate from one task carries over to a new one. So, once she finishes the 200 attempts learning whether she should climb the chair, she has another learning task. The new task is being nice to your friend. The child starts at $Q_0 = 0$. This means no prior about this behaviour. Every act of kindness the father praises pays $v_{fun}$ = 10. And all fathers are nice here; no shocks. The only difference is the learning rate their harsh history left them with.

simulate_new_task <- function(
    trials = 200,
    phase1 = 200,
    lr0 = 0.1,
    v_fun = 10,
    c_penalty = 20,
    initial_q = 10,
    new_task_q0 = 0,
    harsh_prob = 0.01,
    harsh_severity = 3,
    phi = 0.99,
    beta = 10,
    lr_floor = 0.0005
) {
  h <- 0
  q <- initial_q
  lr <- lr0
  history <- data.frame()
  for (t in 1:(2 * trials)) {
    harsh_event <- ifelse(t <= phase1 & runif(1) < harsh_prob, 1, 0)
    if (t == phase1 + 1) {
      q <- new_task_q0
    }
    if (t <= phase1) {
      reward <- ifelse(harsh_event == 1, v_fun - harsh_severity * c_penalty, v_fun - c_penalty)
    } else {
      reward <- v_fun
    }
    q <- q + lr * (reward - q)
    h <- phi * h + harsh_event
    lr <- max(lr0 / (1 + beta * h), lr_floor)
    history <- rbind(history, data.frame(trial = t, lr = lr, q = q, harsh_event = harsh_event))
  }
  return(history)
}

set.seed(10)
transfer_all <- bind_rows(
  simulate_new_task(harsh_prob = 0.00) %>% mutate(regime = "never harsh"),
  simulate_new_task(harsh_prob = 0.01) %>% mutate(regime = "occasional harsh"),
  simulate_new_task(harsh_prob = 0.60) %>% mutate(regime = "chronic harsh")
)

transfer_final <- transfer_all %>%
  group_by(regime) %>%
  summarise(
    q_old_end = round(q[100], 1),
    q_new_end = round(tail(q, 1), 1),
    lr_new_start = signif(lr[101], 3),
    lr_new_end = signif(tail(lr, 1), 3)
  )

transfer_long <- transfer_all %>%
  pivot_longer(c(q, lr), names_to = "metric", values_to = "value") %>%
  mutate(
    task = factor(ifelse(trial <= 200, "old task", "new task"), levels = c("old task", "new task")),
    trial_in_task = trial - 200 * (trial > 200)
  )
q_panel2 <- transfer_long %>% filter(metric == "q")

ggplot(transfer_long, aes(x = trial_in_task, y = value, color = regime)) +
  geom_line(linewidth = 1) +
  facet_grid(metric ~ task, scales = "free_y", labeller = labeller(
    metric = c(q = "Q-value", lr = "learning rate"),
    task = c("old task" = "old task: climbing is forbidden",
             "new task" = "new task: be nice to your friend")
  )) +
  geom_hline(aes(yintercept = 0), linetype = "dotted", color = "grey60", data = q_panel2) +
  geom_text(data = transfer_long %>% group_by(metric, task, regime) %>% slice_tail(n = 1),
            aes(x = 203, label = ifelse(metric == "q", round(value, 1), signif(value, 3))),
            hjust = 0, size = 3, show.legend = FALSE) +
  scale_x_continuous(limits = c(0, 218)) +
  scale_color_manual(values = c(
    "never harsh" = "#7F8C8D",
    "occasional harsh" = "#BBB111",
    "chronic harsh" = "#E74C3C"
  )) +
  labs(
    title = "A New Task: Only the Learning Rate Carries Over",
    subtitle = "The same 200 attempts, split into the two tasks: the old forbidden rule, then an unrelated task, being nice to your friend, where the father is not harsh and only the learning rate carries over.",
    x = "Attempts in this task",
    y = NULL,
    color = "Regime"
  ) +
  theme_minimal()

The four facets split the same 200 attempts into the two tasks; the labels at the end of each line show the final Q-value and learning rate per task.

All three children enter the new task with the same $Q_0 = 0$ and make the same kinds of actions. What differs is how quickly they absorb them. As one can see, because of the consequences of the harsh treatment, the occasionally-harsh child, shocked in only 4 of the 200 attempts, still carried the consequences and did not reach the healthy $Q = 10$ threshold. ML readers will recognise this: it is a toy version of loss of plasticity, where an agent’s ability to learn new tasks degrades because of how it learned the old one.

Conclusion

I’ve completed the tasks I set up earlier here. I made a model where: i) caretaker consistency is important and ii) harsh treatment affects learning. As noted earlier, this is a simple model; I doubt that a human brain works exactly like that. But as far as my knowledge goes, this isn’t unconventional: humans need consistency for learning, and teachers and fathers who treat their pupils harshly affect their learning negatively in multiple ways. Next part, I’ll do this exact exercise but using a Bayesian framework.