adoption <- read.csv("data/teacher_adoption_survey.csv")
table(adoption$adopted)
0 1
63 117 04 Binary and Ordinal Models
Answering adoption and engagement questions with predicted probabilities
1 Learning Outcomes
By the end of this module, participants should be able to:
- Identify when an education outcome is binary or ordinal.
- State the education question a GLM is answering.
- Interpret binary-outcome results using predicted probabilities.
- Report ordinal outcomes as shifts in response categories rather than as average Likert points.
The education questions in this notebook
This notebook answers two applied questions.
First, among teachers, does prior training make adoption of a new instructional practice more likely, after accounting for teaching experience and school support?
Second, among learners, does an intervention shift engagement ratings towards higher categories, after accounting for baseline motivation and teacher feedback?
The statistical models matter only because these outcomes are not ordinary continuous scores. The reporting target is still practical: probabilities that can inform an education decision.
2 Binary Outcome Scenario
The first question is about teacher adoption. A district has introduced a new instructional practice and wants to know whether prior training is associated with adoption. The outcome is simple to record: each teacher either adopted the practice or did not.
That makes the outcome binary. The analysis should therefore answer a probability question:
For otherwise similar teachers, how much more likely is adoption among those who had prior training?
This is why a linear regression is not the right tool here. It could produce predicted probabilities below 0 or above 1. A logistic model keeps the answer on the probability scale, where the education decision lives.
The count table gives the raw number of adopters and non-adopters. It is the first check before modelling: if almost everyone adopted, or almost no one adopted, there would be little information for learning what predicts adoption.
aggregate(
adopted ~ prior_training,
data = adoption,
FUN = mean
)This second table keeps the education question visible. It compares adoption rates for teachers with and without prior training before adjusting for years of experience and school support.
3 Frequentist Framing
The frequentist baseline is a logistic regression fitted with glm(). This model assumes the outcome follows a Bernoulli distribution and links the linear predictor to the probability of adoption through the logit function. Fitting it first gives a reference point and exposes the interpretive difficulty that motivates moving to predicted probabilities later.
freq_logit <- glm(
adopted ~ prior_training + years_experience + school_support,
data = adoption,
family = binomial()
)
summary(freq_logit)
Call:
glm(formula = adopted ~ prior_training + years_experience + school_support,
family = binomial(), data = adoption)
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.53025 0.66457 -2.303 0.02130 *
prior_training 0.39655 0.36172 1.096 0.27295
years_experience 0.10467 0.03131 3.343 0.00083 ***
school_support 0.30154 0.17897 1.685 0.09202 .
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 233.08 on 179 degrees of freedom
Residual deviance: 214.64 on 176 degrees of freedom
AIC: 222.64
Number of Fisher Scoring iterations: 3exp(coef(freq_logit)) (Intercept) prior_training years_experience school_support
0.2164815 1.4866848 1.1103467 1.3519406 exp(confint(freq_logit)) 2.5 % 97.5 %
(Intercept) 0.05632868 0.7737492
prior_training 0.73702113 3.0623506
years_experience 1.04642511 1.1837447
school_support 0.95632010 1.9363595AIC(freq_logit)[1] 222.6444For the education question, focus on the Prior training row. That is the row asking whether teachers with prior training differ from otherwise similar teachers without prior training.
The first output reports coefficients on the log-odds scale, which is the natural scale of the model but not a scale most readers can use. The second output reports the same coefficients as odds ratios. For prior training, the odds ratio is about 1.49, with a 95% confidence interval from about 0.74 to 3.06 and a p-value of about 0.273. That means the frequentist model does not provide strong evidence that prior training is associated with adoption after adjusting for experience and school support.
The p-value in the odds-ratio table still comes from a hypothesis test. For prior training, it tests whether the odds ratio is 1, which would mean no difference in adoption odds between trained and untrained teachers after adjustment. It does not say that there is a 27.3% probability of no training effect, and it does not say that the true odds ratio is probably 1.49. As in linear regression, the probability statement lives on the data side, not on the parameter.
The interpretive problem is real. Even after the second table, the analyst still has to translate odds into something the policy audience can act on. The Bayesian model below addresses the same data, but, more importantly, the workflow it enables ends in predicted probabilities rather than odds. That is the move that matters for applied reporting.
4 Bayesian Logistic Model
The Bayesian version uses the same structural form as the frequentist glm:
\[ \begin{aligned} A_i &\sim \mathrm{Bernoulli}(p_i), \\ \mathrm{logit}(p_i) &= \alpha + \beta_{\mathrm{training}}T_i + \beta_{\mathrm{experience}}E_i + \beta_{\mathrm{support}}S_i . \end{aligned} \]
Here, \(A_i\) records whether teacher \(i\) adopted the practice, \(p_i\) is that teacher’s probability of adoption, \(T_i\) records prior training, \(E_i\) is years of teaching experience, and \(S_i\) is perceived school support.
The priors must be specified on the log-odds scale, which is where most newcomers find logistic-regression priors counter-intuitive. A standard normal prior Normal(0, 1) on a coefficient sounds restrictive, but on the log-odds scale it allows odds ratios from roughly 0.14 to 7.4 within two standard deviations, which is far wider than most realistic effects in social-science settings. The Normal(0, 1.5) prior on the intercept allows the baseline probability to range across most of (0, 1), which is appropriate when the marginal adoption rate is not strongly anchored a priori. As with the linear-regression case, these priors are chosen on the scale of the model and can be challenged on substantive grounds: a reviewer who thinks a coefficient as large as 3 in log-odds is implausible has a specific number to argue with.
library(brms)
adoption_model <- brm(
adopted ~ prior_training + years_experience + school_support,
data = adoption,
family = bernoulli(link = "logit"),
prior = c(
prior(normal(0, 1.5), class = "Intercept"),
prior(normal(0, 1), class = "b")
),
backend = "cmdstanr",
chains = 4,
cores = 4,
iter = 2000,
seed = 2026,
refresh = 0
)summary(adoption_model) Family: bernoulli
Links: mu = logit
Formula: adopted ~ prior_training + years_experience + school_support
Data: adoption (Number of observations: 180)
Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup draws = 4000
Regression Coefficients:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept -1.55 0.66 -2.87 -0.32 1.00 4661 3163
prior_training 0.37 0.34 -0.28 1.06 1.00 4161 3402
years_experience 0.11 0.03 0.05 0.17 1.00 3966 2788
school_support 0.30 0.18 -0.04 0.64 1.00 4252 2907
Draws were sampled using sample(hmc). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).
How to read this output
The coefficient table is useful for checking the model, but it is not the final education answer. The coefficient for prior training is on the log-odds scale. The next section translates the same model into predicted adoption probabilities, which are easier to report to school leaders or policy colleagues.
5 Interpret With Predicted Probabilities
The interpretive payoff is that the model can be reported as predicted probabilities. The next chunk compares two otherwise similar teachers: both have average years of experience and moderate school support, but only one has prior training.
new_teachers <- data.frame(
prior_training = c(0, 1),
years_experience = mean(adoption$years_experience),
school_support = 3
)
fitted(adoption_model, newdata = new_teachers) Estimate Est.Error Q2.5 Q97.5
[1,] 0.6137873 0.04583912 0.5220831 0.7021880
[2,] 0.6940331 0.06333840 0.5624463 0.8112612The table now reports the predicted adoption probability for each teacher type, with a credible interval, on the 0-to-1 scale. In this fitted model, teachers without prior training have an adoption probability of about 0.62, while teachers with prior training have an adoption probability of about 0.70, holding experience at the sample mean and school support at 3.
predicted_prob <- posterior_epred(adoption_model, newdata = new_teachers)
training_difference <- predicted_prob[, 2] - predicted_prob[, 1]
c(
mean_difference = mean(training_difference),
lower_95 = quantile(training_difference, 0.025),
upper_95 = quantile(training_difference, 0.975),
probability_positive = mean(training_difference > 0),
probability_above_10_points = mean(training_difference > 0.10)
) mean_difference lower_95.2.5%
0.08024579 -0.06548844
upper_95.97.5% probability_positive
0.22288151 0.85875000
probability_above_10_points
0.40250000 This second table answers the education question directly. It estimates the difference in adoption probability between trained and untrained teachers. In the current dataset, the training advantage is positive in most posterior draws, but the 95% credible interval still includes small or even negative differences. That is a useful result for decision-making: prior training looks promising, but the evidence is not strong enough to claim a large training advantage with high confidence.
Plain-language binary result
For teachers with average experience and moderate school support, prior training is associated with a higher predicted probability of adopting the instructional practice. The estimated increase is about 8 percentage points, but the uncertainty is wide. This supports a cautious claim: training may help adoption, but the size of that advantage is still uncertain.
6 Exercise: Binary Outcome
Use the tables above to answer the adoption question:
- What is the predicted adoption probability for teachers without prior training?
- What is the predicted adoption probability for teachers with prior training?
- How large is the estimated difference in percentage points?
- Would you describe the evidence for a training advantage as strong, moderate, or uncertain?
7 Ordinal Outcome Scenario
The second question is about learner engagement. Learners report engagement on a five-point scale, where 1 means very low engagement and 5 means very high engagement. The district does not only want to know whether the average engagement score moved. It wants to know whether the intervention shifted learners towards the higher engagement categories.
That makes the outcome ordinal. The categories have a clear order, but the distance from 1 to 2 does not have to mean the same thing as the distance from 4 to 5. A model that respects the ordered categories is therefore a better match to the education question.
The dataset student_engagement_ordinal.csv contains an engagement rating from 1 to 5, with predictors for an intervention, baseline motivation, and teacher feedback.
engagement <- read.csv("data/student_engagement_ordinal.csv")
table(engagement$engagement)
1 2 3 4 5
63 47 52 44 14 prop.table(
table(engagement$intervention, engagement$engagement),
margin = 1
)
1 2 3 4 5
0 0.40566038 0.26415094 0.20754717 0.10377358 0.01886792
1 0.17543860 0.16666667 0.26315789 0.28947368 0.10526316The first table gives the overall distribution across categories. The second keeps the education question visible by comparing engagement ratings for learners in the intervention and comparison groups. Sparse categories matter because very rare categories are hard to estimate precisely.
8 Cumulative Ordinal Model
The cumulative-logit model is the standard ordinal model and the one brms uses by default. The technical output is still on a log-odds scale, but the reporting target is not log-odds. The reporting target is the predicted probability of each engagement category.
In this model, the intervention coefficient asks whether the intervention shifts learners towards higher engagement categories after accounting for baseline motivation and teacher feedback.
ordinal_model <- brm(
ordered(engagement) ~ intervention + baseline_motivation + teacher_feedback,
data = engagement,
family = cumulative(link = "logit"),
prior = c(
prior(normal(0, 1), class = "b")
),
backend = "cmdstanr",
chains = 4,
cores = 4,
iter = 2000,
seed = 2026,
refresh = 0
)summary(ordinal_model) Family: cumulative
Links: mu = logit
Formula: ordered(engagement) ~ intervention + baseline_motivation + teacher_feedback
Data: engagement (Number of observations: 220)
Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup draws = 4000
Regression Coefficients:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept[1] 3.57 0.66 2.34 4.89 1.00 5297 3091
Intercept[2] 4.73 0.68 3.42 6.09 1.00 5075 2930
Intercept[3] 6.01 0.72 4.65 7.45 1.00 4716 3200
Intercept[4] 7.96 0.80 6.47 9.56 1.00 4824 3250
intervention 1.02 0.26 0.51 1.53 1.00 6030 3246
baseline_motivation 0.63 0.13 0.38 0.89 1.00 5661 3275
teacher_feedback 0.68 0.14 0.40 0.96 1.00 5012 3381
Further Distributional Parameters:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
disc 1.00 0.00 1.00 1.00 NA NA NA
Draws were sampled using sample(hmc). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).
How to read this output
The intervention coefficient is not an average Likert-point increase. In a cumulative ordinal model, a positive intervention coefficient means the model shifts probability away from lower engagement categories and towards higher engagement categories. The next table translates that shift into category probabilities.
9 Communicating Ordinal Results
The same translation principle from the logistic model applies, with one extra step. The coefficients of the cumulative model are on the log-odds scale and apply across category transitions; reporting them directly is not useful for most audiences. The recommended reporting strategy is to derive predicted category probabilities for substantively meaningful cases. For an intervention study on engagement, this means showing the predicted probability of each engagement level for intervention and comparison learners with the same baseline motivation and teacher feedback values.
new_learners <- data.frame(
intervention = c(0, 1),
baseline_motivation = 3,
teacher_feedback = 3
)
predicted_categories <- posterior_epred(ordinal_model, newdata = new_learners)
apply(predicted_categories, c(2, 3), median) 1 2 3 4 5
[1,] 0.4155838 0.2741194 0.1962303 0.09131184 0.01748039
[2,] 0.2029577 0.2416517 0.2940380 0.20503855 0.04730643The category-probability table is the social-science version of the ordinal model. It lets the researcher say where the intervention appears to move the response distribution: away from categories 1 and 2, towards categories 3, 4, and 5.
For a shorter education decision, it is often useful to combine categories 4 and 5 as “high engagement”.
high_engagement <- predicted_categories[, , 4] + predicted_categories[, , 5]
apply(high_engagement, 2, quantile, probs = c(0.025, 0.50, 0.975)) [,1] [,2]
2.5% 0.06971447 0.1799319
50% 0.10926460 0.2541523
97.5% 0.16325337 0.3419671high_engagement_difference <- high_engagement[, 2] - high_engagement[, 1]
c(
mean_difference = mean(high_engagement_difference),
lower_95 = quantile(high_engagement_difference, 0.025),
upper_95 = quantile(high_engagement_difference, 0.975),
probability_positive = mean(high_engagement_difference > 0),
probability_above_10_points = mean(high_engagement_difference > 0.10)
) mean_difference lower_95.2.5%
0.14502676 0.07017851
upper_95.97.5% probability_positive
0.22645399 1.00000000
probability_above_10_points
0.86775000 A common reporting mistake is to convert the ordinal output back into a continuous summary, with statements like “the intervention increased engagement by 0.4 Likert points.” That collapses the response categories and makes the scale look more precise than it is. The defensible alternative is to state predicted probabilities directly.
Plain-language ordinal result
For learners with average baseline motivation and teacher feedback, the intervention is associated with a higher probability of high engagement. The predicted probability of a 4-or-5 engagement rating is about 11% for the comparison case and about 25% for the intervention case. The estimated increase is about 14 percentage points, with most of the posterior evidence favouring a positive shift.
10 Takeaway
Bayesian models are not limited to linear regression. The same workflow applies across outcome types: choose a model appropriate for the outcome, specify priors on the natural scale of the model, fit, check, and communicate results on a scale that answers the applied question. Predicted probabilities for binary outcomes, and predicted category probabilities for ordinal ones, are usually the right communication device. The choice of likelihood is a substantive choice about the kind of data being modelled, and treating an ordinal outcome as continuous is the most common way researchers paper over that choice.