What I Learned from the GMDS Biostatistics Competition 2026
Ensembling heterogeneous treatment effect estimators, and where the pipeline breaks when every component is evaluated separately
I signed up for the GMDS Biostatistics Competition 2026 mostly to test something I’d been curious about for a while: whether heterogeneous treatment effect estimation gets more robust if you combine several learners instead of picking one and trusting it.
My submission ran a single R pipeline across the five competition tasks. I estimated individual treatment effects with a handful of approaches - random forests, elastic-net models, gradient boosting, and causal forests1 - and then combined their subgroup assignments through a co-association consensus step inspired by Evidence Accumulation Clustering.
What made the competition useful wasn’t just finding out whether the overall approach worked. It was seeing exactly which parts of the pipeline held up and which ones didn’t.
What worked
Combining complementary evidence turned out to help with detecting treatment effect heterogeneity in the first place. I used three signals here - the GRF calibration test, an interaction test between treatment and the estimated subgroup, and a comparison of treatment effects between the subgroup and its complement - and decided by majority vote. That left me more convinced that HTE detection shouldn’t lean on a single diagnostic.
The consensus framework also did a reasonable job estimating how large the treatment-responsive subgroup was overall, even in cases where individual patient assignments were shaky. Ensemble methods seem to recover the population-level structure before they recover the individual-level labels.
What I would change
A few things didn’t hold up as well, and I’d do them differently next time.
1. Variable selection needs its own ensemble strategy
I used elastic-net regression with treatment-covariate interactions to separate predictive variables from prognostic ones - non-zero interaction coefficients as predictive, non-zero main effects as prognostic. In hindsight this leaned on a single model too heavily, and lambda.min is tuned for predictive performance, not for recovering the true sparse structure. Things I’d try instead: stability selection, stronger penalization like lambda.1se, hierarchical interaction selection, importance measures from causal forests, and consensus variable selection across several HTE learners. Basically the same ensemble idea I used for treatment effects, applied one step earlier to variable discovery.
2. Don’t binarize CATE estimates too early
For subgroup assignment, each learner’s CATE estimate got collapsed into a binary benefiter / non-benefiter call based on whether it was above or below zero, and those binary calls fed the co-association matrix. Simple, robust, and it throws away a lot of information: a predicted effect of 0.001 gets the same label as 1.5, while two estimates that are both close to zero but on opposite sides of it get treated as completely different.
Next time I’d keep the continuous CATE predictions around longer and build the consensus straight from standardized treatment-effect estimates - cluster patients on the matrix
\[ \left[ \hat{\tau}_{RF},\; \hat{\tau}_{glmnet},\; \hat{\tau}_{XGB},\; \hat{\tau}_{GRF} \right]. \]
This would allow both the direction and magnitude of predicted treatment benefit to contribute to the consensus.
3. Subgroup discovery and treatment-effect estimation must be separated
This is probably the most important lesson.
After identifying the subgroup, I estimated treatment effects within it and its complement using AIPW via grf::average_treatment_effect(). The problem is that the same dataset fed both steps: patients get selected partly because they look like they have a large treatment effect, and then the treatment effect gets estimated in that same selected group. AIPW is a good estimator, but it doesn’t fix bias that comes from data-adaptive subgroup selection on its own.
What I’d do instead is cross-fit the whole thing: build the CATE model and subgroup rule on a training fold,
\[ \text{training data} \rightarrow \text{CATE model} \rightarrow \text{subgroup rule} \]
then apply that rule to a held-out fold and estimate the effect only there,
\[ \text{out-of-fold patients} \rightarrow \text{subgroup assignment} \rightarrow \text{treatment-effect estimation}. \]
Repeating that across folds should stabilize both the subgroup membership and the effect estimates.
The broader lesson
The thing I keep coming back to is that good subgroup discovery doesn’t automatically give you good treatment-effect estimation. “Heterogeneous treatment effect analysis” is really several different problems stacked on top of each other:
\[ \text{Detect HTE} \neq \text{Identify modifiers} \neq \text{Assign patients} \neq \text{Estimate subgroup prevalence} \neq \text{Estimate subgroup treatment effects}. \]
A method can be good at one of these and bad at another, and it’s easy to miss that if you only evaluate the pipeline end to end. Next time I’d rather design around these pieces separately - ensemble CATE estimation, consensus variable selection, continuous consensus subgrouping, cross-fitting, and independent effect estimation - and then wire them together, instead of treating the pipeline as one black box.
That’s probably the real value of entering a competition like this: not the ranking, but finding out exactly where a pipeline that looked fine end-to-end breaks once you check each piece on its own. I’ll be carrying these lessons into the next version.
Footnotes
The pipeline was built in R using grf for causal forests, the GRF calibration test, and AIPW average treatment effect estimation; glmnet for elastic-net variable selection; and xgboost for gradient-boosted CATE estimation. The co-association consensus step was inspired by Evidence Accumulation Clustering (Fred & Jain, 2005, IEEE TPAMI).↩︎