THE RARE-DISEASE PROBLEM
A senior clinician sees thousands of common cases and a handful of rare ones across a career. For a one-in-ten-thousand condition, even a teaching hospital may accumulate only a few dozen examples over decades — too few for any individual doctor to develop pattern recognition.
WHY RANDOM FORESTS, NOT DEEP LEARNING
A random forest trains hundreds of decision trees on random subsets of the data and averages their votes. It works on small, tabular medical datasets where deep neural networks overfit. For 1,522 records with structured fields — age, weight, lab values, diagnostic codes — forests are the workhorse, not transformers.
THE SIMILARITY APPROACH
Rather than predict a diagnosis directly, the system retrieves the historical patients whose feature vectors sit closest to the new case. The clinician then reads those charts. This sidesteps the black-box problem — the model's output is a list of real prior patients, not an opaque probability.
GREAT ORMOND STREET
Founded in 1852, GOSH was Britain's first hospital dedicated entirely to children and remains one of the world's leading paediatric centres. Its case archive — deep, longitudinal, and specialised — is exactly the kind of dataset rare-disease ML needs, and exactly what most hospitals lack.
THE BENCHMARKING TRAP
Beating clinicians on a retrospective dataset is not the same as helping them prospectively. The historical records carry the outcomes the clinicians eventually reached — the model is graded on a test where the answer key was written by the people it's being compared to. Real validation requires prospective trials where the model's recommendation changes a decision.
LENGTH-OF-STAY AS A HIDDEN METRIC
Predicting length of stay sounds operational, but it encodes diagnostic severity, complication risk, and case complexity in a single number hospitals already track. It's a useful proxy precisely because it's measured consistently — unlike 'diagnostic accuracy,' which depends on who's adjudicating.