The problem
Each study is a full knee MRI session: several series (sagittal, coronal, axial; fluid-sensitive or not; fat-suppressed or not), each made of dozens of DICOM slices from scanners all over the world. The task is to output a probability for each of twelve findings: ACL and MCL injury, medial and lateral meniscal tears, medial, lateral, and patellofemoral osteoarthritis, effusion, synovitis, Baker's cyst, bone contusion, and fracture.
The hard part is supervision. Only 58 of 4,407 training studies have expert labels. Every study has its original radiology report, often not in English, but reports are not available at test time, so the final model must work from images alone.
Approach
- Audit before modelling. A read-only, resumable inventory of every DICOM header and an exhaustive pixel-decode pass produced explicit exclusion policies for corrupt series, plus geometry audits of slice orientation, pixel spacing, and physical field of view. (dicom_audit.py, physical_geometry_audit.py)
- Turn reports into labels. A local Qwen2.5-3B-Instruct extractor read multilingual reports and emitted per-finding probabilities for the 4,349 unlabeled studies. Benchmarked against the 58 gold studies, it reached macro AUC 0.753. An equal-mean ensemble of three public CC0 label sets reached 0.896 and became the weak target. (report_extractor.py, weak_label_benchmark.py)
- Leakage-safe validation. Deterministic multilabel-stratified five-fold splits over the gold studies. Gold studies are removed from weak pretraining entirely, so no validation image is ever seen in training. (gold_folds.py)
- Model. A DINOv2 ViT-S/14 encoder embeds each slice. Target-specific attention pools slices into a series vector, series metadata is appended, and a second target-specific attention pools series into a study vector. Weak pretraining on 4,349 studies is followed by gold fine-tuning per fold. (imaging_training.py)
- Offline inference. Each test study is decoded once, every fold checkpoint scores it, and scores are rank-averaged per target to cancel calibration drift between checkpoints. The submission is validated for exact columns, identifier order, and finite in-range probabilities before it is written. (imaging_inference.py, submission.py)
Preprocessing: why the full field of view stays primary
Scanners differ widely. Across 24,386 series, the shorter in-plane field of view spans 70 to 310 mm, with the middle 90% between 140 and 182 mm, and pixel spacing runs from 0.17 to 0.56 mm over the same percentiles. A fixed crop would cut anatomy from some series and pad others, so every series is fit into 224 × 224 by physical size with symmetric padding. A square crop is only an optional detail view, and the audit picked its size: the largest candidate that needs no padding for at least 90% of series, overall and in every plane.
Weak labels: which source to trust
Scoring each label source against the expert labels came before any imaging change. The ensemble beat the Qwen extractor on all twelve findings, with macro AUC 0.896 against 0.753. The ensemble covers 57 of the 58 gold studies; Qwen covers all 58. This comparison chose a label source. It does not estimate leaderboard performance.
Sources: gold_benchmark_summary.json, public_weak_label_benchmark.json
Results
Out-of-fold AUC pooled across all 58 gold studies, where every prediction comes from a model that never saw that study. Between v1 and v2, only the weak-label source changed; architecture, cache, folds, and fine-tuning settings were held fixed.
| Finding | v1: Qwen labels | v2: label ensemble | Change |
|---|---|---|---|
| ACL | 0.635 | 0.755 | +0.120 |
| MCL | 0.644 | 0.603 | −0.041 |
| Medial meniscus | 0.599 | 0.780 | +0.181 |
| Lateral meniscus | 0.754 | 0.733 | −0.021 |
| Medial OA | 0.811 | 0.888 | +0.078 |
| Lateral OA | 0.696 | 0.652 | −0.044 |
| Patellofemoral OA | 0.735 | 0.766 | +0.031 |
| Effusion | 0.815 | 0.919 | +0.104 |
| Synovitis | 0.749 | 0.792 | +0.043 |
| Baker's cyst | 0.741 | 0.889 | +0.149 |
| Bone contusion | 0.521 | 0.822 | +0.301 |
| Fracture | 0.536 | 0.803 | +0.267 |
| Macro AUC | 0.686 | 0.784 | +0.097 |
Five-fold mean macro AUC rose from 0.724 to 0.812, and every individual fold improved. Brier score fell from 0.197 to 0.160. Source: reports/training.
With 58 validation studies, these numbers carry wide uncertainty. They guided engineering decisions; they are not a leaderboard score, and this is not a clinical tool.
Code walkthrough
Two excerpts from the repository, unmodified apart from trimmed input checks. Everything else is one click away on GitHub.
Target-specific attention over slices and series
Each finding gets its own attention weights, so the ACL head can focus on sagittal slices through the notch while the effusion head looks at the suprapatellar recess. Series metadata (plane, fluid sensitivity, fat suppression) is fused before series-level pooling.
features = self.slice_projection(self._encode(images, encoder_chunk_size))
series_target_features: list[Any] = []
offset = 0
for length in series_lengths:
part = features[offset : offset + length]
scores = self.slice_attention(part)
weights = torch.softmax(scores, dim=0)
series_target_features.append(torch.einsum("st,sd->td", weights, part))
offset += length
stacked = torch.stack(series_target_features, dim=0)
expanded_metadata = metadata[:, None, :].expand(-1, len(TARGET_COLUMNS), -1)
enriched = self.series_projection(
torch.cat([stacked, expanded_metadata], dim=-1)
)
series_scores = torch.einsum(
"rtd,td->rt", enriched, self.series_queries
) / math.sqrt(feature_dimension)
series_weights = torch.softmax(series_scores, dim=0)
study_features = torch.einsum("rt,rtd->td", series_weights, enriched)
return torch.sum(study_features * self.head_weight, dim=-1) + self.head_bias
Rank-averaged fold ensemble
Pooled out-of-fold analysis showed that fold checkpoints rank studies well but disagree on score scale. Averaging percentile ranks per target instead of raw probabilities removes that drift at test time without using labels.
def _percentile_ranks(values: np.ndarray) -> np.ndarray:
"""Return average percentile ranks with deterministic tie handling."""
order = np.argsort(values, kind="stable")
sorted_values = values[order]
ranks = np.empty(len(values), dtype=np.float64)
start = 0
while start < len(values):
stop = start + 1
while stop < len(values) and sorted_values[stop] == sorted_values[start]:
stop += 1
ranks[order[start:stop]] = (start + stop - 1) / 2.0
start = stop
return (ranks + 0.5) / len(values)
def rank_average_probabilities(model_probabilities: Any) -> np.ndarray:
"""Normalize each model/target score scale before ensemble averaging."""
values = np.asarray(model_probabilities, dtype=np.float64)
ranked = np.empty_like(values)
for model_index in range(values.shape[0]):
for target_index in range(values.shape[2]):
ranked[model_index, :, target_index] = _percentile_ranks(
values[model_index, :, target_index]
)
return np.mean(ranked, axis=0)
Full source map
| Module | Purpose |
|---|---|
| dicom_audit.py | Read-only, resumable DICOM inventory and decoder audit |
| dicom_loader.py | Geometry-aware, bounded-memory DICOM series loading |
| orientation_audit.py | Per-slice orientation variation analysis |
| physical_geometry_audit.py | Pixel spacing and physical field-of-view audit |
| pixel_integrity.py | Series exclusion policy from exhaustive pixel decoding |
| preprocessing_cache.py | Resumable full-cohort preprocessing and tensor caching |
| report_extractor.py | Multilingual LLM extraction of findings from radiology reports |
| weak_label_benchmark.py | Benchmark and ensemble report-derived labels against gold |
| gold_folds.py | Deterministic, leakage-safe multilabel folds |
| imaging_training.py | Multi-series model, weak pretraining, and fold fine-tuning |
| oof_evaluation.py | Strict pooled out-of-fold evaluation |
| evaluation.py | Competition metrics implemented from scratch |
| imaging_inference.py | Offline multi-checkpoint inference and rank ensemble |
| submission.py | Strict submission generation and validation |
| kaggle_package.py | Self-contained offline Kaggle model package |
Also in the repository: 18 pipeline scripts, 7 notebooks, and 22 pytest test modules.
Tools
Python, PyTorch, DINOv2, Hugging Face Transformers (Qwen2.5-3B-Instruct), pydicom, NumPy, pandas, pytest, Jupytext, and Google Cloud Spot L4 GPUs with checkpointed, resumable training runs.