Add megatron_ray_fault_tolerant example with comprehensive fault tolerance implementation#19

Open

xyuzh wants to merge 18 commits intoanyscale:mainfrom

xyuzh:megatron_ray_fault_tolerant

Commits on Nov 19, 2025

some initial code

robertnishihara
authored and
xyuzh
committed
Add vllm

robertnishihara
authored and
xyuzh
committed
Scale image processing pipeline for 2B+ images
xyuzh
committed
updates

robertnishihara
authored and
xyuzh
committed
Add megatron_ray_fault_tolerant example with comprehensive fault tolerance
xyuzh
committed
remove redundant branch
xyuzh
committed

Commits on Nov 26, 2025

fast(ish) model checkpointing/loading with s3
erictang000
committed

Commits on Nov 27, 2025

test with 8b on 4 nodes - models with tied word embeddings might fail with > 2 nodes now, but that is ok
erictang000
committed