Distributed Training with Ray Train

Hands-on training for distributed training with Ray Train. You learn how Ray distributes a training job across workers, then build distributed XGBoost end to end: an efficient data loader, fault tolerance, MLflow tracking, and cluster sizing.

The material comes in two views of the same content:

• Notebooks (top level): instructor-led student notebooks, run them in an workspace, top to bottom.
• agent/: dense reference skills (SKILL.md) for a coding agent such as Claude Code, to use when you write or debug Ray Train code after the workshop.

Notebooks

Run them in order.

	Topic
00a_ray_fundamentals.ipynb	The Ray cluster, tasks, actors, resource requests, and the Ray Dashboard
00b_ray_data.ipynb	Ray Data APIs and best practices
01_ray_train_architecture.ipynb	The controller and worker model, the train context, ScalingConfig, and data ingest
02_xgboost_best_practices.ipynb	Distributed XGBoost: the histogram trick, worker coordination, loading data, fault tolerance, MLflow tracking, and cluster sizing

Each notebook runs on its own; notebook 02 does not depend on output from notebook 01.

Layout

.
├── 00a_ray_fundamentals.ipynb
├── 00b_ray_data.ipynb
├── 01_ray_train_architecture.ipynb
├── 02_xgboost_best_practices.ipynb
├── code/xgboost/      # full-scale scripts notebook 02 links to
├── images/            # screenshots used in the notebooks
└── agent/
    ├── references/ray-train-architecture.md
    └── xgboost-best-practices/SKILL.md

Cluster

CPU only. Notebook 02 trains on 2 workers with 2 CPUs each. Notebook 01 writes its own sample data to /mnt/cluster_storage; notebook 02 reads a public S3 dataset with no credentials.

Scripts

code/xgboost/ holds the full-scale versions of notebook 02:

• train.py: the golden path at TB scale, with parallel S3 reads and checkpoint resume
• train_with_mlflow.py: the same pattern with MLflow tracking
• tune_loader.py: sweeps the S3 read settings for your instance and dataset
• job.yaml: submits train.py as an Anyscale Job (anyscale job submit -f job.yaml)

Agent skills

• agent/references/ray-train-architecture.md: the execution model from notebook 01
• agent/xgboost-best-practices/SKILL.md: DMatrix choice, sizing math, S3 read tuning, and MLflow from notebook 02

To install the XGBoost skill in Claude Code, copy its folder into ~/.claude/skills/, or into .claude/skills/ inside your project.