FragileML — When Simpler Is Safer

A structured study of ML model fragility across 5 independent failure axes on an e-commerce dataset.

Central thesis: A model can be statistically better while being systemically less reliable.

---

Axes

#	Axis	What varies	Frozen
1	Capacity	Model tier (LR → DT → RF → XGB → LGB)	Everything else
2	Fidelity	Label noise / feature noise levels	Model, features, seed
3	Stability	Random seeds / bootstrap draws	Model, data, features
4	Representation	PCA variance / top-k features	Model, noise, seed
5	Temporal	Train/test time window	Model, features, noise

---

Setup

# 1. Clone
git clone https://dagshub.com/Y-R-A-V-R-5/FragileML.git
cd FragileML

# 2. Install dependencies
pip install -r requirements.txt

# 3. Add your dataset
#    Place dataset.csv at:  data/dataset.csv
#    (file is gitignored — not tracked in the repo)

# 4. (Optional) connect DagsHub for experiment tracking
dagshub login

---

Run Experiments

# Axis 1 — full model sweep (default)
python scripts/run_axis.py

# Axis 2 — label noise (default) or feature noise or both
python scripts/run_axis.py --axis fidelity --task C4
python scripts/run_axis.py --axis fidelity --fidelity-mode feature_noise
python scripts/run_axis.py --axis fidelity --fidelity-mode both --models LR RF XGB

# Axis 3 — bootstrap (default) or seed variation or both
python scripts/run_axis.py --axis stability --task C4
python scripts/run_axis.py --axis stability --stability-mode seed
python scripts/run_axis.py --axis stability --stability-mode both

# Axis 4 — PCA (default) or top-k or both
python scripts/run_axis.py --axis representation --task C4
python scripts/run_axis.py --axis representation --repr-mode topk
python scripts/run_axis.py --axis representation --repr-mode both

# Axis 5 — temporal drift
python scripts/run_axis.py --axis temporal --task C4

# See plan without running
python scripts/run_axis.py --dry-run
python scripts/run_axis.py --axis stability --dry-run

# Track to DagsHub
python scripts/run_axis.py --track
python scripts/run_axis.py --axis fidelity --fidelity-mode both --track

Model aliases

Alias	Full name	Tier
LR	Logistic Regression	1 — linear
LinReg	Linear Regression	1 — linear
DT	Decision Tree	2 — single tree
RF	Random Forest	3 — ensemble
XGB	XGBoost	4 — boosting
LGB	LightGBM	4 — boosting

---

Data

The dataset (data/dataset.csv) is not tracked in git (244 MB).

Place your dataset at data/dataset.csv before running any experiments.
Column schema is described in config/data.yaml.

---

Project structure

FragileML/
├── config/
│   ├── data.yaml          # Dataset schema, tasks, preprocessing
│   ├── models.yaml        # Model registry (4 tiers, 6 models)
│   └── axes/              # One config per axis
├── src/
│   ├── axes/              # 5 axis runners + BaseAxis
│   ├── data/              # Loading, splitting, noise, representation
│   ├── metrics/           # Performance, stability, calibration, reporter
│   ├── models/            # Registry, evaluator
│   └── utils/             # Constants, logger, seed
├── scripts/
│   └── run_axis.py        # Main entry point
├── data/                  # Dataset goes here (gitignored)
└── artifacts/             # Experiment outputs (gitignored)