Module 4.1.2 · Phase 4 Enforcement · Step 4.1 ValidationSupplemental
Experimentation & A/B Testing
How does a change affect each subgroup before it rolls out?
Tests a model change before rollout, looking at the effect on each subgroup as well as overall.
What it does
You choose the design, state the hypothesis, pick outcome and fairness metrics and use a power analysis to see how much data each subgroup needs. It ends in a recorded recommendation: deploy the treatment, keep the control, extend or redesign the experiment.
Why it matters
This step validates whether implemented interventions meet defined fairness targets and assesses residual risks using statistically sound methods. It also defines ongoing monitoring requirements to detect fairness degradation after deployment.
Where it sits
- Validation
- Release Gate
- Operating Model
- Scrum Fairness Oversight
- Interventions Validation Framework
- Experimentation & A/B Testing
How to use it
In the Fairness Navigator Beta
Work through the steps in order. Several modules open from the Navigator's steps and hand their results back.
On its own Live
Open 4.1.2 from the platform's Modules page and run it when your project needs it. Work is saved with the assessment.
After a Pulse reading Live
Start with a fast first read, then go deeper with the modules where it points.