CD practices can evolve from controlled rollouts to controlled experiments. The impact of new functionality is measured against business metrics such as conversions or subscription rates. Dev & product management work hand in hand with definition and analysis to make outcome-based product decisions.

Controlled rollouts using feature flags are a fundamental requirement for continuous delivery. Separating feature release from code deploy means code can be incrementally merged into the main branch but kept dark from customers until fully tested. Phased rollouts help release teams assess application health as new functionality is added, reducing the risk of production failure with techniques that limit user exposure and provide fast kill switches of problematic changes.

In addition, QA teams can shift-right and eliminate the need for a ‘production-like’ staging environment, going directly to test in the production environment for functional, performance, and UAT testing. This improves the partnership between dev and test, maintains QA’s role in the continuous delivery process, and reduces wait states in the value stream. Product teams can ensure a consistent user experience across all devices and touchpoints by aligning release schedules with universal code completion.

With these methods in place, CD practices can evolve from controlled rollouts to controlled experiments. The targeting system is expanded to support more complex segmentation that goes beyond simple on/off flagging. The impact of new functionality can be measured against business metrics such as product conversions, engagement, growth, subscription rates, or revenue. Dev and product management work hand in hand with definition and analysis to make outcome-based product decisions. The final step in the evolution is when the entire company views every feature release as an experiment, facilitating a culture of continuous improvement.

Lenore will highlight product and dev team she's worked with have used feature experimentation to make better product decisions. She will also share common and creative use cases, the statistical rigor needed to correctly interpret results, and recommended best practices that you can implement immediately for your experimentation practice.