When you’re monitoring an app rollout post-release, you’re in the hot seat. Your team is relying on you to make sense of a tangle of different indicators of release health, across multiple tools, and perhaps with input from multiple other teams and stakeholders as well. You need to continuously digest all of those inputs and be ready to make a big call — to halt the rollout if needed — without hesitating. If you’re lucky, your team shares rollout responsibilities around, to reduce mental load on any one individual, but the exercise is still the same complicated and fragile puzzle.
Rollouts are hard because they’re simultaneously complex and mission-critical.
Rollouts involve many moving pieces, in tools and data and stakeholders collaborating, and your team has to get them right, each and every time — otherwise the negative impact to users can be huge. Without a single source of truth, rollouts typically involve bouncing between different dashboards and looping in other team members who have access to the right tools, and the know-how to interpret that particular data. For example, while engineers are usually the ones monitoring crash data, it’s often product managers who own analytics and who understand expected user behavior. Both inputs are crucial. And despite the complexity of the tools and data, interpretation and action on signals of release health has to be quick and confident. Teams can’t afford to lose any time in halting a bad rollout before it affects even more users.
Rollouts by Runway: a single source of truth for release health, instantly understandable – and actionable
As former mobile engineers ourselves, the Runway team has experienced the challenge of rollouts firsthand — and now, we’ve built something that can help. We’re excited to announce Rollouts, a new part of the Runway platform that helps you navigate rollouts with less collective stress, and more confidence. Rollouts allows your team to easily keep tabs on release health in just one place instead of many, alerts you whenever configured metrics become unhealthy, and safeguards release health by automatically halting rollouts based on thresholds you define.
Rollouts integrates with the mix of different tools you use to measure app health – crash reporting, observability & product analytics, the app stores (for per-version user ratings) – to create a single source of truth and unified dashboard that gives your team a holistic and instantly understandable view of release health. No more context-switching, no more mental overhead, and no more bugging other folks on the team to dig up and interpret data for you.
Across all tools and every metric, you can configure granular thresholds that together define what “healthy” looks like for your team. Runway will surface a crystal clear view of how each release is tracking relative to that definition. With expectations around health codified this way, there’s no room for ambiguity or hand-waving when it comes to your team’s standards for quality of the product you’re shipping.
The moment any of your configured metrics become unhealthy during a rollout, Runway will alert your team, in specific channels of your choosing and with full context on which metrics are problematic. This helps prevent blind spots and delays: you can be confident that no bad trends will go unnoticed, and that bad releases can be acted on more quickly.
To safeguard and streamline rollouts even further, Runway’s rollout automations can automatically halt unhealthy releases, or even accelerate healthy ones. If any metric falls afoul of one of your defined thresholds during a rollout, Runway can automatically stop the rollout and alert your team. Conversely, if all of your metrics are looking good and your rollout has reached a critical mass of users you define, Runway can accelerate things and release to all users. These automations prevent delays and human error during a critical part of the release process, when the consequences of a bad release making it out to even more users are serious — and when getting a good release out to all users more quickly can be equally impactful.
Rollouts is a key next step in furthering Runway’s mission of helping mobile teams build and ship better products, more collaboratively, and we’re excited for everyone to try it out! Let us know what you think, and feel free to reach out for a personalized, guided tour!
