Human interaction can never be automated.
And that’s a good thing! Humans have to be involved in the release process because humans are collaborating on building your software. No robot, no matter how smart, can take over the complex array of decisions and actions the team needs to make as part of every release.
This will remain true unless we one day find a way to turn people into automations too. Maybe one day we’ll do that. Trade in engineers and artists and accountants and everyone else for ChatGPT 15, then speed away from our trashed planet on a luxury spaceship that caters to our every whim while we ride around on floating barcaloungers drinking Mai Tais.
But we are not quite there yet. Which means that no matter how many fastlane scripts you write or automated App Store Connect API calls you set up, your release process will still require human beings to make sense of things and bring it all together, What’s important is that you find the right mix of tooling and processes to support those humans instead of assuming you can write enough scripts to automate their (or your) cares away.
But let’s take a step back — why can’t you just automate everything away?
Automation only gets your mobile releases halfway there
This isn’t all to say that the Runway team doesn’t believe in automation or that we think you shouldn’t embrace it. We love automation. We provide dozens of automations for our own users. For people who don’t use Runway, we write blog posts about creating scripts to automate and improve disparate parts of your mobile releases. We clearly see enormous value in it.
Taken on their own though, automations often only end up picking the lowest hanging release fruit, replacing repetitive, annoying, prone-to-human-error actions like generating new builds or handling version bumps. It’s incredibly helpful for release captains not to need to take all this stuff on, holding a bunch of little tasks in their brain while they context switch through 18 different tabs. But releases are more than just lists of tasks.
There are cross-functional collaborative and human elements impacting how all these systems work together. You have an automation that generates a new build. Now what? Is there a script to run now? Who is responsible for it? When should QA be looped in? How can Product keep up with what’s happening and make appropriate, informed plans for what feature work can be included in this release and what needs to wait for next time? For that matter, how can anyone in engineering keep an eye on how things are going?
Workflows and tools for releases are very segmented and siloed. There’s no connecting tissue or glue between them — save perhaps for a poor 40 line spreadsheet that was last updated eight months ago. Which means someone (and usually many someones) have to pull all this together themselves.
You can’t automate away the need to orchestrate product, engineering, and marketing
A product manager’s job is to put together a plan for an update or feature, and then ensure it’s shipped on schedule. An engineer’s job is to ensure all the planned updates are finished in time to make it into the release. A marketer’s job (yes, even they have a part to play in the release, they’re not just writing LinkedIn ads all the time) is to write new copy and put together media to show on the app stores.
Usually a release captain is in charge of handling all of this orchestration. They ping all the contributors to ensure their work is completed, keep stakeholders up to date on progress, and put every single piece of the release puzzle in the right place, while also manually moving all the automations, scripts, and other work of the release along, jumping from tool, to tool, to tool while they do so. (Editor — please note that I purposely wrote that as a run-on sentence to generate empathy for the run-on nature of the work the release captains are doing.)
When a script or automation or some other part of the process doesn’t work properly (which is very common), the release pilot has to either fix it on their own or bring someone in to help fix it, keeping all the stakeholders updated as to why this part of the release is taking two hours longer than usual while leadership is breathing down their neck asking why the latest release hasn’t already been submitted.
That’s not even taking into account working with QA on testing the build, which has its own long list of coordination concerns. Here are a few questions to ask yourself:
- While QA is testing, how do they keep everyone updated on progress? Do they silently test until someone asks them how things are going? Do they constantly stop what they’re doing and announce how things are going like a town crier?
- Let’s say that your CI pipeline is generating builds and uploading them for pre-release distribution. That’s great. Now how does your QA team know this build is ready to test? Does someone update a Jira ticket? Ping the QA team on Slack? Tell a PM who assigns it to them?
- What if they find a bug? Who do they alert? Is there a clear product owner? An engineer to jump in and help? Where do they log all the details? How do they make sure the work is prioritized? How does everyone else make sure the work is prioritized?
- When a fix is made and a new build is generated, how does the QA team know this happened and which build to now test? How does engineering verify that QA isn’t accidentally still testing the last build?
Turns out that was more than a few questions and this is not even a comprehensive list. Every time a question needs an answer, there is an unavoidable, built-in delay. Even if the answer to a given question is easily available — and doesn’t require the expertise of someone who is ignoring Slack right now to get work done and has turned off their Jira email notifications (because they’re a normal person) — it’s still a distraction. Even short delays quickly add up.
Taking all this work on is a big responsibility. The release captain role is either being done by one or two people on a regular basis (taking up a huge chunk of their week) or tackled on a rotation by members of the mobile team who individually only have to pitch in a few times a year, but likely dread their turn because by the time it comes they’ve forgotten 75% of the things they need to do get the release out the door. Regardless of how your own team approaches it, automations don’t really solve any of this.
You can automate rollouts and monitoring, but you can’t automate transparency and judgement
The release isn’t over when it’s submitted and approved by the stores. Someone (usually the release captain) then has to keep an eye on application health data and user reviews to ensure the rollout of the newest release isn’t causing any notable problems that need to be handled before the newest release is rolled out to everyone. “Notable” is a very broad term, which means this monitoring monitorer also needs easy access to data and guidelines on how they should make judgment calls as to when problems are big enough to warrant halting a release to take action.
App health data is pulled from numerous sources: error monitoring tools to keep an eye on stability, product analytics tools to watch for changes in user behavior, reviews to see what people have to say after they’ve tried the newest release. Is all this data available in one place? Do you have robust notifications setup to alert the release captain when something goes wrong? How are the serious issues that need to generate a red alert separated out from the minor ones that may still generate a ton of noise?
Let’s say they make a judgment call that the rollout does need to be halted. Then the work of executing that halt needs to be done, causes need to be investigated, hotfixes need to be spun up, and new builds need to be tested and submitted — quickly. Unless you’re using Runway’s Rollout automations to auto-halt problematic rollouts and rollback to the last stable release, getting through these steps requires that the right information is put in front of the right people, and this work has to be managed across many of the same tools that were used to get the release initially out the door, but now in an even more stressful and time-constrained environment. And while the initial response (halting a rollout, initiating a rollback) can (with tools like Runway) be automated, decisions ultimately need to be made by the right people at the right time about how to proceed – whether that means completing a full rollback while the right team investigates an issue, or fixing things quickly and shipping a hotfix.
Automation cannot alone fix your mobile release process. You also need the sort of visibility that can better support the necessary orchestration of all the moving parts, scripts, people, and work that comes with the release of each and every version of your app. Otherwise, you risk automation turning your release process into a black box that solves a lot of small problems, but generates a bunch of new ones.
Though you can never fully step away from the mobile release process to let it run on its own, you can provide stakeholders with a single source of truth from which they can always see their part in an ongoing release, and a release captain can manage everything without going insane. How? Use Runway! Or, barring that, just ensure that when you lay out your own strategy and approach you make absolutely sure that visibility into your process and the connections between all the moving pieces are treated with equal importance as automation and the process itself.
