Methodical delivery experiments

How can you learn to leverage product experimentation methods to continuously improve the delivery process in your team?
March 6, 2023
3 minutes
Matthias Dyckerhoff
Product Manager

Why experiment with delivery?

It shouldn’t come as a surprise to anyone, let alone Product Managers, that there is no one size fits all method for product development. Part of the rekindled debate about the effectiveness of Scrum lies in the inherent truth that Scrum itself leaves intentional gaps that are meant to be navigated by teams themselves. Indeed, a number of now-standardized practices like 2-week sprints, story points, and measuring velocity came from teams that took the framework of Scrum and built their own additions on top of it. The same principle of making a framework your own can be applied to teams that have adapted other methodologies like Kanban or Shape Up.

In a way, it would be nice if there was a magical solution that worked for every team. But at the same time, part of what makes the PM role exciting is being able to break away from convention, discover new ways of working, and making the delivery process you use your own.

Indeed, as I have experimented with a number of frameworks over the past few years, I have found that creating an effective working environment for your team is not a matter of following a given framework more precisely, but continuously adapting the framework you choose in order to increase its effectiveness.

The reason that no one size fits each team is that there are a lot of dynamics that affect product delivery across industries, companies, and even teams in the same company. Team familiarity, team size, level of industry regulation, company culture, team seniority, team composition, and tech stack can all play a role in effecting your delivery process and what success looks like as a product team.

Experimentation is all about optimization and continuous improvement. And just like in-product experiments, it’s important to set the goals and standards before you start experimenting so that you stay focused and are able to measure success. Rather than focusing on profitability or conversion rates, however, you’re focusing on concepts like agility and predictability.

Photo by Markus Winkler on Unsplash

Understand the tradeoffs

The first step in improving your delivery process is identifying what success looks like. Based on your organization, industry, team makeup, and goals, what does success on your team look like? Is it having predictability in release cycles? Is it speed of execution? Is it to foster a product mindset in your development team or to give them tasks and let them focus solely on execution?

Many development teams these days are used to measuring velocity and generally know how effective they are at releasing on time. However, these metrics do not share the whole story about how your team is performing, and over-indexing in one area tends to effect the ability of your team to perform in other areas. For example, focusing primarily on velocity may effect how involved your engineers are in ideating and making product decisions. Focusing on predictability of releases may lead to building a roadmap that doesn’t allow for rapid implementation of new insights and a may place a lesser value on continuous discovery.

Before implementing any changes, understand the tradeoffs you are making, and be aware of how one area of focus may effect the performance of your team in other areas.

Sample areas of focus:

  • Predictability of release cycles: Are you delivering on time?
  • Impact of delivered work: How effective is what you release at hitting your targets?
  • Agility: How quickly does your team turnaround new work?
  • Agency of engineering team over work & tasks: How close are your engineers to product decisions?
  • Collaboration with cross-functional teams: To what extent are tickets collaborated on vs. being passed down from team to team?
  • Connecting tasks to business goals & outcomes: Does your team know how their work relates to company success?

These areas of focus should come from your functional and leadership teams in order to get their buy-in, but can (and should) also come from your knowledge of other teams, best practices, and what kind of team you aspire to build and lead as a Product Manager.

Leverage existing frameworks & research other companies to ideate solutions

I personally tend to favour work environments where engineers are treated as more than ticket takers. During my research, I found the delivery framework Shape Up refreshing in terms of its approach to creating a more equitable product decision making and execution, where the people shaping the work are close to execution. I was therefore more prone to build experiments that increase agency of designers and engineers to work more closely in parallel, and increase our turnaround time for new work. This impacted pure velocity and accurate forecasting of releases but lead to releases that were more impactful when delivered.

Here are some quick examples of experiments to try based on a given area of focus:

Listen to thought leaders and PMs that are paving the way in experimenting in these areas. When you see a team working well in one area that you want to improve in, take initiative and try ways to work it into your current work environment. However, don’t just experiment for the sake of it – pick out changes that will help you move the needle in the areas you identified at the start.

Be patient when measuring success

It can be tempting to try to implement new working methods in parallel in order to bring the team where you feel they need to be, especially when joining a new team. However, unlike in-product experiments that only take weeks, days, or hours to see a statistically significant result, changes you make on a team may take anywhere from weeks to months before they stick in your team and start delivering the results you’re after. Stay patient, introduce changes marginally over time, and adjust your craft as your team matures, grows, and adapts. It’s the only way to keep improving.

Matthias Dyckerhoff
Product Manager

Interested in how our team can help you?

Get in touch with our partners to learn more
Thank you! A member from our team will be in touch soon.
Oops! Something went wrong while submitting the form.