Identify and Remove Impediments with the PR Resolution Report

News & Updates

Identify and Remove Impediments with the PR Resolution Report

A common practice in engineering teams is to hold regular meetings throughout and after projects, events, or sprints. Whether a “debriefing,” a “retrospective,” or a “standup,” these are all generally mechanisms for looking at where we are or what we did—and for discussing topics like what worked, what didn’t work, and what we’re going to do moving forward.

The aim of any these meetings is to get the team together to find common ground, to learn, and to create alignment around what’s needed moving forward.

A manager’s role might be to facilitate a productive session and encourage healthy collaboration. It might mean bringing data about “what happened” or “how it happened” and codifying the lessons learned. As for outcomes—beyond the benefits of providing direction and creating alignment—managers can leverage this time to find opportunities to better support the team or help remove impediments that are slowing the team down.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

Agile Manifesto, 2011

The problem managers often face, however, is that these meetings don’t lend themselves to clear opportunities for the manager to remove impediments or better support their team. They might notice one-off situations that they can help with, but the broader patterns are difficult to see and manage.

The reason for this might be that the team members don’t regularly notice process issues, so they don’t bring them up. When you are working hard on the task at hand, it can be difficult to notice the broader systemic issues. A small process adjustment could make a significant impact, but it’s difficult for managers to identify where those opportunities lie.

When this topic arises in a discussion with a customer, one my favorite places to point them toward is the code review process. While there are opportunities to adjust processes and remove bottlenecks across the development lifecycle, the code review process in particular often shows clear opportunities for quick wins that will help the team as a whole.

Today we’ll focus on two metrics in the PR Resolution report that are great examples of this: Time to First Comment, and Time to Resolve.

Time to First Comment

I’ve done some informal polls with managers and one of the things I like to ask is, “what consistent feedback do you receive from your Engineers regarding the delivery process? What drives them crazy?” And I frequently hear the response: “They open a PR, and then they wait. They have to wait for a long time before a Reviewer picks it up.”

When that happens over time, you start to see these storylines form: “Sally takes forever to pick up my PRs,” or “anytime I work with Billy, I always end up waiting a lot.” For a manager, the instinct might be to go nudge Sally or Billy. But there might be an underlying systemic issue at hand that needs a different approach.

So when I hear this, I typically head straight for the PR Resolution report and look at Time to First Comment. There, you can immediately see how quickly the team usually responds to PRs. You can view the outliers to figure out why it took so long for someone to respond. Maybe the PR was massive so everyone was avoiding it. Maybe the PR was submitted at 5pm on a Friday and wasn’t commented on until Monday morning, so it’s showing up as an outlier. Or, maybe the manager needs to set expectations around responding to PRs so the team knows that it’s valuable and important.

Time to First Comment: the average time in hours between when a pull request is opened and the time the first Reviewer comments on it.

The problem with having an unusually long Time to First Comment is straightforward: it puts people into long wait states that can erode at the respect that’s shared within the team. Conversely, having a short Time to First Comment means that people are quick to acknowledge their colleagues’ work.

In most cases like this, the team members just didn’t realize that this behavior negatively affects their teammates. So by creating visibility, and setting expectations around this metric, you make it known that this behavior is important and can help eliminate wait states. As a result, you also reduce task switching and help team members get feedback earlier and ship code faster.

Reducing Time to First Comment speeds up the entire process that follows: responding to comments, incorporating feedback, getting those changes reviewed, and so on. Keeping this number low is a team effort. The manager can encourage the team by ensuring them that it doesn’t matter who gets assigned to the PR, it matters that the team acknowledges the open PR and gets on it as soon as possible.

Time to Resolve

Time to Resolve: the average time in hours that it takes a Pull Request to be resolved (either merged, closed, or withdrawn).

Because Time to Resolve looks at the time between when a PR is opened and when it’s resolved, it takes into account Time to First Comment. By reducing Time to First Comment, you reduce Time to Resolve.

But Time to Resolve looks at a broader picture of how work is being handled in the review process. A manager can look at the outliers—the PRs that took an unusually long time to be resolved—to figure out why they were atypical. Here are some examples of reasons why a PR might take a long time to review:

  • The work submitted for review was large  (multiple days or even weeks worth of work) and either people were avoiding the PR or it took someone an understandably long time to thoroughly review all of it.
  • There were a few back-and-forth comments early in the PR followed by silence, a sign of potential disagreement or confusion (perhaps caused by unclear requirements).
  • There were long periods of time between the back-and-forth comments.

In any case, these “long-running PRs”—or PRs that took (or are taking) an unusually long time to resolve—are a signal that there’s something outside the normal development process causing an issue.

You can take this information to your next retrospective or standup and open it up for discussion. The data you’re showing is the common ground upon which you can have a productive discussion; you can say, “Here’s what’s happening. What’s occurring here?”

In the case of Time to Resolve, the goal is to focus on shrinking the long tail—identify the behavior contributing to the atypical pull requests so that over time, your x-axis is shortened and has fewer PRs.

As with Time to First Comment, we generally want to drive this number down. Pull Requests are a period of review. They should open, there should be discussion, perhaps some follow-on commits, and they should close. By setting expectations around Time to Resolve and working towards a consistent resolution rhythm, the team will ultimately experience fewer delayed releases and improved predictability.

Final thoughts

We’ve found that when it comes to adjusting processes in a way that helps the team as a whole, it’s best to just focus on one or two concepts at a time. If you’ve already been tracking these two metrics with your team and would like to identify additional ways to support your team with data, reach out in the chat below.

Ultimately GitPrime is designed to help engineering leaders support their teams—whether that be by removing impediments, advocating for the value delivered in the underlying workflows, or creating opportunities for improvement that matter most to the team.