GitPrime builds a suite of reports that engineering organizations use to visualize their software development process. Engineering leaders use this data to understand trends, map initiatives to outcomes, and advocate for their teams.
Our founders’ goal in starting GitPrime was to research the software development process and develop a platform that would provide an unprecedented level of visibility into how engineering teams work.
GitPrime’s CEO and Co-founder Travis Kimmel recently spoke at the Silicon Slopes Tech Summit. In this talk, he examines the product development process behind our recently released Knowledge Sharing Report—shedding light on the outcomes of code review and how we believe software teams should approach ‘productivity’.
Here’s a lightly edited transcript from his talk.
Research, Development and Productivity
Travis: Thanks everyone. My name is Travis and I’m the CEO at GitPrime. I haven’t always been a CEO—I started in engineering, which is why instead of talking about GitPrime’s founding story or going on about what our platform does, I’m going to explore how we do R&D and share how we used that process for one of our recently released features. Researching how software is built is where my passion lies.
If you’ve heard of GitPrime, you might associate us with something like “productivity” for software teams and that’s good—it certainly is part of what we do. But, what doesn’t always get captured in that thought process is the craft. When using data, we don’t want to aim directly at productivity.
It’s about debugging the development process—removing bottlenecks and clearing process issues—and as a result, we improve the overall health and capacity of our teams.”
Engineers show up every day looking to build meaningful projects and have an impact—but sometimes things get in the way. We get pulled into meetings. We have to wait for responses to our questions in the review process. We get unclear requirements and end up having to re-do something. There’s a whole host of hurdles that can create daily brain damage.
But if we can fix recurring organizational issues and clear away all these barriers to getting meaningful work done, we return ourselves to our natural state of productivity. That’s how we think about productivity—it’s not about aiming towards “improving performance” or even “doing more”. It’s about debugging the development process—removing bottlenecks and clearing process issues —and as a result, we improve the overall health and capacity of the team (thus, returning to our natural state of productivity).
So, how do we ‘debug the development process’ and improve the health and capacity of our teams? The answers to those questions are what we’re actively interested in and researching at GitPrime. Today I’ll share with you how we do that research—and some of what we’ve found, specifically in the code review process.
Code Review and Research
If you’re not familiar with code review, here’s a quick summary: as an engineer, I write code, then open it for a review from my colleagues, who then provide feedback to improve the code or give it the thumbs up before goes into production.
So this process takes up a fair amount of time in teams that we’ve talked to. In some cases, it could be 20% of the team’s time or more. And so the first question we asked when we looking at this part of the development process was, why bother?
What are we looking to get from code review? Since reviewing code is competitive with time spent writing code, why do we spend time on it?”
The traditional answer here is safety; specifically, we’re looking to reduce the defect rate, which is the rate at which we ship bugs to customers and they get unhappy.
That’s a perfectly reasonable and good argument for code review. But the more people we talked to and the more studies we read, the more we heard about outcomes beyond just ‘safety’.
This is a quote from Camille Fournier, who wrote The Manager’s Path. Her tweet encapsulates what we saw, which is really that it’s funny how the narrative around the value of code review is often just about “catching bugs,” when there’s a whole lot more that we get from it. So much more, in fact, that “catching bugs” becomes less of the primary reason for why we encourage our teams to do code reviews and more of a side effect.
Our research found a few commonly emphasized outcomes of code review: things “knowledge transfer,” a “shared sense of ownership” of the code base, “mentorship,” and “improved solutions to problems.”
So when we look at these outcomes as a gestalt and we start to bucket it, they all fit under this umbrella of collaboration.
Collaboration and Wealth
It turns out code review is the locus for a lot of the collaboration that we do as engineers, and there are various ways to go about it. So next, we thought, why is the narrative often about ‘catching bugs’? Why are we selling code review as a way to catch ‘bugs’ to the broader organization, or to the leadership teams in our companies?
When you dig in, it starts to make sense: management—specifically upper management or non-technical management—understands bugs.
If someone asks ‘Hey, why are you spending a fifth of your time on this stuff?’ And you respond ‘Do you want good software or do you want buggy software?’ They will leave it alone because you have presented them with a magical ROI case, which is how upper management understands bugs.
What’s unfortunate is that we don’t really have a great way to express a lot of the other value that we get from code review.
We set out to give a voice to all of this other value that we get from code review.”
When we do product development at GitPrime, the first thing that we do when tackling big new ideas is to search for a good metaphor. It’s a really good strategy if you’re dealing with a big juicy concept to help people connect with that feature, out of the gate.
You’ve all seen this before with things like technical debt—allocating a portion of your budget to paying it down. If the debt isn’t paid down, you will go bankrupt which results in waking up one morning, looking at your code base, and thinking it would really just be easier to start over. So we talk about this occurrence as ‘technical debt’, and so we often handle the problem as we would with credit card debt.
we tried on a few metaphors for code review: including the concept of ‘dilution’ where maybe collaboration on teams happens in the similar way as an Ala-Seltzer dissolves in water. *That didn’t work because people have intent—collaboration doesn’t just happen and spread throughout a team. We also considered the concept of ‘hive behavior’ and walked out the math behind that metaphor, applying it to collaboration in teams, but we abandoned this metaphor as well. (Readers note: this section is highly edited and summarized. If you want to hear more about this research, let us know.*)
In our research, we kept hearing about “knowledge silos” and how important it was to account for them. We moved on to the final metaphor, where we landed, which is this idea that perhaps collaboration and knowledge on teams are like a form of wealth.
Collaboration and knowledge on a team is a form of wealth.”
Wealth is an inherently human thing for us to think and talk about. This led us to Corrado Gini, a statistician and sociologists in the early 1900s, who developed what is knows as the Gini Coefficient.
Known as the Gini Coefficient, this theory is used by the UN and the World Bank to evaluate how well a society is doing at distributing the income generated among the people in that society. It has also been applied in chemistry and neuroscience, and so we thought the leap the theory had made to other disciplines was a good indicator that this was something we could use.
The Gini Index does a really good job of distilling the motion of a complex system, like an economy of a country or the code review motion of a thousand engineers on a big team, down to a single trackable metric.
This is great because upper management can walk in and say, ‘you know what, I’m actually not sure that the time spent on code review is worth it.’ In that situation, you need to have something else to express all this other value that we get from the code review process. Part of the way to do that is say, look, we’re really investing in things like spreading out knowledge on the team so that we can all work effectively in the code base; Here’s a picture of it and we can measure it, and we should invest in it and do more of it over time, and I promise you we will.
It’s very powerful when you allow people to pop open the hood and peer underneath to see how the combined actions of teams contributes to that overall number. Helping leadership visualize that information is how we can elevate our company’s and the industry’s understanding of how teams work and how software gets built.
Expression through Quantification
Our users tend to use GitPrime’s reports in a lot of different ways, but the one thing that we hear universally is that quantification is important. It makes a lot of sense because ultimately, numbers are the language of business. While in engineering we have a lot of numbers around the customer side, we don’t have a lot of information around the human side. It’s really up to us as engineers to catch up so that we can express ourselves with the same amount of effectiveness as other departments in the enterprise. If you can express yourself with data, you can get mandates for budgets, tools and all that good stuff that we want at work.
This is the root of what’s underneath our approach to productivity. When we can bring data into the picture and understand how teams work, we can collaborate with each other more effectively, we can adjust processes more effectively, and we can more effectively map initiatives to outcomes —leading to productivity organically along the way.