Upcoming Webinar: Scaling Alignment in Engineering Teams Register Now

Implementing Problem Management at Any Size

Perspectives in Engineering

Implementing Problem Management at Any Size

VP of Engineering at Medidata Solutions
GitPrime elevates engineering leadership with objective data. In this interview series, Engineering Leaders talk about how to build high performing teams.

See if this sounds familiar: A client contacts your customer service team with a technical problem, and customer service representative starts shooting emails to the development team and/or other random associates to attempt to “solve” the problem.

That’s a shotgun problem management approach, where you target as many people as possible and holler for attention to get your client’s problem solved ASAP.

Sure, that model works… unless your definition of “works” involves any level of efficiency in managing problems, enhancing your product, and building lasting relationships with clients. In which case, you might want to instead tear out a chapter of Ben Young‘s playbook.

Young, the VP of Engineering at Medidata Solutions, has focused his professional career on clinical trials software development for the last 20 years. So he’s seen his share of product issues that need immediate correction since ultimately human lives are on the line.

Young has led numerous problem management successes in rapidly expanding companies. The first company he started had four people. When he joined Medidata, it had 450; now it’s growing close to 2,000 employees, with 100 software development teams working on 15 products and 30+ web services in four primary technology stacks/languages (.NET, RoR, Java, Scala).

“Scaling your problem management is about putting layers and processes in place that help you scale beyond relying just on small tribal knowledge to that of a larger-scale, mature organization,” Young says.

We asked him about his strategies for implementing those scalable problem management processes for companies of any size.

quote from VPE at medidata on scaling problem management

It’s all about the client experience

When a company starts off small, you don’t need a system for managing problems with your product. Everyone gathers in one room, and you can triage any issues as a unit. Everyone knows the code base, and they know what systems they have control over.

As your company grows, however, that kind of tribal knowledge doesn’t scale. You must figure out new ways to effectively manage incoming problems – because as you scale, so will your interactions with clients.

“There are two keys to the ultimate client issue experience,” Young says. “Having a closed, well-defined problem management process, plus maintaining a culture that has a fanatical focus on client satisfaction.”

Let’s discuss the problem management process first.

Set up a closed system

“When I say ‘closed system,’ I mean that you have well defined inputs and outputs,” Young explains. “Inputs are incidents and problems that will require investigation. Outputs could be one of many different things, once an investigation is closed.”

At Medidata, all incoming client problems go through the Tier One Customer Success group (help desk). The first tier in our four-tier system is built to filter through issues.

Tier One works with clients on troubleshooting problems such as user account access, password resets, and maintaining optimal internet connections. They also pass along more complex issues through a well-defined system that culminates in Tier Four. The Tier Four team consists of engineers, testers, and product specialists who deeply and intimately know the products and can resolve problems that are severely impacting a client.

graph of medidata's customer success process

The purpose of this tiered system is three fold: to efficiently leverage resources, learn from mistakes, and reduce complexity and burden upon the customer.

“In order to effectively leverage resources, you need a knowledge base in place that allows each tier to quickly respond to issues as clients contact you. However, if you let too much come through to that Tier Four team, then their focus on product development is diluted,” Young says. “They’re constantly focused on resolving support issues.” Conversely — “If you don’t let enough pain-points funnel into the system,” Young adds, “the support issues become so problematic at the early stage tiers that you have an ever-expanding customer support organization that’s growing faster than the development organization, which is not optimal.” This balancing act will require an ongoing evolution of the system.

In other words, if your team can effectively manage the problems that come in (and kick out resolutions quickly), your product development will make your clients happy on the back-end. And you won’t need customer service to constantly do all the client-pleasing work for you.

Saving customer service by prioritizing and problem solving quickly

Focus fanatically on the client

Forget whether or not the customer is always right – a truer (if less catchy) axiom is “where there are customers, it’s up to you to ensure they have a really satisfying user experience.” Mapping this out, the life-cycle would look like:

Client Pain → Temporary Relief and Plan → Full Resolution → Lessons Learned → Optimize Processes

Your approach to managing problems as you scale isn’t necessarily about fixing problems. It’s about addressing your clients’ problems, and making sure they feel that you’re paying attention to them (and relieving their pain) no matter which person or team is addressing the issue.

The crux of this distinction is in the term “problem management.” You’re not “problem solving,” and it’s not always incumbent on you to rewrite code for your client. Instead, if you have a client-focused culture, then your top priority is to offer your client a smooth process through resolving their issue.

“We don’t actually have to fix the defect,” Young explains. “We don’t have to create the knowledge object, nor do we have to create the enhancement to close the investigation. Once we know what is going on and once we know how to relieve the client’s pain, we’ll go back to the client and show them how we’re making progress.”

For instance, let’s say that the problem is a defect in the product. You can let the client know that you’ve registered the defect and that is can be tracked. If it’s a known issue you can thank them for their report (incident) and then provide them with tracking information to follow the defect through its lifecycle. If the system is behaving as expected, you might let the client know that you’ll be improving your product documentation to better set client expectations or perhaps you’ll enhance the UX (feature request) to avoid clients traveling down dead-end paths. In any case, the “problem” is closed once you have a plan in place and it’s communicated back to the client. As more incidents (reports) of the same problem come in they can be connected to the same problem and perhaps drive prioritization of the full problem resolution.

Once the client has been informed of why they had a problem and the path forward has been communicated, it’s critical that all of the promises made in the complete resolution are kept. This is final piece of problem management, but is sometimes forgotten or missed due to tools and/or processes that don’t allow you to track those commitments.

With just one incoming issue, that seems simple enough. But as you scale, you could be dealing with thousands of inputs each month. That’s where a scalable system is invaluable to a company of any size, no matter how small or how green.

No time like the present to get started

Think of the quantity of incoming incidents and problems your company is currently handling. Now imagine, what happens when you start to have 25,000 help desk tickets each month? What if you acquire three more companies, or develop two new products, or added three more tech stacks, or grew to 40 development teams?

The numbers are arbitrary, but the strategy is not. Start to formulate a basic understanding of problem management now and consider how you want it to work in your organization.

“It can be very simple if you start early,” Young says. “A straightforward problem management process serves a world of wonder to an organization of any size.”