Organizations increasingly rely on data to make sense of complex problems. Teams like sales and marketing rely heavily on data to set goals, track progress, and make informed decisions. Engineering teams stand to gain a lot from taking a similar approach.
Still, engineering teams must take extra care to place engineering data within its human context. Numbers can only tell you so much. By considering the data alongside a team’s day-to-day realities, leaders can gain actionable insights to inform decision-making and drive progress.
Visibility Is Critical
Understanding the flow of work on an engineering team can be challenging because it’s often spread across disparate systems, like Jira and Github. Weekly standups and technical demos can help create visibility, but this only works if team members are open to feedback and comfortable presenting in front of a group. Even in a welcoming culture, team members may not speak up about issues they're facing — they may be too shy, want to solve the problem themselves, or not even realize something’s wrong. Data can help teams identify potential issues and figure out where to dig deeper by spotlighting things that aren’t moving as expected and demonstrating correlations between different aspects of the development process.
For example, consider a team that’s constantly berated because customer success and sales think it takes them too long to get a change to production. Individuals blame different parts of the software development cycle, and no one can determine where the slowdown occurs. The team leader looks at the metrics to verify that the team is moving slowly and discovers that mean lead time for changes (MLTC) takes an average of two weeks — twice the industry average.
They dive deeper and analyze metrics corresponding to the stages of their development pipeline, like time to open, time to first review, and time to merge, and identify a bottleneck in the team’s process: Code review is taking over a week and slowing down the entire pipeline. They bring that data to the team and have an informed conversation about what might be slowing down the review process. It turns out that team members are opening a lot of large pull requests (PRs) to proposed code changes, which take a long time to get picked up and make it through review. The team decides to break work into smaller chunks to reduce the size of PRs and boost the efficiency of their code review process, which reduces lead time for changes from two weeks down to one. Data was invaluable in getting to the root cause of the issue. Instead of finger-pointing or blaming a single person for slowing things down, the team identified a bottleneck. They made a team-wide change to improve efficiency and get customer-impacting changes into production twice as fast.
Context, Context, Context
Context is critical when managing engineering teams, especially in a distributed work environment. Consider an engineering team that transitions to remote work for the first time. The team lead notices that throughput goes down when engineers start working from home. Instead of concluding that engineers are slacking off, the lead looks at the data and discovers rework is up. After talking to their team, the leader realizes this is because people don’t understand how their work comes together. In the office, casual check-ins and overheard conversations provided a lot of perspective on what others were doing and how one individual’s work might impact the project as a whole. This lack of understanding meant that developers were operating with less information than usual and making assumptions that weren’t always correct. This resulted in work needing to be redone, ultimately leading to a drop in throughput.
To address this issue, the engineering manager makes a concerted effort to improve communication and build new processes to ensure everyone understands what they’re working on and how it fits into the larger project. The team gets back on track by creating clearer guidelines, improving documentation and creating visibility by asking questions in team channels rather than private ones.
It’s important to consider each team's unique context. Leaders must know how their team functions and identify issues early on to ensure the team can continue working efficiently and effectively. Effective communication, clear guidelines and regular check-ins can all help improve a team's overall performance and ensure that the work is completed efficiently and effectively.
Don’t Take Data at Face Value
There is an ideal way to use data. It’s important to use tools that can aggregate data from multiple sources and present it in a way that’s easy to understand and view in context with other metrics and qualitative information. It’s important for leaders to be open and transparent about what they’re looking at and why and to take any data that doesn’t meet expectations and bring it to the team for further context and discussion. Lastly, leadership should review and refine how data is being used and adjust as needed to meet the needs of their team.
Finding the balance between the human experience and technology is critical. That’s how teams can stay engaged and aligned on goals. There is no “I” in the team, and numbers can only say so much. Don’t take data at face value or take it for granted.
James McGill is vice president of engineering at Code Climate.
***To comment on this article, please scroll down past story recommendations to "Voice Your Opinion."
