Living with the gray areas

Agility is about living with the gray

In our travels as technologists we are constantly engaging in conversations with people about how technology works, should work, or has worked in the past. We spend time considering what we should be doing next and helping organizations to navigate through their individual challenges while attempting to deliver something great to consumers or simply to make the organization run smoother by utilizing technology to automate those things that are hard. We technologists write code, discover new things as we go along - our job as technologists is to live in the area outside of the known paths - we live inherently in the gray spaces where things haven't yet completely solidified. The idea isn't new but is just starting to come into power, this idea of living in the gray spaces was brought to light when the Agile Manifesto was penned - the manifesto opens with the following:

"We are uncovering better ways of developing software..."

Indicating that we don't yet know all the ways we might achieve what we set out to accomplish, but we'll discover at least one way. Along the way we may find many ways in which things don't work as well - but in doing so we'll learn what OUR path should be, or what path works best for us in a given situation at a given time.

Software development is all about living with the gray

When we as technologists (developers or otherwise) set out to do something new or to make changes to a system that already exists we are asked to 'create' something that meets a described need. When we start out, we may not be entirely sure how we are going to do the action of creation. At this moment we are confronted with hundreds of choices and hundreds of potential directions. We may feel a little overwhelmed but we are relatively well practiced at living with the gray space that choices create. We quickly navigate through the list of things we know, the list of things we don't know and start to devise ways in which we could experiment so as to understand what we don't know and "clear the fog" from the gray areas in our understanding.  We may write example code as a way to experiment, perhaps a scratch refactor of existing code to fit in a new feature or trying something completely new by picking up a new technology or a new software stack. We use these techniques to help us get through the gray areas, solidifying things as we go.

One of the big problems is that while technologists are aware of and deal with gray areas in their understanding fairly well, most people that they deal with do not.

Humans deal with gray areas poorly

Inherently we humans seem to like control or at least to think we are in control. Uncertainty makes us uneasy. Equally, when there are a great number of choices that we could possibly choose from, 
we feel trapped and our fight or flight reactions start to take hold. Outside of technology this is actually quite un-nerving to most people.  As I stated above just about everything in technology is about living solidly in the gray areas and slowly working to make them less gray and more certain. The businesses that we work with and for however don't see things quite that way. Every company I have ever worked for would like to think that software development is an inherently known quantity, that software development easily equates to something that they know is straight forward like building things or assembling items. Software however is a creative process that defies being estimated up front.  At the very start of a new endeavor we might be able to provide some context and information which can inform an estimate for how long it will take to accomplish but we should allow that initial estimate to more accurately relate to the gray that surrounds it, all the things we don't yet know. That first estimate should have a confidence interval - say 25% confidence with a list of all the gray areas that surround this beginning estimate and a set of steps for how those gray areas will be solidified and more well understood. This changes the discussion from - "Are we on plan for finishing X" to "What did we learn today and how did it impact our confidence we'll be able to do what was asked".

In short...

Software development, and by extension Agility, and other software development frameworks/methodologies have to help us deal with the gray areas. The systems we choose to use when developing software should allow us to embrace the gray areas. Frameworks we know about currently, Scrum, SAFe, Kanban all describe ways in which we can get context around what we are doing and allowing us to wipe away the gray. We can use small experiments to guide us to better places. We can learn to fail fast, as the faster we fail at doing something the more we learn. We can embrace the gray areas because that is where the learning is and it is where we expand our horizons.