Living with the gray areas

Agility is about living with the gray

In our travels as technologists we are constantly engaging in conversations with people about how technology works, should work, or has worked in the past. We spend time considering what we should be doing next and helping organizations to navigate through their individual challenges while attempting to deliver something great to consumers or simply to make the organization run smoother by utilizing technology to automate those things that are hard. We technologists write code, discover new things as we go along - our job as technologists is to live in the area outside of the known paths - we live inherently in the gray spaces where things haven't yet completely solidified. The idea isn't new but is just starting to come into power, this idea of living in the gray spaces was brought to light when the Agile Manifesto was penned - the manifesto opens with the following:

"We are uncovering better ways of developing software..."

Indicating that we don't yet know all the ways we might achieve what we set out to accomplish, but we'll discover at least one way. Along the way we may find many ways in which things don't work as well - but in doing so we'll learn what OUR path should be, or what path works best for us in a given situation at a given time.

Software development is all about living with the gray

When we as technologists (developers or otherwise) set out to do something new or to make changes to a system that already exists we are asked to 'create' something that meets a described need. When we start out, we may not be entirely sure how we are going to do the action of creation. At this moment we are confronted with hundreds of choices and hundreds of potential directions. We may feel a little overwhelmed but we are relatively well practiced at living with the gray space that choices create. We quickly navigate through the list of things we know, the list of things we don't know and start to devise ways in which we could experiment so as to understand what we don't know and "clear the fog" from the gray areas in our understanding.  We may write example code as a way to experiment, perhaps a scratch refactor of existing code to fit in a new feature or trying something completely new by picking up a new technology or a new software stack. We use these techniques to help us get through the gray areas, solidifying things as we go.

One of the big problems is that while technologists are aware of and deal with gray areas in their understanding fairly well, most people that they deal with do not.

Humans deal with gray areas poorly

Inherently we humans seem to like control or at least to think we are in control. Uncertainty makes us uneasy. Equally, when there are a great number of choices that we could possibly choose from, 
we feel trapped and our fight or flight reactions start to take hold. Outside of technology this is actually quite un-nerving to most people.  As I stated above just about everything in technology is about living solidly in the gray areas and slowly working to make them less gray and more certain. The businesses that we work with and for however don't see things quite that way. Every company I have ever worked for would like to think that software development is an inherently known quantity, that software development easily equates to something that they know is straight forward like building things or assembling items. Software however is a creative process that defies being estimated up front.  At the very start of a new endeavor we might be able to provide some context and information which can inform an estimate for how long it will take to accomplish but we should allow that initial estimate to more accurately relate to the gray that surrounds it, all the things we don't yet know. That first estimate should have a confidence interval - say 25% confidence with a list of all the gray areas that surround this beginning estimate and a set of steps for how those gray areas will be solidified and more well understood. This changes the discussion from - "Are we on plan for finishing X" to "What did we learn today and how did it impact our confidence we'll be able to do what was asked".

In short...

Software development, and by extension Agility, and other software development frameworks/methodologies have to help us deal with the gray areas. The systems we choose to use when developing software should allow us to embrace the gray areas. Frameworks we know about currently, Scrum, SAFe, Kanban all describe ways in which we can get context around what we are doing and allowing us to wipe away the gray. We can use small experiments to guide us to better places. We can learn to fail fast, as the faster we fail at doing something the more we learn. We can embrace the gray areas because that is where the learning is and it is where we expand our horizons.

Roles for Making a great (software dev) team

I wanted to take the time to try to document the roles I think make up a great development team. This is essentially that raw brain dump - would be very interested in engaging discussion and conversation around these. What do you think - how has it worked for you - would you refine these in any way?

The Constructive tinkerer

This role describes a person that spends time looking at the problems that are facing the team and looks for ways to solve them.  The solution can be already contained in the things the team does or may be something that team has not yet thought of or is currently using in the way of technology. This person spends the time to investigate, research, experiment with the different, better, worse ways of doing what needs to be done inside the team. This individual is often not directly involved in the shit triad*, meaning this role is often not doing product development work but is instead creating nice paved road surface for the folk who are part of the shit triad* to move faster and get more done in shorter periods of time. The work this person does tends to lay the ground work for a great deal of other work that the team does. Clearing the path and making things easier, more visible, more operational.

Additionally - this role spends time providing guidance and mentorship for the other members of the team. Sometimes the discussions being had are technical in nature, things like "How should I implement this new hot thing that product asked for..." and sometimes the questions/discussions are procedural and more team oriented... "I think that we need to make this type of error more visible and loud so that we can find a root cause and eliminate it." This role plays a part in providing a sounding board for a great number of activities in the team working to help the team understand the aspects of their software running in a production data center with real users, whom we want to delight - not make angry.

The key for this role - is that the tinkering with things, the new code, the clean up of old code, the discussions are all oriented in positive "We will use this ways". When the result of this roles tinkering isn't useful to anyone, the person playing this role doesn't get upset, nor defensive - they simple tuck that work away for some later time, or they drop the work all together. The constructive tinkerer is focused on what the team working with them needs. Decisions about the longevity of something are biased towards team needs and wants but balanced with what the larger organization needs from the team as well.

The curator (Those that care do)

This is an oft overlooked role for every software development team. This role may not even be a single team member but encompass how the entire team thinks about the code base that they work in every single day. The curator(s) are those that are deeply invested in the well being, well meaning, and intention of each and every part of the code base. Curating the code is important because as we all know, code that is no cared for tends to suffer from:

Bit Rot
Code that is left to sit eventually rots away. Of course it doesn't actually rot, but what does happen is developers lose touch with what the code was supposed to be doing. There is a lack of documentation, so there is no longer any institutional knowledge of the code and no way to gain it other than wading back into the code base to find out whats in there. Bugs stop getting fixed and people eventually would rather replace all the work than 'figure it out' all over again. This can be avoided when there is a curator around - because they are constantly looking at what can be made better - seeing where problems are and will get upset when something sits for too long without getting any touch/attention.

Different points of view
It is rare these days that a code base have a single developer. All the code that I work on is managed and maintained by either a team or multiple teams of people. The curator has the un-envious job of reconciling the differing ways that things in the codebase are accomplished. Ties are broken, discussions are solidified into a 'way' in which the code will represent a specific thing. The code gets cleaner as a result. Differing points of view have to be recognized and addressed by a curator or you will easily be able to tell which sections of a code base were written by whom based on taste and patterns of development.

New developers
New developers to the team are simply new and have to be shown a way. This demonstration is made easier when they can see that the code base looks a certain way and does things in certain ways. This is made better when they are also able to consult a curation individual for a team. Work can be made even cleaner when new developers know that there is a pattern that exists that they can follow and that the team is open to discussion for new ways and new ideas to change existing patterns.

New shiny thing
A curator helps to combat running off and implementing the next hottest, best, new, shiniest thing in terms of development. Sometimes new patterns and new ways of doing things are beneficial or needed. A curator helps to understand how that can be phased into the code base and how existing patterns can be touched and adjusted as a result. The curator helps to understand how much time and effort is involved in moving the WHOLE code base into a new direction or if the new thing should only be used in certain locations under certain conditions.

Those supporting the shit trinity - Knows Shit, Gets Shit Done, Gives a Shit

So every team needs some number of people that have the ability to get things done... and get those things done well. Those that support the shit trinity make up a fair majority of a team and ideally everyone on the team would fall into this category to some degree. This means that everyone on the team has the ability to play a roll in accomplishing the goals that the team has set out to accomplish. Sometimes the goals are development oriented and related to software design and coding, other times the teams goals are about the process that they use to get their work organized and ready to work on.

The helper/garbage man

You always have to be willing to take out the trash. This is another role that the entire team should be adjusted towards. The garbage in the case of a development team is the grunt work of fixing bugs, re-organizing code when it makes sense, basically anything that would be considered the bottom of the barrel in terms of development effort. The tasks involved are generally just tasks that need to get done in advance of getting to the nitty gritty of some new development that has been planned. Every team needs at least one, and hopefully the entire team would be willing to do this work because they know it is necessary. Having the team be adjusted to taking out the trash means that no one on the team feels like it is demeaning to have some amount of their work be involved with taking out the trash in the code base.

The adage that I think goes with this is that a good manager, exec, CEO doesn't believe that certain tasks are somehow beneath them, they see that it needs to get done and that currently no one is doing it so they go in pick up that slack and get it accomplished and then later look for ways to not miss getting that bit of work done in the future. For a team - this mentality in at least one person has to exist both because it is necessary to pick up what is left behind but also because it encourages others to also pick up what is left behind. In essence having an individual on the team that "takes out the trash" encourages people to not throw stones at already broken windows but instead to fix those windows and move onto something else. 

The thinker (although they all have to be this way to some degree)

Software design, architecture and over all capability of the code to do a given task takes thought. It is not something that flows from the fingertips of generally smart people perfect the first time every time. Software development is a thought process, a process sometimes of trial (doing what you think will work) and error (seeing that what you did doesn't work) in order to get to the point where something is produced that meets the requested/stated needs of the larger organization. Sometimes the thinking on the team is encapsulated in one individual but I find more often that while one individual seems to do a great deal of the thinking/design that everyone seems to rally around that "nucleus" adding their own identity and thought process to the original. Making the thinker role more of an instigator role most of the time, using the power of the group to find all the holes in a design, to talk about all the corner cases in what we are doing and to aid in preventing the entire team from backing themselves into a development corner that they can't get out of. The thinker/instigator provides that stable guidance for architecture so that it evolves and morphs rather than needing a revolutionary re-write to get the team out of a bind.

Verifiers  (although they all have to be this way to some degree)

Everyone should be interested in verifying their work.  Sometimes this is a single individual that acts as the last safety valve for a team, making the last checks, T's crossed and I's dotted before the result of a development cycle, no matter how large, is verified to be doing what it was expected to when the development got started. In our team this has been encapsulated into a final check before deployment of the out put of the API.  When we are getting ready to do the deployment for a given day, the code that has been worked on is deployed into a staging environment and then a script is run to look at the syntactic differences in the output of the endpoints in the api space. Differences between Staging and Production are noted, discussed and either acted upon because they were unexpected or dismissed as expected due to the code being deployed. Taking this action in the form of tests that each developer wrote during their development in the early part of a cycle and doing this final check at the end causes us all to 'verify' our work. Similar to the thinker above - when the team sees that this is expected they tend to start executing that way on their own with little to no nudging required.

The reporter - Someone has to tell the world

In the end the WAY in which you work is important, but just as important if you want others to work in the same way is that you go and tell them what you are doing and how you are doing it. Enter the reporter of the team. This is someone who likes to taut what the team does and succeeds in accomplishing. In a shout from the roof top sort of way this individual helps the success of the team become more common knowledge and more well understood by other teams. This helps to spread ideas and processes and allows others to start to ask questions as they investigate their own teams and how they operate to accomplish the goals they have set out on.

Date Chicken

Through out my software development career I have been trying to understand how things work, actively pursuing the thought processes that people use to make their decisions. I spend time asking questions about the hows and the whys of what I am asked to do and produce. This action, over time, has produced a decent picture in my head of how software development should work and in many ways how it should never ever work. I have been able to understand the whys of certain actions and requests but the one 'action' for which I have yet to totally understand is the idea of software development "Date Chicken".

Date Chicken as I would describe it is the propensity of an organization to want to assign a date to a project without understanding or going through an understanding of the complexities around what has been requested to be developed. Date chicken comes about due to not having enough of the discovery executed such that any date that can be provided has to be provided with a long list of provisos and comments/risks about what in the world at large has to line up "Perfectly" in order to meet the date that is being published. So software development teams are asked to produce these "Plans" that indicate that we can meet the date IFF (If and only if) everything else in the world happens perfectly. What happens next, every other team produces the same sort of statement about their ability to meet the date and the fun begins. Management from each team begins to execute on the "PLAN" that has been produced, they organize teams, and instruct people on what needs to be done and they wait patiently for the first 'date' on the plan to come up.

Here the Chicken part of date chicken takes over. Basically the teams continue to plod along towards the date until that first date arises and a team has not produced the things that they 'said' they would produce and everyone else involved in the project gets to point out that 'their' dates will have to slip day for day with every day missed in the development team at the front of the train. Hence the 'Chicken' in date chicken.

Why is this so hard for me to understand you might ask - every software company of any size seems to do this...

Its difficult for me for the following reasons:

1) The date is a lie, perpetrated by someone else and then taken as gospel, which leaves me feeling like my management doesn't want me to tell the truth, ever. Ok - the date is not entirely a lie - because there is an OFF chance that the needed software shows up on "time" with high quality, but so far I have yet to see that happen when this game of date chicken takes hold.
2) Because the date is a lie everyone 'RUSHES' to turn out the thing that they were on the hook for before their date arrives. In turn this produces software of lessor quality, with production issues, that turns out to be hard to manage and maintain.
3) #2 causes our customer experience to essentially suck, driving the customers to complain or look for service else where, making my company look like the best of the worst.

I do understand that sometimes the date is important, there might be legal reasons or contractual reasons that a specific software development item must be completed by a specific date. However - when there is no specific pressure in this way, I would much rather have my management say "I want to give you the time to make the software 'RIGHT', and I am willing to wait to make that happen."

I don't think it appropriate for the waiting to be forever, but do think that a management statement that says something like "I would like to see something demonstrable in one quarters worth of time." or even, "I am ok to wait to get this right, but I would like it to launch this year" to be fine statements that reinforce the please build this right rather than fast. As we get closer to actually delivering the software requested it is possible to provide a VERY high confidence number on 'when' we are going to be complete. The further away from done however the less confidence in the 'plan' being the right 'plan' and the date, so much more a lie.

In the end - none of this is in my control, but I sincerely hope that it will be at some point if I either end up in a position to make changes, or have my own company making software. Date chicken reinforces all sorts of really terrible habits and I would rather not participate.