Monday, November 24, 2014

Generativity, an open door policy

So, I have had some time to sit and consider what it is about software projects in my work a day world that make for successes and what things make for interesting types of failure. Here is where my thinking landed me:

1) Being open, and inviting is a benefit not to be undersold

It sounds so tantalizingly simple when you type it out like that, so much so that you might read that line above as "...If you build it, they will come..." but that isn't quite what that line means.

When teams develop new software, the software being developed is sometimes about a new feature or a new thing that BigCo. wants to have as a product and sometimes is to scratch an itch that the team has. In either case, this new software is likely to be able to assist a whole bunch of people which same or similar needs (it is unusual for software to actually be built 100% single purpose). Because most software can actually serve many 'close knit' needs, once you or your team has taken the time to build, you should also take the time to advertise and allow other teams to make use of what you have built. Being open, means no matter what you develop that you essentially open the door to others to build, manage or deploy with you.  Let other groups make use of the things you built, with as few a number of restrictions, provisos, caveats as humanly possible.

2) Protect the system, but keep the number of rules as low as you can

This is somewhat self explanatory - try to only put in place those rules that help you to make sure that the system is up, available, and working correctly.

3) Trust people to do whats right

When you open the door - you are opening your system up to whatever may come. To that point, you have to be able to trust that peoples self interest will also align with you and yours to some degree. Try to keep a light touch and trust that people will not go out of their way to screw you - or the system you built - that they are now using. If those who have been invited in through the open door do take advantage, know in advance how you would like to react to those coming to seek your new hot thing. Describe your "open door" so that people know what they are signing up for in advance.

4) Don't fall into the trap of N.I.H.

Part of doing work this way, with this mind set, is preventing yourself from going off into a corner and building it yourself because:

   * You believe that building it yourself is the only way to control your own destiny
   * Your team can't possibly be successful because your success is dependent on another team

It will always be simpler to modify something that is already built than it will be to go from scratch no matter how good your developers are. Work already done can trump work to be done in a great number of occasions.

5) Command and control will squash peoples desire to use what you build

Don't try to control the software and how its used too must - exercise just enough control to manage and maintain the quality of the system. I have seen plenty of people that believe that becoming the manager, exec, vp, director or what have you is all about being able to do it JUST the way they want. Nothing really could be further from the truth. The farther up the chain you go, the less desire to control you should have and the more trust you have to have that things will head in the right direction. If you attempt to stringently control things, no one will want to come be a part of the thing that was built and like holding a fist full of sand, the harder you squeeze the more sand you will lose, no way to hold it all.

Be open, be inviting, be willing to negotiate and understand how people will want to use your software. Be generative - providing space for people to get the most amount of benefit out of the building blocks you have already put together. It will eventually help the bottom line because everything being built will start to move faster, eventually.

Thursday, November 20, 2014

A world of change

There are about a gazillion reasons we can come up with, all of which sound logical and justifiable, to avoid change. The #1 reason change gets avoided? People would prefer not to deal with a necessary change until they absolutely have to. As long as life is basically functioning, most people are happy to just not stir the pot. Everyone grows slightly complacent and they settle into a routine which they can wrap around themselves like a warm blanket. I am as guilty of this as the next person. I find ways to focus on work, focus on side projects, focus on anything other than those big blinky, claxon-y, signals that are trying to tell me to break from my routine and pay attention. The world is an interesting place - it is difficult to have the presence of mind and body to focus. If I am the only thing I have control over in the world however (which is true of everyone) then it is 100% up to me to break the trend, remove the blanket and attempt to change, hopefully for the better and hopefully in as much truth and honesty as I can muster.

Thank you world, forever grateful.

Thursday, October 30, 2014

Roles for Making a great (software dev) team

I wanted to take the time to try to document the roles I think make up a great development team. This is essentially that raw brain dump - would be very interested in engaging discussion and conversation around these. What do you think - how has it worked for you - would you refine these in any way?

The Constructive tinkerer

This role describes a person that spends time looking at the problems that are facing the team and looks for ways to solve them.  The solution can be already contained in the things the team does or may be something that team has not yet thought of or is currently using in the way of technology. This person spends the time to investigate, research, experiment with the different, better, worse ways of doing what needs to be done inside the team. This individual is often not directly involved in the shit triad*, meaning this role is often not doing product development work but is instead creating nice paved road surface for the folk who are part of the shit triad* to move faster and get more done in shorter periods of time. The work this person does tends to lay the ground work for a great deal of other work that the team does. Clearing the path and making things easier, more visible, more operational.

Additionally - this role spends time providing guidance and mentorship for the other members of the team. Sometimes the discussions being had are technical in nature, things like "How should I implement this new hot thing that product asked for..." and sometimes the questions/discussions are procedural and more team oriented... "I think that we need to make this type of error more visible and loud so that we can find a root cause and eliminate it." This role plays a part in providing a sounding board for a great number of activities in the team working to help the team understand the aspects of their software running in a production data center with real users, whom we want to delight - not make angry.

The key for this role - is that the tinkering with things, the new code, the clean up of old code, the discussions are all oriented in positive "We will use this ways". When the result of this roles tinkering isn't useful to anyone, the person playing this role doesn't get upset, nor defensive - they simple tuck that work away for some later time, or they drop the work all together. The constructive tinkerer is focused on what the team working with them needs. Decisions about the longevity of something are biased towards team needs and wants but balanced with what the larger organization needs from the team as well.

The curator (Those that care do)

This is an oft overlooked role for every software development team. This role may not even be a single team member but encompass how the entire team thinks about the code base that they work in every single day. The curator(s) are those that are deeply invested in the well being, well meaning, and intention of each and every part of the code base. Curating the code is important because as we all know, code that is no cared for tends to suffer from:

Bit Rot
Code that is left to sit eventually rots away. Of course it doesn't actually rot, but what does happen is developers lose touch with what the code was supposed to be doing. There is a lack of documentation, so there is no longer any institutional knowledge of the code and no way to gain it other than wading back into the code base to find out whats in there. Bugs stop getting fixed and people eventually would rather replace all the work than 'figure it out' all over again. This can be avoided when there is a curator around - because they are constantly looking at what can be made better - seeing where problems are and will get upset when something sits for too long without getting any touch/attention.

Different points of view
It is rare these days that a code base have a single developer. All the code that I work on is managed and maintained by either a team or multiple teams of people. The curator has the un-envious job of reconciling the differing ways that things in the codebase are accomplished. Ties are broken, discussions are solidified into a 'way' in which the code will represent a specific thing. The code gets cleaner as a result. Differing points of view have to be recognized and addressed by a curator or you will easily be able to tell which sections of a code base were written by whom based on taste and patterns of development.

New developers
New developers to the team are simply new and have to be shown a way. This demonstration is made easier when they can see that the code base looks a certain way and does things in certain ways. This is made better when they are also able to consult a curation individual for a team. Work can be made even cleaner when new developers know that there is a pattern that exists that they can follow and that the team is open to discussion for new ways and new ideas to change existing patterns.

New shiny thing
A curator helps to combat running off and implementing the next hottest, best, new, shiniest thing in terms of development. Sometimes new patterns and new ways of doing things are beneficial or needed. A curator helps to understand how that can be phased into the code base and how existing patterns can be touched and adjusted as a result. The curator helps to understand how much time and effort is involved in moving the WHOLE code base into a new direction or if the new thing should only be used in certain locations under certain conditions.

Those supporting the shit trinity - Knows Shit, Gets Shit Done, Gives a Shit

So every team needs some number of people that have the ability to get things done... and get those things done well. Those that support the shit trinity make up a fair majority of a team and ideally everyone on the team would fall into this category to some degree. This means that everyone on the team has the ability to play a roll in accomplishing the goals that the team has set out to accomplish. Sometimes the goals are development oriented and related to software design and coding, other times the teams goals are about the process that they use to get their work organized and ready to work on.

The helper/garbage man

You always have to be willing to take out the trash. This is another role that the entire team should be adjusted towards. The garbage in the case of a development team is the grunt work of fixing bugs, re-organizing code when it makes sense, basically anything that would be considered the bottom of the barrel in terms of development effort. The tasks involved are generally just tasks that need to get done in advance of getting to the nitty gritty of some new development that has been planned. Every team needs at least one, and hopefully the entire team would be willing to do this work because they know it is necessary. Having the team be adjusted to taking out the trash means that no one on the team feels like it is demeaning to have some amount of their work be involved with taking out the trash in the code base.

The adage that I think goes with this is that a good manager, exec, CEO doesn't believe that certain tasks are somehow beneath them, they see that it needs to get done and that currently no one is doing it so they go in pick up that slack and get it accomplished and then later look for ways to not miss getting that bit of work done in the future. For a team - this mentality in at least one person has to exist both because it is necessary to pick up what is left behind but also because it encourages others to also pick up what is left behind. In essence having an individual on the team that "takes out the trash" encourages people to not throw stones at already broken windows but instead to fix those windows and move onto something else. 

The thinker (although they all have to be this way to some degree)

Software design, architecture and over all capability of the code to do a given task takes thought. It is not something that flows from the fingertips of generally smart people perfect the first time every time. Software development is a thought process, a process sometimes of trial (doing what you think will work) and error (seeing that what you did doesn't work) in order to get to the point where something is produced that meets the requested/stated needs of the larger organization. Sometimes the thinking on the team is encapsulated in one individual but I find more often that while one individual seems to do a great deal of the thinking/design that everyone seems to rally around that "nucleus" adding their own identity and thought process to the original. Making the thinker role more of an instigator role most of the time, using the power of the group to find all the holes in a design, to talk about all the corner cases in what we are doing and to aid in preventing the entire team from backing themselves into a development corner that they can't get out of. The thinker/instigator provides that stable guidance for architecture so that it evolves and morphs rather than needing a revolutionary re-write to get the team out of a bind.

Verifiers  (although they all have to be this way to some degree)

Everyone should be interested in verifying their work.  Sometimes this is a single individual that acts as the last safety valve for a team, making the last checks, T's crossed and I's dotted before the result of a development cycle, no matter how large, is verified to be doing what it was expected to when the development got started. In our team this has been encapsulated into a final check before deployment of the out put of the API.  When we are getting ready to do the deployment for a given day, the code that has been worked on is deployed into a staging environment and then a script is run to look at the syntactic differences in the output of the endpoints in the api space. Differences between Staging and Production are noted, discussed and either acted upon because they were unexpected or dismissed as expected due to the code being deployed. Taking this action in the form of tests that each developer wrote during their development in the early part of a cycle and doing this final check at the end causes us all to 'verify' our work. Similar to the thinker above - when the team sees that this is expected they tend to start executing that way on their own with little to no nudging required.

The reporter - Someone has to tell the world

In the end the WAY in which you work is important, but just as important if you want others to work in the same way is that you go and tell them what you are doing and how you are doing it. Enter the reporter of the team. This is someone who likes to taut what the team does and succeeds in accomplishing. In a shout from the roof top sort of way this individual helps the success of the team become more common knowledge and more well understood by other teams. This helps to spread ideas and processes and allows others to start to ask questions as they investigate their own teams and how they operate to accomplish the goals they have set out on.

Friday, May 30, 2014

Please stop RESTfully abusing HTTP

So I have noticed this rather disturbing trend while building, deploying and consuming REST service(s) and it goes something like this:

"HEY all - Lets build a REST service to meet our new product need."

Every one in the room nods and says - "That is an excellent idea." They have heard the buzz word and they understand HTTP fairly well so they feel like they have an excellent bead on REST. People may even do a little light reading on the internet around the topic. They will understand that they can use HTTP to take actions on resources. There is however an odd understanding that creeps into the discussion and that is how to go about representing both good and bad states for responses to the caller.

The most annoying representation I have run across thus far is:

POST ${JSON} … 200 OK {status: error, reason: SYNTAX}

Caller posts to the endpoint over HTTP, a json object. They get back an HTTP status 200 (indicating everything happened ok) with a json body in the response. That json body has an ERROR IN IT! This relegates HTTP to nothing more than a transport protocol. So you look at the above and you say - "...well, whats the big deal with that - looks ok to me."

HTTP 2XX Status Communicates Success to Clients

First and foremost amongst the problems with this pattern is that HTTP clients of all varieties expect that any response with a 2XX class status code can be considered success. This is in the HTTP spec and is what a large number of clients in a variety of languages understand. When I am attempting to write a client against this service there is no way for me, just looking at that HTTP Status code, to understand if my request actually did what I wanted or errored out in some way. Doing this forces the receiver to need to inspect the body of the HTTP response, breaking a programming tenant I have personally had for a while, FAIL FAST. Because I can no longer rely on the nice fast HTTP response header and I am forced to actually read the HTTP body - I have to read the ENTIRE body to see if there is anything in there that would tell me what went wrong with my call to this particular service. Which brings up the next problem.

What format is the Body in: Parsing??!?

So with a client now not being able to fail fast and being forced to read the entire response body what other problems are there you ask? Well how about, the format of the body? Is the body json? is it XML? is it some binary format? HTTP has a way to tell you what was sent with Accept/Vary header combinations, but in all those cases I have to essentially parse out the response in some way. What if that parsing FAILS? What specific format does the response have to be in in order for me to correctly interpret the type of failure that has occurred? Does a failed parse attempt lead to a generic error? a specific error? What do you tell a calling client when its not something the client can actually do anything about? Suffice to say this is a mess. It can be cleaned up a bit - you and your clients can specify what format these things should be in and where you can find specific error messages and other information, however HTTP spec already provides all this information for you in a standard way if you were using it the way it was intended rather than treating it as a simple transport protocol only.

I hear you screaming

I understand that it isn't always a clean mapping between good/bad things that the APIs do and an HTTP status code that currently exists, but you should at least try to find simple ones that make sense and then look for consistent and reasonable HTTP extensions that give you specificity that you need. Take for example returning a 403 Forbidden as an access exception - and then returning a header in the response with a subcode indicating WHY that specific request was 403'ed. Example, HTTP status 403, X-SUBCODE-ERROR: 101 X-SUBCODE-REASON: User over allowed limit at this time.

Please do not RESTfully abuse HTTP as a mere transport, use what you know, fail as soon as you reasonably know you can and for god sake, please don't make me have to parse a response body in order to know what went wrong.

Friday, May 23, 2014

Transparency - These are not the facts you're looking for...

Oh recurring words, why do you haunt me so.  This time the recurring word is transparency. I will focus this post to focus on the transparency in my work a day life as I believe my work and interactions going on at work are what triggered the recurrence of the word "transparency" and allowed me to see how it surfaces in other areas of my life.

Transparency is an interesting word when referring to a 'person' - thinking about this term evoke images of windows, cellophane, things that you are able to see through.  When applied to a person I can here cliche sayings like "...Oh That guy - Yeah, he's an open book... as transparent as they come." Whats interesting to me is that a state of being or feeling transparent also falls along a broad spectrum of feeling. Feeling transparent evokes images of feeling sorry for oneself, of being the 'Wall flower' in the room. No one sees you, nor understands who or what you are. In the work setting though neither of these two types of definition is what people mean when they talk about transparency. In the work setting of Big Co. - transparency seems to mean providing a window into the 'items' being done by a person or group in a digestible way for the original requester of said information.

OK - Pause that statement is really tough to parse. Transparency (at least in Big Co, even if I don't happen to agree with the definition) in the work place is often defined as:

Is a person (not me) able to understand without asking me what was getting done by me or my team, what was planned to get done and how far along that plan had me or we gotten. In a loose sense transparency in Big Co. is quite synonymous with a reported status from me or the team I am working with. So lets break down the various transparency layers that might exist at Big Co.

Transparency with co-workers
Transparency at this level should be easy, these are the people that you work with after all. Wait you say, I don't like some of the people I work with... YIKES! Transparency at this level, I believe, requires a vulnerability on a personal level which will be hard if you don't happen to like the people you work with. You need to be able share life stories, kick back and enjoy some time together in order for those you work with to feel like:

1) They know who you are
2) They know how you will react in certain situations
3) That you will provide them with information you have

Transparency with your co-workers is about trusting that those working with you will understand and take care with what you share with them. Transparency doesn't mean they need to know every little detail of your life in and out of the office, but showing them you are human and have a number of the same life situations and issues that they do allows you to connect, and for them to FEEL like you are transparent to them.

Transparency with (direct) subordinates
Transparency with people that report to you is a different matter, but not entirely. Trust is still at the core of how people feel about reporting to you. Being transparent with people that work for you is about providing them content and context for the work they are doing. Transparency for a manager on the front lines is about connecting the workers to the work in a way that they understand and can engage with. This is a fine line, because it is possible to give them too much or to let your own opinions color how you deliver the information. However as long as people don't feel like you're hiding information from them, you can easily be seen as being transparent.

Transparency with (in-direct) subordinates
Transparency here is a great deal like transparency with direct reports, the same things apply. Still trust is a must, relationship growing is harder but just as important. These people my not be on the same floor or even int he same building - but to be transparent with them is to provide the 'windows' into what and how you are doing things that allows them to feel like they understand. Here - you have to invite them in and show them the house but you don't have to tell them how the house is wired for them to feel comfortable. Outside groups will make certain assumptions about how the mechanizations in your team operate but you may not have to correct them even if some of their assumptions are off or flat out wrong. Just make the information that they are interested in easy to get, and read and let them see it. They will feel you are being transparent.

Transparency with managers
So transparency from the lower levels to the higher levels in the organization is likely a much larger sticking point. In Big Co. managers and managers of managers have a tendency to want to be 'TOLD" rather than have to go "DISCOVER" how the work works or how the work is doing. This drives directly at my commentary above about transparency from bottom towards the top often seems to take the shape of a status report or other document. This status can not possibly communicate all the things so it has to be boiled down to the 'most important' but manager will often have missed the part where they provide a priority and a context to what is 'most important' meaning it is left as a guessing game. This guessing game, a mind reading game, is difficult to get around. Being transparent here is about providing the information in the best way you can, allowing the teams you work with to 'see' that you are doing that so that if there are errors in communication they can be directly and reasonably corrected. But in the end I think managers would be better suited:

Going to the Gemba 
Managers should be interested in seeing the work for themselves now and again. They should be visiting with the teams doing work on their behalf, asking them if they have any questions, if they feel that they have what they need in order to do their work. This makes this higher level of transparency super easy - because the manager can see what is happening and ask questions about it. Conversations and details can break out in a real time way rather than disconnected by a document that is out of date the instant it is written and emailed.

In the end - transparency has so many different facets to it. Transparency shouldn't be about status - but in Big Co. it is, so be it. It is important then to know how to have and use transparency in the work you do. It will help you move in the organization when you would like to because people will know what they are getting rather than feeling like you're a surprise waiting to happen.