Thursday, November 20, 2014

A world of change

There are about a gazillion reasons we can come up with, all of which sound logical and justifiable, to avoid change. The #1 reason change gets avoided? People would prefer not to deal with a necessary change until they absolutely have to. As long as life is basically functioning, most people are happy to just not stir the pot. Everyone grows slightly complacent and they settle into a routine which they can wrap around themselves like a warm blanket. I am as guilty of this as the next person. I find ways to focus on work, focus on side projects, focus on anything other than those big blinky, claxon-y, signals that are trying to tell me to break from my routine and pay attention. The world is an interesting place - it is difficult to have the presence of mind and body to focus. If I am the only thing I have control over in the world however (which is true of everyone) then it is 100% up to me to break the trend, remove the blanket and attempt to change, hopefully for the better and hopefully in as much truth and honesty as I can muster.

Thank you world, forever grateful.

Thursday, October 30, 2014

Roles for Making a great (software dev) team

I wanted to take the time to try to document the roles I think make up a great development team. This is essentially that raw brain dump - would be very interested in engaging discussion and conversation around these. What do you think - how has it worked for you - would you refine these in any way?

The Constructive tinkerer

This role describes a person that spends time looking at the problems that are facing the team and looks for ways to solve them.  The solution can be already contained in the things the team does or may be something that team has not yet thought of or is currently using in the way of technology. This person spends the time to investigate, research, experiment with the different, better, worse ways of doing what needs to be done inside the team. This individual is often not directly involved in the shit triad*, meaning this role is often not doing product development work but is instead creating nice paved road surface for the folk who are part of the shit triad* to move faster and get more done in shorter periods of time. The work this person does tends to lay the ground work for a great deal of other work that the team does. Clearing the path and making things easier, more visible, more operational.

Additionally - this role spends time providing guidance and mentorship for the other members of the team. Sometimes the discussions being had are technical in nature, things like "How should I implement this new hot thing that product asked for..." and sometimes the questions/discussions are procedural and more team oriented... "I think that we need to make this type of error more visible and loud so that we can find a root cause and eliminate it." This role plays a part in providing a sounding board for a great number of activities in the team working to help the team understand the aspects of their software running in a production data center with real users, whom we want to delight - not make angry.

The key for this role - is that the tinkering with things, the new code, the clean up of old code, the discussions are all oriented in positive "We will use this ways". When the result of this roles tinkering isn't useful to anyone, the person playing this role doesn't get upset, nor defensive - they simple tuck that work away for some later time, or they drop the work all together. The constructive tinkerer is focused on what the team working with them needs. Decisions about the longevity of something are biased towards team needs and wants but balanced with what the larger organization needs from the team as well.

The curator (Those that care do)

This is an oft overlooked role for every software development team. This role may not even be a single team member but encompass how the entire team thinks about the code base that they work in every single day. The curator(s) are those that are deeply invested in the well being, well meaning, and intention of each and every part of the code base. Curating the code is important because as we all know, code that is no cared for tends to suffer from:

Bit Rot
Code that is left to sit eventually rots away. Of course it doesn't actually rot, but what does happen is developers lose touch with what the code was supposed to be doing. There is a lack of documentation, so there is no longer any institutional knowledge of the code and no way to gain it other than wading back into the code base to find out whats in there. Bugs stop getting fixed and people eventually would rather replace all the work than 'figure it out' all over again. This can be avoided when there is a curator around - because they are constantly looking at what can be made better - seeing where problems are and will get upset when something sits for too long without getting any touch/attention.

Different points of view
It is rare these days that a code base have a single developer. All the code that I work on is managed and maintained by either a team or multiple teams of people. The curator has the un-envious job of reconciling the differing ways that things in the codebase are accomplished. Ties are broken, discussions are solidified into a 'way' in which the code will represent a specific thing. The code gets cleaner as a result. Differing points of view have to be recognized and addressed by a curator or you will easily be able to tell which sections of a code base were written by whom based on taste and patterns of development.

New developers
New developers to the team are simply new and have to be shown a way. This demonstration is made easier when they can see that the code base looks a certain way and does things in certain ways. This is made better when they are also able to consult a curation individual for a team. Work can be made even cleaner when new developers know that there is a pattern that exists that they can follow and that the team is open to discussion for new ways and new ideas to change existing patterns.

New shiny thing
A curator helps to combat running off and implementing the next hottest, best, new, shiniest thing in terms of development. Sometimes new patterns and new ways of doing things are beneficial or needed. A curator helps to understand how that can be phased into the code base and how existing patterns can be touched and adjusted as a result. The curator helps to understand how much time and effort is involved in moving the WHOLE code base into a new direction or if the new thing should only be used in certain locations under certain conditions.

Those supporting the shit trinity - Knows Shit, Gets Shit Done, Gives a Shit

So every team needs some number of people that have the ability to get things done... and get those things done well. Those that support the shit trinity make up a fair majority of a team and ideally everyone on the team would fall into this category to some degree. This means that everyone on the team has the ability to play a roll in accomplishing the goals that the team has set out to accomplish. Sometimes the goals are development oriented and related to software design and coding, other times the teams goals are about the process that they use to get their work organized and ready to work on.

The helper/garbage man

You always have to be willing to take out the trash. This is another role that the entire team should be adjusted towards. The garbage in the case of a development team is the grunt work of fixing bugs, re-organizing code when it makes sense, basically anything that would be considered the bottom of the barrel in terms of development effort. The tasks involved are generally just tasks that need to get done in advance of getting to the nitty gritty of some new development that has been planned. Every team needs at least one, and hopefully the entire team would be willing to do this work because they know it is necessary. Having the team be adjusted to taking out the trash means that no one on the team feels like it is demeaning to have some amount of their work be involved with taking out the trash in the code base.

The adage that I think goes with this is that a good manager, exec, CEO doesn't believe that certain tasks are somehow beneath them, they see that it needs to get done and that currently no one is doing it so they go in pick up that slack and get it accomplished and then later look for ways to not miss getting that bit of work done in the future. For a team - this mentality in at least one person has to exist both because it is necessary to pick up what is left behind but also because it encourages others to also pick up what is left behind. In essence having an individual on the team that "takes out the trash" encourages people to not throw stones at already broken windows but instead to fix those windows and move onto something else. 

The thinker (although they all have to be this way to some degree)

Software design, architecture and over all capability of the code to do a given task takes thought. It is not something that flows from the fingertips of generally smart people perfect the first time every time. Software development is a thought process, a process sometimes of trial (doing what you think will work) and error (seeing that what you did doesn't work) in order to get to the point where something is produced that meets the requested/stated needs of the larger organization. Sometimes the thinking on the team is encapsulated in one individual but I find more often that while one individual seems to do a great deal of the thinking/design that everyone seems to rally around that "nucleus" adding their own identity and thought process to the original. Making the thinker role more os an instigator role most of the time, using the power of the group to find all the holes in a design, to talk about all the corner cases in what we are doing and to aid in preventing the entire team from backing themselves into a development corner that they can't get out of. The thinker/instigator provides that stable guidance for architecture so that it evolves and morphs rather than needing a revolutionary re-write to get the team out of a bind.

Verifiers  (although they all have to be this way to some degree)

Everyone should be interested in verifying there work.  Sometimes this is a single individual that acts as the last safety valve for a team, making the last checks, T's crossed and I's dotted before the result of a development cycle, no matter how large, is doing what is expected when the development got started. In our team this has been encapsulated into a final check before deployment of the out put of the API.  When we are getting ready to do the deployment for a given day, the code that has been worked on is deployed into a staging environment and then a script is run to look at the syntactic differences in the output of the endpoints in the api space. Differences between Staging and Production are noted, discussed and either acted upon because they were unexpected or dismissed as expected due to the code being deployed. Taking this action in the form of tests that each developer wrote during their development in the early part of a cycle and doing this final check at the end causes us all to 'verify' our work. Similar to the thinker above - when the team sees that this is expected they tend to start executing that way on their own with little to no nudging required.

The reporter - Someone has to tell the world

In the end the WAY in which you work is important, but just as important if you want others to work in the same way is that you go and tell them what you are doing and how you are doing it. Enter the reporter to the team. This is someone who likes to taught what the team does and succeeds doing. In a shout from the roof top sort of way this individual helps the success of the team become more common knowledge and more well understood by other teams. This helps to spread ideas and processes and allows others to start to ask questions as they investigate their own teams and how they operate to accomplish the goals they have set out on.

Friday, May 30, 2014

Please stop RESTfully abusing HTTP

So I have noticed this rather disturbing trend while building, deploying and consuming REST service(s) and it goes something like this:

"HEY all - Lets build a REST service to meet our new product need."

Every one in the room nods and says - "That is an excellent idea." They have heard the buzz word and they understand HTTP fairly well so they feel like they have an excellent bead on REST. People may even do a little light reading on the internet around the topic. They will understand that they can use HTTP to take actions on resources. There is however an odd understanding that creeps into the discussion and that is how to go about representing both good and bad states for responses to the caller.

The most annoying representation I have run across thus far is:

POST ${JSON} … 200 OK {status: error, reason: SYNTAX}

Caller posts to the endpoint over HTTP, a json object. They get back an HTTP status 200 (indicating everything happened ok) with a json body in the response. That json body has an ERROR IN IT! This relegates HTTP to nothing more than a transport protocol. So you look at the above and you say - "...well, whats the big deal with that - looks ok to me."

HTTP 2XX Status Communicates Success to Clients

First and foremost amongst the problems with this pattern is that HTTP clients of all varieties expect that any response with a 2XX class status code can be considered success. This is in the HTTP spec and is what a large number of clients in a variety of languages understand. When I am attempting to write a client against this service there is no way for me, just looking at that HTTP Status code, to understand if my request actually did what I wanted or errored out in some way. Doing this forces the receiver to need to inspect the body of the HTTP response, breaking a programming tenant I have personally had for a while, FAIL FAST. Because I can no longer rely on the nice fast HTTP response header and I am forced to actually read the HTTP body - I have to read the ENTIRE body to see if there is anything in there that would tell me what went wrong with my call to this particular service. Which brings up the next problem.

What format is the Body in: Parsing??!?

So with a client now not being able to fail fast and being forced to read the entire response body what other problems are there you ask? Well how about, the format of the body? Is the body json? is it XML? is it some binary format? HTTP has a way to tell you what was sent with Accept/Vary header combinations, but in all those cases I have to essentially parse out the response in some way. What if that parsing FAILS? What specific format does the response have to be in in order for me to correctly interpret the type of failure that has occurred? Does a failed parse attempt lead to a generic error? a specific error? What do you tell a calling client when its not something the client can actually do anything about? Suffice to say this is a mess. It can be cleaned up a bit - you and your clients can specify what format these things should be in and where you can find specific error messages and other information, however HTTP spec already provides all this information for you in a standard way if you were using it the way it was intended rather than treating it as a simple transport protocol only.

I hear you screaming

I understand that it isn't always a clean mapping between good/bad things that the APIs do and an HTTP status code that currently exists, but you should at least try to find simple ones that make sense and then look for consistent and reasonable HTTP extensions that give you specificity that you need. Take for example returning a 403 Forbidden as an access exception - and then returning a header in the response with a subcode indicating WHY that specific request was 403'ed. Example, HTTP status 403, X-SUBCODE-ERROR: 101 X-SUBCODE-REASON: User over allowed limit at this time.

Please do not RESTfully abuse HTTP as a mere transport, use what you know, fail as soon as you reasonably know you can and for god sake, please don't make me have to parse a response body in order to know what went wrong.