Friday, May 30, 2014

Please stop RESTfully abusing HTTP

So I have noticed this rather disturbing trend while building, deploying and consuming REST service(s) and it goes something like this:

"HEY all - Lets build a REST service to meet our new product need."

Every one in the room nods and says - "That is an excellent idea." They have heard the buzz word and they understand HTTP fairly well so they feel like they have an excellent bead on REST. People may even do a little light reading on the internet around the topic. They will understand that they can use HTTP to take actions on resources. There is however an odd understanding that creeps into the discussion and that is how to go about representing both good and bad states for responses to the caller.

The most annoying representation I have run across thus far is:

POST ${JSON} … 200 OK {status: error, reason: SYNTAX}

Caller posts to the endpoint over HTTP, a json object. They get back an HTTP status 200 (indicating everything happened ok) with a json body in the response. That json body has an ERROR IN IT! This relegates HTTP to nothing more than a transport protocol. So you look at the above and you say - "...well, whats the big deal with that - looks ok to me."

HTTP 2XX Status Communicates Success to Clients

First and foremost amongst the problems with this pattern is that HTTP clients of all varieties expect that any response with a 2XX class status code can be considered success. This is in the HTTP spec and is what a large number of clients in a variety of languages understand. When I am attempting to write a client against this service there is no way for me, just looking at that HTTP Status code, to understand if my request actually did what I wanted or errored out in some way. Doing this forces the receiver to need to inspect the body of the HTTP response, breaking a programming tenant I have personally had for a while, FAIL FAST. Because I can no longer rely on the nice fast HTTP response header and I am forced to actually read the HTTP body - I have to read the ENTIRE body to see if there is anything in there that would tell me what went wrong with my call to this particular service. Which brings up the next problem.

What format is the Body in: Parsing??!?

So with a client now not being able to fail fast and being forced to read the entire response body what other problems are there you ask? Well how about, the format of the body? Is the body json? is it XML? is it some binary format? HTTP has a way to tell you what was sent with Accept/Vary header combinations, but in all those cases I have to essentially parse out the response in some way. What if that parsing FAILS? What specific format does the response have to be in in order for me to correctly interpret the type of failure that has occurred? Does a failed parse attempt lead to a generic error? a specific error? What do you tell a calling client when its not something the client can actually do anything about? Suffice to say this is a mess. It can be cleaned up a bit - you and your clients can specify what format these things should be in and where you can find specific error messages and other information, however HTTP spec already provides all this information for you in a standard way if you were using it the way it was intended rather than treating it as a simple transport protocol only.

I hear you screaming

I understand that it isn't always a clean mapping between good/bad things that the APIs do and an HTTP status code that currently exists, but you should at least try to find simple ones that make sense and then look for consistent and reasonable HTTP extensions that give you specificity that you need. Take for example returning a 403 Forbidden as an access exception - and then returning a header in the response with a subcode indicating WHY that specific request was 403'ed. Example, HTTP status 403, X-SUBCODE-ERROR: 101 X-SUBCODE-REASON: User over allowed limit at this time.

Please do not RESTfully abuse HTTP as a mere transport, use what you know, fail as soon as you reasonably know you can and for god sake, please don't make me have to parse a response body in order to know what went wrong.

Friday, May 23, 2014

Transparency - These are not the facts you're looking for...

Oh recurring words, why do you haunt me so.  This time the recurring word is transparency. I will focus this post to focus on the transparency in my work a day life as I believe my work and interactions going on at work are what triggered the recurrence of the word "transparency" and allowed me to see how it surfaces in other areas of my life.

Transparency is an interesting word when referring to a 'person' - thinking about this term evoke images of windows, cellophane, things that you are able to see through.  When applied to a person I can here cliche sayings like "...Oh That guy - Yeah, he's an open book... as transparent as they come." Whats interesting to me is that a state of being or feeling transparent also falls along a broad spectrum of feeling. Feeling transparent evokes images of feeling sorry for oneself, of being the 'Wall flower' in the room. No one sees you, nor understands who or what you are. In the work setting though neither of these two types of definition is what people mean when they talk about transparency. In the work setting of Big Co. - transparency seems to mean providing a window into the 'items' being done by a person or group in a digestible way for the original requester of said information.

OK - Pause that statement is really tough to parse. Transparency (at least in Big Co, even if I don't happen to agree with the definition) in the work place is often defined as:

Is a person (not me) able to understand without asking me what was getting done by me or my team, what was planned to get done and how far along that plan had me or we gotten. In a loose sense transparency in Big Co. is quite synonymous with a reported status from me or the team I am working with. So lets break down the various transparency layers that might exist at Big Co.

Transparency with co-workers
Transparency at this level should be easy, these are the people that you work with after all. Wait you say, I don't like some of the people I work with... YIKES! Transparency at this level, I believe, requires a vulnerability on a personal level which will be hard if you don't happen to like the people you work with. You need to be able share life stories, kick back and enjoy some time together in order for those you work with to feel like:

1) They know who you are
2) They know how you will react in certain situations
3) That you will provide them with information you have

Transparency with your co-workers is about trusting that those working with you will understand and take care with what you share with them. Transparency doesn't mean they need to know every little detail of your life in and out of the office, but showing them you are human and have a number of the same life situations and issues that they do allows you to connect, and for them to FEEL like you are transparent to them.

Transparency with (direct) subordinates
Transparency with people that report to you is a different matter, but not entirely. Trust is still at the core of how people feel about reporting to you. Being transparent with people that work for you is about providing them content and context for the work they are doing. Transparency for a manager on the front lines is about connecting the workers to the work in a way that they understand and can engage with. This is a fine line, because it is possible to give them too much or to let your own opinions color how you deliver the information. However as long as people don't feel like you're hiding information from them, you can easily be seen as being transparent.

Transparency with (in-direct) subordinates
Transparency here is a great deal like transparency with direct reports, the same things apply. Still trust is a must, relationship growing is harder but just as important. These people my not be on the same floor or even int he same building - but to be transparent with them is to provide the 'windows' into what and how you are doing things that allows them to feel like they understand. Here - you have to invite them in and show them the house but you don't have to tell them how the house is wired for them to feel comfortable. Outside groups will make certain assumptions about how the mechanizations in your team operate but you may not have to correct them even if some of their assumptions are off or flat out wrong. Just make the information that they are interested in easy to get, and read and let them see it. They will feel you are being transparent.

Transparency with managers
So transparency from the lower levels to the higher levels in the organization is likely a much larger sticking point. In Big Co. managers and managers of managers have a tendency to want to be 'TOLD" rather than have to go "DISCOVER" how the work works or how the work is doing. This drives directly at my commentary above about transparency from bottom towards the top often seems to take the shape of a status report or other document. This status can not possibly communicate all the things so it has to be boiled down to the 'most important' but manager will often have missed the part where they provide a priority and a context to what is 'most important' meaning it is left as a guessing game. This guessing game, a mind reading game, is difficult to get around. Being transparent here is about providing the information in the best way you can, allowing the teams you work with to 'see' that you are doing that so that if there are errors in communication they can be directly and reasonably corrected. But in the end I think managers would be better suited:

Going to the Gemba 
Managers should be interested in seeing the work for themselves now and again. They should be visiting with the teams doing work on their behalf, asking them if they have any questions, if they feel that they have what they need in order to do their work. This makes this higher level of transparency super easy - because the manager can see what is happening and ask questions about it. Conversations and details can break out in a real time way rather than disconnected by a document that is out of date the instant it is written and emailed.

In the end - transparency has so many different facets to it. Transparency shouldn't be about status - but in Big Co. it is, so be it. It is important then to know how to have and use transparency in the work you do. It will help you move in the organization when you would like to because people will know what they are getting rather than feeling like you're a surprise waiting to happen.