Something that I have been working on with some people that I work with:
Benchmarking the HttpClient Caching Module
It was my first foray into submitting to an opensource project. I have had a lot of fun working with the team and with the Apache moderators in getting this into the 4.1 version of the HttpClient, which is in beta/RC currently. The caching client supports HTTP 1.1 response caching.
The beta can be found here: HttpClient Dev Site
If you use HttpClient - I encourage you to down load this new version and take a peek at the caching client. We would enjoy getting feedback on the usefulness of the tool and how it can be improved.
Thursday, December 2, 2010
Wednesday, November 24, 2010
Volkswagon - An unavoidable rant
Long Time Owner
Let me first lay the groundwork for this post. I have been a very long time VW owner and enthusiast, from as far back as when I was thinking about my getting my own drivers license. Before that my family owned VWs as well, my father had a brown gasoline rabbit (1972 I think) and my mother owned a silver diesel rabbit (a 1979 I believe). I owned a number of odd cars before also getting my first VW, which was a 1997 Passat 1.9 TDI Sedan. In between my other cars and finally getting the 1997 TDI, I purchased a diesel rabbit for my wife which eventually was replaced by a 1994 Jetta, which was then replaced with a white roll top Cabrio. As the Cabrio declined, her and I switched cars for a little, while my wife bought her first brand new car purchasing a new green metallic Cabrio with a tan convertible top.
To say that I like my VWs is a bit of understatement. Now to be sure, I am NOT as hard-core as some, but I have been very loyal to the VW brand. The cars are generally fun to drive, easy to repair and maintain for the average gear head and at least for all the VWs I owned relatively cheap to own and maintain, that is until my purchase of a 2003 Passat gasoline 1.8T Wagon.
Track Record
In fairness before I get into the rant proper, all the VWs I have had have had some trouble with specific items. The diesel rabbit had the 'rabbit leak' over the fuse panel giving it all sorts of interesting issues including a propensity to short and run the battery down while it was 'sitting' for multi-day periods. The same problem was prevalent in the white Cabrio that my wife had for a while. Despite those problems the cars continued to operate and run and the fixes for any 'major' problems either of those cars ever had averaged a MAX of $500. We had a belt wear out on the 1994 Jetta, never had any trouble with the green Cabrio right up to trading it in on the 2003 Passat. The 1997 TDI that I purchased had relay109 fail, has gone through a number of window regulators and other than normal maintenance has had few if any real problems. My TDI currently has 215,000 miles on it and recently got the infusion of the largest amount of TLC that it has ever gotten when I spent $3200+ to do the following:
The reason I will NEVER own another VW Automobile
The 2003 Passat Wagon 1.8T is 100% the reason that I will never own another VW in my life. This car has had so many problems that it makes me slightly sick:
Post after Post after Post after Post after Post of people with the same oil sludge problem and VWNoA essentially turning a blind eye to the problem. The engine should have just been recalled en-mass because it was prone to this issue.
No car I have EVER OWNED has had this level of problem with oil sludging. Not my TDI, not any of the rabbits, cabrios, jettas that I have owned have ever had a problem like this, even with some serious abuse of the change intervals - NOT A SINGLE SLUDGE PROBLEM. Change with conventional oil, change with synthetic oil, change with some mix of both in the same engine... not a single problem. The 1.8T seems to MAKE sludge due to its design. It would have been interesting if VW had made some improvements/revamps to some engine parts to attempt to address this issue. Instead VW redesigned the whole thing in a beefier 2.0T engine and did away with the 1.8T engine all together.
Quality Counts
I have never felt so abused/betrayed by a car in my life. Every time I turned around there was another problem leaving my wife stranded in the middle of nowhere. If it wasn't the oil sludge costing me money it was the fuel pump (eventually recalled) or the ignition coils (also recalled) or water in the cabin from the moon roof drains being clogged. This by far has been the worst VW I have ever owned and despite a long history of VW ownership - this car has caused both my wife and myself to turn away in disgust from VW in general. VW's inability to take responsibility for poor design and parts choices is inexcusable in my mind and I cannot take the chance of getting another car like this one. Sorry VW - with this latest round of engine oil sludge in my 1.8T (despite religious, possibly even anal retentive oil changes) you have just lost a long time customer (from childhood till now) of your cars.
Let me first lay the groundwork for this post. I have been a very long time VW owner and enthusiast, from as far back as when I was thinking about my getting my own drivers license. Before that my family owned VWs as well, my father had a brown gasoline rabbit (1972 I think) and my mother owned a silver diesel rabbit (a 1979 I believe). I owned a number of odd cars before also getting my first VW, which was a 1997 Passat 1.9 TDI Sedan. In between my other cars and finally getting the 1997 TDI, I purchased a diesel rabbit for my wife which eventually was replaced by a 1994 Jetta, which was then replaced with a white roll top Cabrio. As the Cabrio declined, her and I switched cars for a little, while my wife bought her first brand new car purchasing a new green metallic Cabrio with a tan convertible top.
To say that I like my VWs is a bit of understatement. Now to be sure, I am NOT as hard-core as some, but I have been very loyal to the VW brand. The cars are generally fun to drive, easy to repair and maintain for the average gear head and at least for all the VWs I owned relatively cheap to own and maintain, that is until my purchase of a 2003 Passat gasoline 1.8T Wagon.
Track Record
In fairness before I get into the rant proper, all the VWs I have had have had some trouble with specific items. The diesel rabbit had the 'rabbit leak' over the fuse panel giving it all sorts of interesting issues including a propensity to short and run the battery down while it was 'sitting' for multi-day periods. The same problem was prevalent in the white Cabrio that my wife had for a while. Despite those problems the cars continued to operate and run and the fixes for any 'major' problems either of those cars ever had averaged a MAX of $500. We had a belt wear out on the 1994 Jetta, never had any trouble with the green Cabrio right up to trading it in on the 2003 Passat. The 1997 TDI that I purchased had relay109 fail, has gone through a number of window regulators and other than normal maintenance has had few if any real problems. My TDI currently has 215,000 miles on it and recently got the infusion of the largest amount of TLC that it has ever gotten when I spent $3200+ to do the following:
- Belts + Pulleys
- Suspension on ALL 4 corners
- Struts, Springs, etc.
- Water Pump
- Window Regulator on the Drivers Side
- Add a CCV Filter (Mann Provent)
- Remove and Clean the Intake Manifold
- Remove and clean the Intercooler and Piping
- Replace the Entire Exhaust from the Turbo Back to the tail (The OEM had essentially fallen OFF the car)
- New Clutch
- Starter Cleaning/Rebuild
- Water Thermostat
The reason I will NEVER own another VW Automobile
The 2003 Passat Wagon 1.8T is 100% the reason that I will never own another VW in my life. This car has had so many problems that it makes me slightly sick:
- Fuel pump
- Fuel level sender/indicator
- Water Thermostat
- Radio Antenna Shorting/Rusting
- Moon roof 'Flooding'
- Ignition/Coil Pack Failure
- A/C Bearing Failure
- OIL SLUDGE (x2)
- An oil sump that is too small
- A quantity of oil that is also too small to dissipate the engine heat appropriately
- An originally specified oil filter that is too small to cope with oil sludge that occurs because of 1 and 2
- A turbo that can get hot enough to glow with a slight cherry color
- Insufficient cooling of the turbo means that engine oil comes in contact with VERY high temps compounding oil sludge issues
- Using 'conventional' oil compounds these issues
- Oil screen and uptakes that are too small
Post after Post after Post after Post after Post of people with the same oil sludge problem and VWNoA essentially turning a blind eye to the problem. The engine should have just been recalled en-mass because it was prone to this issue.
No car I have EVER OWNED has had this level of problem with oil sludging. Not my TDI, not any of the rabbits, cabrios, jettas that I have owned have ever had a problem like this, even with some serious abuse of the change intervals - NOT A SINGLE SLUDGE PROBLEM. Change with conventional oil, change with synthetic oil, change with some mix of both in the same engine... not a single problem. The 1.8T seems to MAKE sludge due to its design. It would have been interesting if VW had made some improvements/revamps to some engine parts to attempt to address this issue. Instead VW redesigned the whole thing in a beefier 2.0T engine and did away with the 1.8T engine all together.
Quality Counts
I have never felt so abused/betrayed by a car in my life. Every time I turned around there was another problem leaving my wife stranded in the middle of nowhere. If it wasn't the oil sludge costing me money it was the fuel pump (eventually recalled) or the ignition coils (also recalled) or water in the cabin from the moon roof drains being clogged. This by far has been the worst VW I have ever owned and despite a long history of VW ownership - this car has caused both my wife and myself to turn away in disgust from VW in general. VW's inability to take responsibility for poor design and parts choices is inexcusable in my mind and I cannot take the chance of getting another car like this one. Sorry VW - with this latest round of engine oil sludge in my 1.8T (despite religious, possibly even anal retentive oil changes) you have just lost a long time customer (from childhood till now) of your cars.
Tuesday, November 23, 2010
JRugged - Making your code more RUGGED
Cowboy coding
We have all done some cowboy coding at some point in our life. I think we can even recognize when we may be asked to perform our cowboy coding. The scenario is something like:
Your boss or your boss's boss comes down and says "... we have to have this new thing - and we have to have it by this deadline or the sky will fall and we will all die..."
It is at this point that as a developer you begin to run through how you might develop what is being asked for and work through in your head exactly how you will tackle the problem at hand. It is also usually right at this point that decisions about what short cuts need to be taken in order to get the job complete on time get made. We, as developers, short cut things like:
Out of the box monitoring
When I mention monitoring what is the first thing that comes to mind? Is the machine my software is deployed on running? Does it have connectivity? What is the program’s memory footprint? How much free memory or disk is available? While all of these are important base questions to know the answers to - they do very little to help you understand your running, deployed software. To understand your running software you need to be able to answer questions like:
How often is it that case that the software you build has to call an outside resource? Maybe your software needs to make an API call to another system to get some information/data or maybe your system has to integrate with a remote system in a specific way; how do you insulate yourself from that other system's failures? How do you go about keeping the system you develop responsive and allowing it to fail quickly and respond back to the end users in a gracefully degraded way?
I believe that making a software system that "gracefully and quickly" fails when an outside resource is not available is usually accomplished by introducing something like timeouts, retries or other systematic back-off mechanisms. Adding timeouts or back-off can be problematic and can cause additional unforeseen issues like threads that hang or thread counts that run out of control. What we really would like to do is detect errors and if the error rate is high enough turn off ALL remote calls to that resource to save the user and the system from the cost of having to 'wait' for timeouts to occur.
Enter a Java project that helps you move beyond out of the box monitoring and timeout-based fault tolerance with ease: JRugged.
JRugged
JRugged provides straightforward add-ons to existing code to make it more tolerant of failures and easier to manage/monitor. In other words, it makes your Java code more rugged!
The purpose of the project is to help answer the questions we posed above in a straightforward and easy to understand way. By answering questions like how many requests per second am I processing currently, JRugged makes it dead simple for any project to be able to gather and understand the metrics for their running systems as well as assisting in making those production systems as resilient to failure as possible.
For collecting performance statistics we have a PerformanceMonitor object that provides the following output:
For adding fault tolerance and resilience we have CircuitBreakers. CircuitBreakers provide the following characteristics:
There are three modules in the project:
jrugged-core
Contained in jrugged-core are building block classes that can be used by independently to build out functionality within your application. Most of the items in jrugged-core utilize a simple decorator pattern allowing them to be easily inserted into your existing Java projects with little or no impact.
For example, if a developer wanted to wrap a performance monitor around a section of code for a backend call it might look like the following:
If you were then interested in the number of requests per second made to that back-end you could interrogate the 'perfMonitor' object to find out.
jrugged-spring
The jrugged-spring library is built upon the classes and items exposed in jrugged-core to provide an easy Spring integration path. jrugged-spring provides Spring interceptors that can be utilized in conjunction with a Spring proxy to 'wrap' methods in classes based on regular Spring configuration files. If you are currently using Spring, this is the way to go, as there is no change to your existing code needed. Just add needed lines to the Spring config and you automatically have the performance information gathered into an object that can be exposed easily with JMX.
jrugged-aspects
Similar to the jrugged-spring library, the aspects library provides the user with handy annotations that can be used on methods to wrap the performance monitoring or circuitbreaker classes around the target method. Getting statistics or incorporating graceful and quick failures becomes simply a matter of adding the appropriate annotations and assigning a name to get it to start collecting information or providing the fault tolerance of a circuit breaker.
What is the take away?
Having lots of information about how our software runs and handles failures is important to the business and we should already be building mechanisms into our developed code to provide them; the problem is that there is rarely time to do so. JRugged makes adding these critical components to your software so easy that it would be criminal not to add them. Please go and check out the project at http://code.google.com/p/jrugged/; we are always looking for comments and enhancements on how this works out for you and suggestions for future enhancements.
We have all done some cowboy coding at some point in our life. I think we can even recognize when we may be asked to perform our cowboy coding. The scenario is something like:
Your boss or your boss's boss comes down and says "... we have to have this new thing - and we have to have it by this deadline or the sky will fall and we will all die..."
It is at this point that as a developer you begin to run through how you might develop what is being asked for and work through in your head exactly how you will tackle the problem at hand. It is also usually right at this point that decisions about what short cuts need to be taken in order to get the job complete on time get made. We, as developers, short cut things like:
- unit-testing (because we tend to do it after development, making it feel superfluous)
- monitoring/data collecting about the system we are developing
- resilience to failure
Out of the box monitoring
When I mention monitoring what is the first thing that comes to mind? Is the machine my software is deployed on running? Does it have connectivity? What is the program’s memory footprint? How much free memory or disk is available? While all of these are important base questions to know the answers to - they do very little to help you understand your running, deployed software. To understand your running software you need to be able to answer questions like:
- How many requests per second has my software performed in the past minute, hour, day?
- How often did my software fail in the past minute, hour, day?
- How often did my software succeed in fulfilling a request in the past minute, hour, day?
- What was the latency for the calls that were made into my software in the past minute, hour, day?
How often is it that case that the software you build has to call an outside resource? Maybe your software needs to make an API call to another system to get some information/data or maybe your system has to integrate with a remote system in a specific way; how do you insulate yourself from that other system's failures? How do you go about keeping the system you develop responsive and allowing it to fail quickly and respond back to the end users in a gracefully degraded way?
I believe that making a software system that "gracefully and quickly" fails when an outside resource is not available is usually accomplished by introducing something like timeouts, retries or other systematic back-off mechanisms. Adding timeouts or back-off can be problematic and can cause additional unforeseen issues like threads that hang or thread counts that run out of control. What we really would like to do is detect errors and if the error rate is high enough turn off ALL remote calls to that resource to save the user and the system from the cost of having to 'wait' for timeouts to occur.
Enter a Java project that helps you move beyond out of the box monitoring and timeout-based fault tolerance with ease: JRugged.
JRugged
JRugged provides straightforward add-ons to existing code to make it more tolerant of failures and easier to manage/monitor. In other words, it makes your Java code more rugged!
The purpose of the project is to help answer the questions we posed above in a straightforward and easy to understand way. By answering questions like how many requests per second am I processing currently, JRugged makes it dead simple for any project to be able to gather and understand the metrics for their running systems as well as assisting in making those production systems as resilient to failure as possible.
For collecting performance statistics we have a PerformanceMonitor object that provides the following output:
RequestCount: 26 AverageSuccessLatencyLastMinute: 974.3247446446182 AverageSuccessLatencyLastHour: 1051.3236248591827 AverageSuccessLatencyLastDay: 1052.9298656896194 AverageFailureLatencyLastMinute: 0.0 AverageFailureLatencyLastHour: 0.0 AverageFailureLatencyLastDay: 0.0 TotalRequestsPerSecondLastMinute: 0.34042328314561054 SuccessRequestsPerSecondLastMinute: 0.34042328314561054 FailureRequestsPerSecondLastMinute: 0.0 TotalRequestsPerSecondLastHour: 0.006920235124926995 SuccessRequestsPerSecondLastHour: 0.006920235124926995 FailureRequestsPerSecondLastHour: 0.0 TotalRequestsPerSecondLastDay: 2.893097268271628E-4 SuccessRequestsPerSecondLastDay: 2.893097268271628E-4 FailureRequestsPerSecondLastDay: 0.0 TotalRequestsPerSecondLifetime: 0.6241298190023524 SuccessRequestsPerSecondLifetime: 0.6241298190023524 FailureRequestsPerSecondLifetime: 0.0 SuccessCount: 26 FailureCount: 0
For adding fault tolerance and resilience we have CircuitBreakers. CircuitBreakers provide the following characteristics:
LastTripTime: 0 TripCount: 0 ByPassState: false ResetMillis: 10000 HealthCheck: GREEN Status: UP FailureInterpreter: org.fishwife.jrugged.DefaultFailureInterpreter@39ce9085 ExceptionMapper: null
There are three modules in the project:
jrugged-core
Contained in jrugged-core are building block classes that can be used by independently to build out functionality within your application. Most of the items in jrugged-core utilize a simple decorator pattern allowing them to be easily inserted into your existing Java projects with little or no impact.
For example, if a developer wanted to wrap a performance monitor around a section of code for a backend call it might look like the following:
public BackEndData processArgument(final String myArg) {
final BackEndService theBackend = backend;
public BackEndData call() throws Exception {
return perfMonitor.invoke(new Callable() {
public BackEndData call() throws Exception {
return theBackend.processArgument(myArg);
}
});
}
} If you were then interested in the number of requests per second made to that back-end you could interrogate the 'perfMonitor' object to find out.
jrugged-spring
The jrugged-spring library is built upon the classes and items exposed in jrugged-core to provide an easy Spring integration path. jrugged-spring provides Spring interceptors that can be utilized in conjunction with a Spring proxy to 'wrap' methods in classes based on regular Spring configuration files. If you are currently using Spring, this is the way to go, as there is no change to your existing code needed. Just add needed lines to the Spring config and you automatically have the performance information gathered into an object that can be exposed easily with JMX.
jrugged-aspects
Similar to the jrugged-spring library, the aspects library provides the user with handy annotations that can be used on methods to wrap the performance monitoring or circuitbreaker classes around the target method. Getting statistics or incorporating graceful and quick failures becomes simply a matter of adding the appropriate annotations and assigning a name to get it to start collecting information or providing the fault tolerance of a circuit breaker.
What is the take away?
Having lots of information about how our software runs and handles failures is important to the business and we should already be building mechanisms into our developed code to provide them; the problem is that there is rarely time to do so. JRugged makes adding these critical components to your software so easy that it would be criminal not to add them. Please go and check out the project at http://code.google.com/p/jrugged/; we are always looking for comments and enhancements on how this works out for you and suggestions for future enhancements.
Subscribe to:
Posts (Atom)
