Author Archive

Architectural Mission pt 2 – At Your Service

June 23, 2011

Grandpa: Nothing gave Buttercup as much pleasure as ordering Westley around.
Buttercup: Farm boy, polish my horse’s saddle. I want to see my face shining in it by morning.
Westley: As you wish.
Grandpa: “As you wish” was all he ever said to her.
Buttercup: Farm boy, fill these with water – please.
Westley: As you wish.
-The Princess Bride

Here to serve.

That sounds fantastic when somebody says that to me. It’s a bit harder for me to say that even though I know full well that when I serve others I usually benefit greatly if not more than the one I’m serving. My focus is not about you or me but about software. Software that can be a service to others.

Service Oriented Software – got it. SOA, SaaS, ASP (not that one, the other one*), ESB, EJB, ORB, CORBA, COM, ETC.

Wow, really!? …and that’s just scratching the surface. This is not a new idea by any means. There seems to be an ebb and flow to centralizing and decentralizing the elements that make up a given application. What I have found is that in and of themselves they are neither right nor wrong. They are what they are. It’s for a given context that a level of appropriateness comes to play. Hardware capabilities, intranet and internet speeds and accessibility, user expectations and usage patterns. These all start telling a story and create a setting in which to place an application.

Distributed applications are cheered and jeered at the same time. Sometimes by the same people. They represent an incarnation of DRY (Don’t Repeat Yourself). As such they can be awesome. “You need widgets? Call the Widget Service!” The ease in which you can get data and functionality can be very impressive. Pulling an application into a hundred remote calls can also be something else. Slow.

If you have a single application it may not make sense to go the SOA route. You can still design and build this way to think separate but deploy combined smartly making everything a local call without the over head of serialization. This pattern really works best to serve multiple applications that have shared needs or need shared data.

If the functionality is not intended to be shared with multiple applications do not make it a service. Don’t pull in consultants and fire up a major ESB project with crazy orchestration to negotiate and transform data between disparate systems when there’s only one consumer. We’ll talk more about application design another time. Right now I’m focused on services.

What is a service?

To me it’s just an application. Instead of the consumer being a user it is another application. It’s also data with behavior. Hey… that sounds like encapsulation from the OOD school. Go figure. It is. At it’s core SOA is a very Object Oriented approach to designing an application. This is why you can design this way regardless if you’re deploying as a separate service or as an integrated business unit within your application. You should be thinking this way already. We’re just gonna pull part of the application out so others can call it without:

1. Copy and pasting the original code (bad coding practice fraught with maintenance issues)
2. Making a static reference to your application (inappropriate coupling)

This service should offer value. It should be strongly cohesive and fully encapsulated. The service should only do what it is named to do. A ‘beach finding’ service should not know anything about gas stations. It knows beaches. You should not have any beach logic in your application, it’s in the beach service. If you want to know where it is ask the service. If you want to know how popular it is ask the service. If you want to know how to get there… ask the mapping service, the beach service doesn’t understand directions.

I like to think of services as Legos. Very simple to use and combine. By connecting a few together I have a working application. That’s when the magic happens. The real benefit is that the service is independent of your application and has it’s own delivery and maintenance schedule. If it’s tested and solid that part of your app is already done. Don’t do it again. It’s free for everybody after the first consumer. When your library is big enough you can build a robust new application quickly and primary focus on the business needs at hand. That’s adding value!

From a deployment standpoint I like my service layer to be out of the DMZ and locked up tighter. It’s an extension of your data and is equally sensitive. Lock it up and protect it as best you can. Sure this is a hardware issue but at some point we’re going to actually deploy something on a real box and it’ll be subject to real users in the real world. I recommend doing a security audit periodically so you know where you stand.

If you’ve followed our RESTful talks you should already be thinking of making RESTful services. Keeping them stateless will allow you to load balance and scale this tier as needed with minimal effort. Be careful as you may be stateless but if you’re caching data (yeah, yet another topic to dig into) make sure you’re either caching that externally or have a way to synchronize.

I’d be remiss not to state that your RESTful service should probably return plain text JSON constructs. They’re so easy to use these days. Most languages have utilities available to generate and consume JSON so you don’t even have to sweat this one. So you’re service can now be consumed by anyone, anywhere, and… wait. No, I don’t really like that all that much. I’m going to go back to locking up my service tier. If I really want this exposed externally I recommend creating a gateway application to handle those chores. Keep your tiers intact, and in ship shape. You’ll do just fine.

“I am Juan Sánchez Villalobos Ramírez, Chief metallurgist to King Charles V of Spain. And I’m at your service.”
-Highlander

*ASP is an acronym that meant “Application Service Provider” before “Active Server Pages” from Microsoft came along.

Architectural Mission pt 3 – Bottoms Up!

June 23, 2011

Previously I’ve mentioned that we’re on a mission (part 1, part 2). We’re instituting a new platform. A central part of this initiative is a remote service layer available to all of our applications. There is a lot of talk out there about services, orchestration, and RESTfulness, but who’s talking about what is IN that service?

Me, that’s who. Buckle up as we’re about to begin part 3 of our journey!

I was originally going to discuss the dangerous phenomenon of scope creep. This is the result of the scope of a project being expanded over its lifecycle. While this is indeed something to watch for, and can lead to disaster, I’m going to go in a different direction for this entry. Recent conversations within our team brought to light a more poignant topic. One that still gets to the heart of the matter I wanted to address!

“It is not the mountain we conquer but ourselves.” – Edmund Hillary

Top-Down vs. Bottom-Up design strategy.

Top Down means to start with the business use case and break it down into parts and implement those parts. From a services standpoint this would mean once the application has defined what it needs appropriate services are created to satisfy those needs.

This is where you’ll see odd service names, or service names that mirror business units. You’ll also see something else if you look closer. Methods that don’t quite fit together. If you take the business use case out do the calls seem to be a jumbled mess? Do they cross specific areas or domains?

Let’s define some more terms:

Separation of Concerns
This is the practice of ensuring there is minimal to no overlap in functionality between any two systems – in our case applications and services.

Areas of Responsibility (AOR)
This is a military term that defines a geographic area that a commander has authority over. For our purposes that refers to logical concepts and models and which application or service is the commanding officer.

If the only things that makes the individual method calls make sense is the common consumption by a single application you have a warning sign. Break the key ideas into groupings. These groupings can lead you to see the real AOR’s.

Something that may not be visible, and it one of the harder ideas to grasp is what you’re missing with Top Down design. You can meet the needs as requested and be successful. You had a request and you responded. Huzzah. You consolidated logic and optimized performance. You’ve applied caching and load balancing and have a robust system. These are all incredibly valuable and worthy in their own right… but would you believe that there’s more?

This is where Bottom Up design comes in.

Step away from the originating application and become the service. You really need to step into these shoes and now look at the world from this point of view. Ask the hard questions! Should my Beach service respond with ratings? Yeah, that makes sense. It’s information that is very specific to the beach. When I want the rating I’ll most naturally want to start there to see if I can find this data.

Should it respond with directions? Should it contain it’s own address? Oh, now we’re approaching the grey area.

How many other objects have an address? How many other objects will need to find directions? These concepts are related to the beach, but are not in the beach’s domain. I recommend keeping them out.

The beach can have a URI to an address object served by the address service. This is logical. The address service can return latitude and longitude coordinates. This is also logical. And the mapping service can give me directions, even if that service is really just Google Maps. Does Google know about my Beaches? Anything about my rating scheme? Nope, it doesn’t have to and is more powerful because it doesn’t.

Fiercely protect what is and is not in the domain of each service. When you do this something grand can now occur. Emergent applications.

When your services are designed to meet the applications needs, and yet remain flexible and locked into a core definition of what that service represents. It can be used by others. Used in ways that you didn’t design. This is the critical part. If you create a Bucket service intended to be used at the beach you could have success defining how much sand it can hold. But what a missed opportunity. The service that simply defines the bucket and rules for the bucket is more powerful. If that Bucket can be used to hold water, oil, rocks… whoa. How about turning it upside down to be stacked! You didn’t think of that one, but the service should. Not. Care. Define the entity and let the use case and applications define the rest.

In OOD terminology the service is the abstract class and the usage is the concrete implementation.

A system of services that can be leveraged for uses beyond their design. New concepts and ways of assembling applications that were previously impossible. This is what we’re doing. We’re starting to see the potential. We’re fighting for the services because they can’t fight for themselves.

“Tough times never last, but tough people do.” – Robert Schuller

Hold onto the vision, guard and guide it. Success is out there. Bottoms up!

Architectural Mission

June 15, 2011

Elwood: It’s 106 miles to Chicago, we got a full tank of gas, half a pack of cigarettes, it’s dark… and we’re wearing sunglasses.
Jake: Hit it.
-The Blues Brothers

I’m on a mission. I’ve been tasked with stewarding a new paradigm in how we build our software here at Lampo. There are many facets to this and being inspired by Kevin’s recent posts I, too, am going to try exposing the process as I go through it. I welcome you to join me on my journey and hopefully we’ll learn something along the way.

A new paradigm.

Have you introduced something new to your organization? It can be met with criticism or fanfare… or both. I’m fortunate to work at a company that believes in the book Who Moved My Cheese. This is required reading for our employees. The essence is that change is inevitable and there are ways to enjoy this process. I recommend reading this lil’ classic if you haven’t already. Some change ends up moving a lot of cheese. Changing fundamental architecture of all applications falls into this bucket.

What is this change?

Simply put we are moving to a service oriented architecture. That’s overly simplified and doesn’t encompass the whole strategy. “One Platform” is more like it. Our existing software is not bad by any means. There’s nothing fundamentally wrong with it. It’s exactly what has made us successful. If you’ve ever had the thought of changing how things are take a moment to remind yourself that your company’s success is based on the state of things today. It’s successful. The new change is untested, and high risk. Proceed with caution!

Why change?

Yes we have been successful. Our current apps have brought us to where we are. Will they get us to where we want to go? No. We’re growing and are experiencing growing pains. This is where change comes in!

We have a fair number of applications. We also have a fair number of capabilities that are similar across many of these applications and are not truly specific to any one app. This includes, but is not limited to concepts such as handling content, managing users, security, user interface and user experience. Kevin’s recent posts on Implementing Modular Web Design is talking about one of these new foundational concepts. We are very decentralized and desire to move to a centralized system.

We’re weaving SOA into our fabric because it fits in with our objectives and our end goal to give hope. Always, always, always know why you’re in business and what you are trying to achieve. This change isn’t something we’re doing because it’s popular, or a great way to impress peers. We chose a direction that fit our team and fit our mission.

Fundamental changes are risky ventures. I’ve heard them called “science projects” and anyone bringing one up was asking for trouble. I’ve known some companies that have almost sunk their business with a “major re-write”. Embracing change also means to understand what’s at risk – both good and bad.

This type of open discussion has helped form our strategies around introducing these changes. This level of change is deep. It effects our estimates, project management, developer communication, code management, testing, deployment process, and governance. Risk management needs to be ever present when every stage of the development life cycle is effected.

Be Confident!

We have an amazing team. I’m confident in my teammates, the mission, and myself. I have no doubt we will be successful. I’m not going to measure success on adhering to a spec or standard on how to implement an SOA architecture. It’s about abstracting out common elements into a reusable toolkit in order to deliver better products quickly.

Elwood: It’s got a cop motor, a 440 cubic inch plant, it’s got cop tires, cop suspensions, cop shocks. It’s a model made before catalytic converters so it’ll run good on regular gas. What do you say, is it the new Bluesmobile or what? – The Blues Brothers

I’d love to hear about your experiences with change! Please respond below.

What’s in YOUR work week?

April 6, 2011

“Time – got the time tick-tick-tickin’ in my head” – Joe Jackson

I need more time!

Let’s take a look at the average work week.  A handful of one hour meetings, maybe one large meeting, many smaller meetings.  By mid-week the calendar is almost full.  People are coming to your desk constantly asking for small adjustments/favors/tasks.  That time-saving-must-do project you had in mind, once again, received no time.  It’s the end of the week, you’re drained, and looking back you’re not sure what really happened.

You can read time management books, take a Covey course and recite all of the key phrases.  These are all good, and highly recommended.  But I’m going to fast forward to a different ending that’s not in the popular reading materials.  First let’s post a supposition:

Jim’s Axiom – Creativity does not exist without constraints.

You may be thinking “I can’t be creative WITH constraints!”  I respectfully say you are wrong.  Let’s go back to when you were younger.  You had a clean, unwrinkled, pristine sheet of paper (or an entire pad) and a pencil.  You looked down and… nothing.  “Mom, what should I draw?”

Did you see what just happened there?  Maybe you remember that, maybe you just heard your own child ask that very question.  There are no constraints.  You (or they) can draw anything, and you draw… nothing.  Where do you start?  What should you do?  If only you had a starting point you could go anywhere.  You need a constraint.

When the pressure is on, and you need to reach the goal line you do what it takes, you look for the opportunity, and you press forward and find a way.  Once there are constraints your creativity can take root and navigate through and bring you to success.

Jim’s Corollary – Creativity is the ability to solve a problem when there are constraints.

I need to find a way to the store and the main road is closed.  Creativity.
I need to buy groceries and I’m low on funds.  Creativity.
I need to get more done and I’m running out of time.  Creativity.

The Pony Express was infamous for delivering mail.  We could have grown that system and bought more horses.  When they weren’t fast enough we could have bred faster horses.  But the limitations were real, and no matter how much you did more of the same thing you couldn’t solve the pressing issue.  It doesn’t scale properly, and was not cost effective.  There had to be a better way.

This new way did not involve a horse.  It was the telegraph.  A completely different way of dealing with the issue.  Taking a broken system and spending more time doing it doesn’t get you the advance you’re really looking for.  The same is true during your work day.  If you take your current process and just add more time you do get more done.  But at what cost?  And what value are you really adding?

If you find yourself working 60+ hour weeks I’m going to challenge you.  Are you really only smart enough to just do more of the same to beat out your competition?  Is that how you sell yourself?  What happens when the next guy comes along and works 70 hours?  80?  They win?

But what about studies that state efficiency decreases as hours per week increases?  A quick search on Google yielded this: “The average efficiency for 50 hr, 60 hr, and 70 hr weeks was 0.92, 0.84, and 0.78, respectively.”  That means that at 70 hours per week you’re as efficient as actually working 55 hours.  You’ve heard the adage… why have you denied it?  Work smarter, not harder.

• Look at the work you’re doing and note the time spent.  What’s really eating your day?  What new process can you put in place to replace the old, inefficient model?  Focus on the important/non-urgent, and manage the important/urgent.  Avoid the non-important items!  Recommended reading:  First Things First by Steven Covey (Full disclosure: I’ve taken the Seven Habits course and love his style on time management, but I haven’t read this book yet.  Ironic I know.)

• The next step is to examine your processes.  Why and when you do what you do.  Keep your meetings focused and only invite those that should be there.  Set an agenda with a desired outcome, and once reached, end the meeting, even if it’s early!  Recommend reading: Death By Meeting by Patrick Lencioni.

• Are you trying to do it all on your own?  Who’s your model or mentor for that?  Look for strategic partnerships to accomplish more than you could do on your own.  Recommend reading:  Mentored By A Millionaire by Steven Scott.

As a young boy I heard a wealthy and successful man say something that has stuck with me.   I believe I’m finally starting to grasp the full intent, and I’ll leave it with you now:

“If you can’t do it in 40 hours a week, you’re doing it wrong.”

Grails and Rabbits

January 12, 2011

Tim: There he is!
King Arthur: Where?
Tim: There!
King Arthur: What? Behind the rabbit?
Tim: It *is* the rabbit!
-Monty Python and the Holy Grail

This. Is. Ridiculous.

After SpringOne I was anxious to try Grails with RabbitMQ on my own.  I downloaded the complete bundle for Windows and set it up.  I’ve never run an Erlang application so I felt a bit funny, but it was simple and painless.  I set an environment variable for Erlang (ERLANG_HOME).  Then I just hard coded a ‘base’ directory for logging and whatnot in Rabbit (server.bat).  Yeah, I could have set another env var there too, but I was too anxious.

I started the server.bat file and had RabbitMQ running.  Though it was lonely.

Sir Bedevere: Well, now, uh, Lancelot, Galahad, and I, wait until nightfall, and then leap out of the rabbit, taking the French by surprise – not only by surprise, but totally unarmed!
-Monty Python and the Holy Grail

I jumped into STS and ran ‘install-plugin rabbitmq’
A simple config setting (Config.groovy):

rabbitmq {
   connectionfactory {
      username = 'guest'
      password = 'guest'
      hostname = 'localhost'
      consumers = 5
   }
   queues = {
      jimski()
   }
}

My Controller only needed to call:

rabbitSend 'jimski', msg

…and for fun I setup a Service to pull messages off the queue:

static rabbitQueue = "jimski"
void handleMessage(msg) {
   println "received message: $msg"
}

Voila! Messages taken in by the Controller make a call to rabbitSend and magically the Service sees them and pulls them out to display.  Crazy easy.  Crazy cool.

I’m going to keep at it.  Let me know your experiences with queuing.

Asynchronicity

December 21, 2010

“Daddy grips the wheel and stares alone into the distance.
He knows that something somewhere has to break” – The Police

I recently attended SpringOne 2GX and had a great time.  The folks behind Spring, Groovy, and Grails are fantastic.  Go find a NFJS event and be dazzled.  During one session I learned about RabbitMQ, a recent acquisition of SpringSource.  This is an impressive project done in only 12,000 lines of code!

I’m greatly intrigued by a message queuing system.  My only experience is as a gateway to a mainframe system.  Not exactly a keen architectural strategy as much as “that’s how you gotta do it.”  So I’ve seen something there, but have struggled to find a good use.

But I feel like I’ve found something!  We have many applications that have a common need to interface with an external system via SOAP.  This is an expensive operation and is currently handled synchronously.  Ah HA! Toss it behind a queue and make it asynchronous and regain some performance.  This call doesn’t need to have a response so the application doesn’t have to wait for the return!

Now throw in other ideas like an enterprise language-agnostic logging system, and step that up a bit with a destination for critical errors  to be stored for investigation and triage.  I think I now have a trifecta of ideas, and enough critical mass to justify revving up a new service!

What are your thoughts and experiences with asynchronicity?

Code Wants to be Stoopid

October 27, 2010

Clever code never did no-one nay but no good.  Not know-how, not no-way.

Yeah, I’m not sure what I just said either.  I do know that “clever” and “slick” are words that I don’t care to hear about code, or a coding solution to a given problem.  Code needs to be simple.  How simple?  As simple as possible [but no simpler, yadda yadda yadda].  “But my code IS simple” you say.  I’m done.

Cool.

…just one more thing.  Do all your methods only do one, and only one, thing?  Yes, I said all.

Every method (or function) should accomplish only what it’s name says it does.  If you have a method “saveThing” it should only save that thing.  It should not check for something else, optionally set a flag, make any other call, then save that thing, and return the price of tea.  It should only save the thing.  This means you will have many very small methods.

At a high level this will lead to something really powerful:  Readable code.  You see, as your fine granularity builds upwards you are essentially utilizing the ‘composition design pattern’ in a micro-economy.

What are some signs that you may have code that can be refactored into it’s own method?

  • Length.  If a method is growing beyond 10 lines, take a close look and make sure it’s still doing only one job.  As methods grow they want to do more than they should.  You can’t convince me that 1,000 line method is as small as it can get.
  • Unused parameters or only using one or two variables off an object.  Too much data will lead to putting process in the wrong place.  If you don’t have the data you can’t do the wrong thing with it.  Don’t pass in a few extra objects just in case.  If you don’t need them pull them out.  Use static analysis to remove unused and dead parameters (and code blocks).  If you really only need to do work on a username… just pass in the string, and not the whole user object.
  • Comments.  If you feel the need to explain a section of code inline, it should be it’s own function with a well suited name.  You either use too many comments (rare, and not cool but that’s another topic) or you use one or two in confusing places.  That’s the tingly feeling you’re trying to learn that says “put this code in it’s own method”.

This is the path to better code.  Code that does what it says it does, nothing more and nothing less.  It sounds so simple, but is the essence of a truly powerful system.  Keep your code dumb, you’ll thank me for it.

P.S.  I know it’s spelled “stupid”.  😉

BFusion/BFlex 2010

October 18, 2010

Last month, 16 Web Monkeys with Laserbeams went to BFusion/BFlex 2010.  What a cool little gem of a conference.  There were some truly world class presenters there.  One even shed new light on a sore spot of mine that made me reconsider my entire belief system.

Well, my entire belief system on Unit Testing.

Hi.  My name is Jim.  …and I’ve been abused by bad Unit Tests.

I’m sure to talk more about that eventually.  For now let me say that Michael Labriola over at Digital Primates is a fantastic guy with a solid view on writing solid apps.  He introduced the idea that separation of concerns is what’s key to unit tests.  The issue with tests is that they can’t test the code cause it’s untestable.  A method should do only one thing which makes for easier testing.

I like it.  I like it a lot.

Our very own Doug Smith spoke about RESTful services as well.  Yeah, I’ve been a POST-heavy addict as well.  It seems I’ve got issues.  REST makes sense.  Almost too much sense.

You have to catch these guys talk.  They’re really top notch!  Keep your eyes on next years Bfusion/Bflex conference.  It’s hot, and you’re sure to learn something and have a great time!