Be Driven by event(s) not dependencies

A very “sexy” term in today’s programming world, (Java anyways, especially pertaining to web-cum-commercial software development) is Dependency Injection, coined by Fowler and then made popular by IoC frameworks like Spring, it has made life easier for web projects providing an easy life cycle management for every entity associated with the project.

Dependency injection while solves many problems and allows projects to be simplistic, it has a very serious shortcoming, that is, it only EASES dependency management but does not ELIMINATE it. Surely people who have worked with such projects have realized and seen how tightly coupled tiers are when building a commercial web application using Java and DI frameworks such as those. The crux of such applications is being driven by how efficiently they manage their dependencies and not how they try and produce a de-coupled architecture so as to eliminate dependencies completely. Lot of people would think that producing de-coupled architecture leads to:
A.) Confusion
B.) Complex architecture that is hard to follow and also needless.

Both those points are agreeable to some extent, but it is also very important to point out how these applications must then be treated as just one application without any future. A big difference in thought process of building a product family vs. a single application meant for a specific (very specific in fact) purpose is that product family attempts to solve family of similar problems or interconnected or chained problems (Not to sound too theoretical), whereas the purpose of a single application is being driven by immediate needs. Both types of applications have their niche, and increasingly the second one is being adopted as a de-facto standard for building applications.

This is True for environments where time constraints are just un-avoidable and care has to be taken to meet deadlines; However it is VERY VERY unfortunate in my opinion. While a single application that “just works” might work and solve a single problem without any scope creep, without a long term software product strategy that allows each of those “single problem solving” applications that “just work” to be added seamlessly to existing architecture (I wont use term SOA here), no organization can survive. (Wow that really sounds like bunch of business folks mumbo jumbo, but its true, or I think it is ).

Hence building de-coupled architecture for each and every application and taking care of the factors that will allow each component to be re-used across several applications will empower an organization to use software tools and applications in a big way. Conceptually De-coupled architectures are not very new, in-fact one of the oldest architectures that exudes de-coupled architecture at its very heart is the concept of Event Driven Architecture.

Let us refer to Event Driven architecture definition on Wikipedia, which is I guess simplest to understand.

Event-driven architecture (EDA) is a software architecture pattern promoting the production, detection, consumption of, and reaction to events. An event can be defined as “a significant change in state”. For example, when a consumer purchases a car, the car’s state changes from “for sale” to “sold”. A car dealer’s system architecture may treat this state change as an event to be produced, published, detected and consumed by various applications within the architecture.

Several components of event driven architecture make it a suitable candidate for any and all kinds of applicable fields/domains. Some of those are:

• Separation of concernsEvent driven applications provide with purest form of separation of concerns as each of the so-called listeners will listen for event or group of events (more on this later) and so are more naturally reactive when compared to traditional counterpart components (especially in java web applications)
• Simplified architecture, oh well it sounds implausible at first, but it is true. True event driven architectures are easier and more systematic to follow, this is primarily because of the fact that there are only two components at each tier or for each tier, listeners and events broadcasters (well mostly, terminology would make it sound complicated but it really is not).
• Business Intelligence friendlyDriven by events allows for application to be more conducive to “learn” (This is primarily true because series of events in a particular order and their result can allow applications to formulate rules or rule-sets that can be applied in future series of events to detect stuff (more on this later)

Now that we have discussed and have sung hymns in praise of Event Driven architecture, let us look at basic structure of an event driven application. An event driven application (This is simplified to make it practical and easier to understand), typically consists of:

• Event(s)
  Events form the heart of an event driven applications, An event in its most basic and simplified form encapsulates a particular state of a particular system or a model that stays immutable (but can be enriched with some additional “resulting” data (typically results from processing of the event by listener). It makes sense that events are immutable because if we think about events, once they happen, well they have happened, we cannot go back in time and reverse those events (unless our application uses a time machine to go back and make those changes, oh and that will drive our applications and its rules crazy ).
• Event Broadcaster(s)
  A broadcaster broadcasts an event to “stream it” (the term that is often used to say to provide the event to the event “sink” or the event engine. A broadcast of an event happens once an event has been triggered, A broadcaster typically hands over the event to an event engine which will do some internal processing and then hand it over to an appropriate event listener / processor.
• Event Engine
  An engine typically uses a context or a placeholder or first level cache of some kind to know what events are being handled by whom, At present I cannot think of a way to make this more dynamic (NOTE: We computer science folks are always driven by behavior of the natural world, I guess in natural world when an event happens if you happen to be vicinity with all sensory perceptions, you will see / perceive … but lets not get carried away by this ). Event engine loads the cache and uses it to dispatch the event to the appropriate listener(s).
• Event Listener(s)
  A listener typically registers itself to the context/cache that is used by the engine to associate events it is registered for. When an event occurs a listener is the place where final processing of the event will be done and the event itself will be “enriched” with the resultset data to provide final results.

Now that we have a fair idea as to what event driven architecture is like, let us try and associate how this can be transformed easily into an atypical three-tiered commercial web application and be converted to a reliable model for software development. In a traditional web application, there are three tiers, namely presentation tier <--> business tier <--> DAO tier. I will not go into details of each tier(s) and their component(s), but if you are curious refer to J2EE Design pattern catalog here: J2EE Design patterns catalog. In essence each tier is dependent on the tier below to get its results and then finally provide results back to the client.

It is this architecture that makes each of the tier highly coupled with the tier below. Lets try and use this paradigm for a moment and transform it into an EDA (event driven architecture). Our sample application will include three tiers as above, but components will change within tiers to use Event Driven programming paradigm to allow for thorough de-coupling of the tiers. The structure will be as follows: NOTE: Just to emphasize, I’ll discuss event engine and other internals and possible coding snippets in next post, I don’t want this blog post to be too long to read)

• Event(s): To add to what is already been explained, several types of events will be assumed:
  
  • Business Events that are handled by business listeners
  • DAO events that are handled by DAO listeners.
  • Auditing events that are handled by auditing listeners
  • Generic events that are handled by pretty much all listeners
• Presentation Tier:
  • Event DTO shuttle: (This needs to be a post in itself, on which I’ll elaborate later), for now we can think of event DTO as a wrapping entity for an Event itself. (typically will contain getter and setter for an event type) and a broadcast method that will use a broadcasting agent to deliver an event. When an event occurs (like a mouse click by user for a purpose, broadcast method is invoked on this DTO shuttle which in turn uses broadcasting agent to broadcast an event.
  • Broadcasting Agent: Essentially responsible for event broadcasts and queue an event for processing by an appropriate event listener that is listening for such an event. In simplified coding terms, this will be as simple as:
eventQueue.add(event);,Note that this happens in a broadcast method.
• Business Tier : Will typically contain business listeners, these listeners will have a method onEvent( ) that takes in the actual event it is being provided, extracts the model out of the event and processes it in some way, puts it back in the event, uses the current event to construct a new DAO event if necessary and broadcasts it, NOTE for now we will assume that DAO events be broadcast by only business listeners themselves and not by client in a direct manner.
• DAO Tier: Will typically container DAO listeners that will do communicate with database either by using native JDBC or with some ORM framework (which is irrelevant to the topic for now) and finally will provide a final result based event that will be again dumped to the queue managed by event engine.

Using above architecture as our basic guide, lets analyze one basic flow for a bit (I am too lazy to draw diagrams so I guess I have to write 1000 words ). Steps are as follows:

1. An event is populated based on request gotten from the client (this could be a JSON string to any web service request … whatever we don’t care for now)
3. Wrap the event onto DTO and call new SomeModelEventDTO().broadcast(new SomeEvent());
4. A call to DTO broadcast results in broadcasting of an event using Broadcasting agent, which will simply dump the event to the event queue.
5. Next, Internally engine takes over and uses context and configurations to call appropriate listener by exhaustively going through queue of concurrent events. This call is essentially achieved by doing new SomeEventListener().onEvent(new SomeEvent())
6. Business listener performs basic validation and uses logic to process events it gets and then finally once everything is ready, calls a broadcast to add DAO events to DAO event queue that is once again managed by event engine.
7. DAO listener for dao events are invoked using the same mechanism by event engine to allow for further processing and persisting the data if necessary. A DAO listener once finished will further make a call to an event (something like event.setResult(new ProcessedResultFromDB()); and then finally add the event enriched to event queue.
8. Once event is added to the event queue, it is then picked up by the engine and dispatched to the DTO (fancy term to say dispatched, a simple new DTO().setResult(new ProcessedResultFromDB()); would do .

Observations

One of the first things that we can notice is that everything is dependent on event dispatching mechanism. But leaving that piece of discussion for the next post for now, one of the basic and very big advantages of the above flow is that no tier is dependent on the next tier or tier “underneath” from a traditional java web applications perspective. This makes for highly customizable applications where tiers can be swapped easily, re-used. For example, if we want to re-use business tier listeners, we can do that easily by simply adding and associating those listeners with another application (this could be done by adding them in either XML or annotating them using some @EventListener(type=Business) kind of annotation, anything). What does this buy me? Well for one thing, nothing is inter-dependent and therefore everything is so highly re-usable, All we need to know from an application development standpoint is what kind of events i want to track/isolate and how do i react to those events.

This makes for easy and highly customizable applications where presentation tier swapping (with a better UI tier) or DAO tier re-use becomes a distinct reality amounting just config changes either through XML or through annotations.

Summary:
We discussed, very briefly what an event driven architecture is, what are its advantages, how it can de-couple tiers and allow for re-usability, Next we discussed a possible sample application and how the hearts and parts of the application would work under event driven paradigm. In the next post, I’ll discuss more about:

• Event engine
• Event engine context / first level cache
• How event based frameworks give way for intelligent systems and perhaps AI?
• How event driven applications be used to still shell out applications really fast
• How does HCIS (my own framework) scale up to the task.
• SOA 2.0 so as to speak where SOA when combined with Event driven architecture makes way for a new paradigm altogether.

Finally, thanks for encouraging emails and queries about when the next post is due … that kept me going. See you next time!.