Event processing is a subject as broad as all of data processing; however, the infrastructure that is unique to events and event messages is essentially simple. This simplicity, which I will bring out in various posts to this blog, stands in contradiction to the prevailing paradigm known as Complex Event Processing (CEP). This post will present a very high-level architecture of an event processing infrastructure that has a universal application because of its very simplicity. Its simplicity is presented as challenge to the currently prevailing but awkward solution known as CEP.
Event processing is essentially about event producers, event consumers, and the messages that are transferred between them. It is not about the numerous possible ways that producers generate the event message and it is not about the infinite number of ways that event data is processed by event consumers. Event processing is ultimately about how producers can transfer information about a given event to consumers.
Event producers are software components that record the occurrence of an event in the physical world. They may be drivers that support the input of data from drivers and keyboards, or they may clock-driven programs that find geographical position and other information on a periodic basis. The production of events must be left to a universe of possibilities and not limited by a certain field of inquiry.
Event consumers, on the other hand, are software components that can only be defined by the fact that they receive messages representing events and consume them in their own processing. To say more about consumers would open the door to all data processing – they are capable of doing any kind of data processing task once they have received an event message and what tasks they perform cannot be the business of any definable event processing infrastructure.
To be a viable field of IT, event processing must set aside the infinite possibilities of how events are produced and consumed. It needs to limit itself as an IT solution to the delivery of event messages from the undefined producers to undefined consumers. This gives it universality as well as simplicity. Everything beyond the boundary of event messages and their endpoints opens the door to every requirement in the universe of IT.
In fact, the function of the event processing infrastructure should be to make the various producers and consumers as independent of each other by preventing them from becoming tightly coupled to each other. Producers should construct event messages without having to have any knowledge of the universe of message consumers that they could potentially serve. In turn, event consumers need to be able to consume events messages without having to have specific knowledge of their origin. Finally, both producers and consumers need to be as independent as possible from the actual event messages themselves. Allowing event producers and consumers to be developed and operated independently from each other should be the primary function of the field of event processing.
As shown in the diagram above, the essence of the event processing infrastructure being designed here is a simple publish-and-subscribe router that enables producers to serve consumers without being tightly coupled to them. Event producers only need to know about the universal router where they send messages. Consumers can subscribe to the event messages without being tightly coupled to the producers. In later posts we will see how both producers and consumers can be made largely independent of the event messages themselves.
Event processing is essentially the infrastructure that makes producers, consumers, and event messages uncoupled from each other. At the center of the infrastructure is a router that serves to make these components independent of each other and thereby allowing the components to be adaptable and agile.
