You may have missed it but Progress has sold its Apama division to Software AG and StreamBase has been acquired by TIBCO. This means that there are now no significant independent vendors of complex event processing (CEP) products. Now, Software AG is about integration and middleware while TIBCO, notwithstanding its ownership of Spotfire, is pretty much about the same things. No doubt both companies will continue to play in the capital markets space but I expect both of them to drop any pretensions to being general-purpose analytic platforms for high velocity data. Certainly, I don't see either of them competing very successfully with either SAS or IBM for general-purpose environments, which is probably why the sellers were happy to sell.
Since neither SAS nor IBM refers to its respective product as complex event processing we may as well kiss that term goodbye: (event) streaming is where it is now at. However, it is worth considering whether event streaming is always the best approach. In particular, advocates of Cassandra would suggest that event streaming is fine if you just want to analyse what's happening right now but that if you want to understand what's happening now in the context of what was happening five minutes ago and five minutes before that - in other words, trending - then Cassandra is better option.
And, of course, there's Impala, which is being touted as real-time Hadoop. Well, only if you conveniently ignore "near" or even possibly "far". Cloudera Impala allows "real-time queries against data stored in HDFS or HBase" and, it is claimed, is an order of magnitude faster than Hive. Good. How long does it take to store the data in Hadoop? If we are talking about scads of high velocity data this isn't going to compete with either streaming or Cassandra. It's good, I like it, but don't confuse it with either of these two, which have been engineered from the outset for real-time analytics against streaming data.
Of course, the other problem with streaming platforms, as opposed to Cassandra, is that they are very expensive. This has limited the market opportunities of companies in this space, more or less, to fraud, security and capital markets plus a few isolated use cases. Cassandra, it seems to me, has much greater potential to grab market share and until and unless the big boys with the big streaming platforms bring their prices down dramatically, what we are going to see is more and more Cassandra implementations, which must be good news for DataStax and Acunu.
24th August 2013: 'Jerry Baulier' said:
I agree that there has been a move away from the term Complex Event Processing (CEP) in favor of Event Stream Processing (ESP) for these engines that analyze/process event streams. Iâve always favored the Event Stream Processing label better even though we called the Aleri product CEP when we built and marketed it. There was a lot of buzz around the CEP term after Luckhamâs book and the EPTS (Event Processing Technical Society) rallied around it. I find the Event Stream Processing label much more intuitive, and when we built the SAS stream product we called it event stream processing rather than CEP.
BTW, most CEP/ESP systems do support trending using pattern matching. So for example, in the SAS ESP Engine one can use Pattern Matching Windows to define patterns of interest. Patterns have âevents of interestâ defined using something like a âwhereâ clause and pattern trees where the nodes are the âevents of interestâ or operators like AND, OR, NOT FOLLOWED-BY with optional temporal conditions. This happens to be one of my favorite features of our system and can be used to find correlations of events (including non-events) over time.
While there are still ESP opportunities in Capital Markets, I believe the higher growth is for surveillance opportunities like personalized marketing, operational management (oil, energy, Telco, manufacturing, etc.), cyber security, and big data analytics. ESP stand-alone is not nearly as interesting as ESP front-ending predictive analytics, which is why doing this now at SAS is so exciting.
Reply to Jerry Baulier?