First of all, for full disclosure, for the last 1.5 years, I've been working as a consultant for FuseSource (now Red Hat) supporting SOA and Integration projects for large and small companies in diverse verticals from retail, shipping, banking/finance, health, etc etc. My specialty has been architecting solutions with high scalability and throughput demands using some of the best open-source projects in this space: Apache ActiveMQ, Apache Camel, Apache ServiceMix, Apache CXF, et al.
I am the author of one of the DZone Refcardz for Apache Camel titled "Apache Camel Essential Components" as well as speaker for the same titled webcast. I'm also a technical reviewer for an upcoming book on Apache Camel for Packt Publishing, which I hope to soon post about publicly.
Needless to say, I've done some work with Apache Camel, I contribute to it, and I encourage people to check it out.
But before I got started with FuseSource, I had experiences with other middleware integration projects -- some open-source, some commercial, but my first foray with light-weight open-source integration projects was not with Apache Camel.
I stumbled upon Spring Integration back in 2009 and quite liked it.
I wrote blogs about Spring Integration, contributed to the user forums, contributed bug fixes and examples, etc, etc. stayed a little active in the community. I have tremendous respect for Mark, Oleg, Gary, Gunnar, et. al. for their amazing work on Spring Integration.
I liked how it extended Spring and how closely it implemented the Enterprise Integration Pattens as cataloged by Gregor Hohpe and Bobby Wolf. I used it on a few projects back then, and I firmly believe you don't get a real feel for a project unless you've used it on a real project to solve real problems. But since I've had the opportunity to use both projects deeply enough, I would like to contribute to the blogosphere my observations and opinions :)
First, there are quite a few write ups comparing the two projects. Some focus on stats like community activity or #s of components. While relevant to some extent, I'd like to go a tiny bit deeper. So the following blog post is my thoughts about Apache Camel and Spring Integration, and (spoiler alert!) why I would choose Camel for integration projects.
Abstraction, Abstraction, Abstraction
Both projects aim to fill similar needs: light-weight integration library with full implementations of EIPs, for mediation and routing, as well as provide abstractions on top of commonly used technologies to interface with external systems. Some common technologies include Web Services (SOAP/REST), JMS or asynchronous messaging, File-based batch systems, TCP sockets, FTP/SFTP, RDBMS and an many others. Basically whenever two heterogeneous systems need to interact, either synchronously or asynchronously, and exchange data, a light-weight integration library could help you significantly. For those of you asking "isn't that what an ESB does?".. well... kinda... but I can write about that another time.
For those of us integrating systems, we find out two things really quickly about integration. #1 it's not sexy and #2 it's difficult.
Understanding the differences in the systems, both technical as well as semantic and how it's used to implement business functions is crucial. The code that you do end up writing should be focused on those business concerns and you should leverage existing commodity components where possible. Writing code to interact with the file system, or send messages to queues is definitely considered commodity code and just adds to the code base that you'll have to maintain. If you code it by hand/roll your own, you also accept the risk of introducing bugs into areas that are not crucial to the business logic or business semantics. In my humble opinion, that's an unnecessary risk.
So that's where Apache Camel or Spring Integration come in. They help make integration projects easier by providing access to commodity components that have been vetted by the community, as well as implement common EIPs like transformations, routing, filtering, splitting/aggregating, etc... again.. vetted by the community. If you code these things by hand, good luck :) Of course it can be done, people do it all the time, but it's an unnecessary risk for most projects.
Where was I....
Yes, so you can think of either of these two projects as "abstractions" above specific technologies. This abstract notion allows you to focus on "what the integration is supposed to do". And in an effort to solve this question, I believe Apache Camel shines.
Domain Specific Languages
Integrations are read FAR MORE often than they're written .. just the same as code. So just like you aspire to write nice clean code with short functions, descriptive variables, proper levels of data hiding, etc, etc, the same is true with coding for integration projects but taken to the next level. You want to code it so you can read it and understand it. Though with integration code, the full meaning and intent of "what is being integrated" ends up lost in piles of code even if your code is written neatly. You can leverage an integration abstraction library, but as good as they are for the purposes described above, what you want is clarity to be able to answer "what is being integrated".
Apache Camel brings to the table a Domain Specific Language and I believe this is a big key difference between the two projects. Using the DSL, you can very concisely and cleanly express "what is being integrated", even in the face of moderate to complex integrations which is where other projects begin to lack clarity. Apache Camel offers DSLs in Java (my favorite), Spring XML, Scala, Blueprint-OSGI, Groovy.. and for what it's worth, even Kotlin :)
Using the DSL, you build integration "Routes" with constructs like "from" "split" "choice" and "to". These allow you to specify in common language "what the integration is doing."
When I was using Spring Integration, they didn't have a DSL. You dealt directly with the Channels and Components (pipes and filters) using Spring XML and Channels were the main abstraction. They were working on a Scala DSL back then, but as much as I love Scala, it's still not very widely used, so for the purposes of using it on integration projects in common language or XML, there weren't many choices. Basically you hooked together endpoints like jms-gateway or http-gateway with channels and connected the other end of the channel to EIPs like Splitter or Router. But for every component you wanted to connect up, you also had to pay attention to what Channel you used what its inputs were and what outputs it had.
And once your project starts to grow, you find an explosion in channel objects. And it ends up diluting the meaning of the integration.
For example, take a look at the Cafe Spring Integration Example. This example isn't super complicated, but it illustrates what I ran into. Imagine splitting channels and components among many Spring context files and you can see how this can get quite messy.
Here is how the exact same example is implemented with Apache Camel:
Of course, I don't expect one to be able to pick up exactly what each example is doing without knowing or have used each of the respective libraries, but without a doubt I find reading the expressive DSL represented in the Camel DSL to be easier vs trying to interpret the SI route/flow with all of its channels. This gets me closer to "what is this integration doing" without being bogged down by details.
Testing
Another very important part to writing integration flows is testing. I won't go on my usual rant about how you'd be a fool not to test your code, but even doubly so for not testing your integrations. As complex as they can get, and the mediation that's involved, you want to verify you've got it working!!
At the time, Spring Integration didn't have any good practices for testing routes specifically. Presumably because they already have a generic test framework for Spring itself. You can get by with this for sure, because we're all used to writing unit test or integration tests with Spring and mocking out collaborators with EasyMock or Mockito, right? And if you wanted any extra capabilities you'd have to build them out yourself.
But with Apache Camel, rich testing support comes out of the box, and it's easy enough to use as to encourage testing your routes. Camel's testing framework is built on top of some of the Spring Test features, so it ends up being the best of both worlds. For example, you can mock out parts of a route that hit live systems and focus on the routing and mediation capabilities. You can set expectations and assert behaviors just like you can with Mockito, but it's built right into the library and adds additional capabilities. A colleague of mine, David Valeri has an excellent Testing and Debugging blog as well as his talk at CamelOne
For example, in the following snippet, we can assert that we've received two messages at the mock endpoint within a period of 5000ms:
You can even mock out live endpoints by applying AOP advice simply like this:
Along with the mock components, you also get functionality to assist in testing Blueprint XML if you're doing OSGI without having to deploy to a container, you can do soak tests or stress tests with the DataSet component, and many other complex testing scenarios.
Check out these links for more:
- Endpoint testing: DataSet
- Endpoint testing: Mock
- Endpoint testing: Test
- Stub out live technologies
- Camel Test
- Camel Spring Test
- Blueprint Test
Community
Lastly, but certainly not any less importantly, I would like to point out that Apache Camel has a vibrant, diverse community made up of contributors from all around the world at different companies working on different projects. Upon writing to the mailing list or submitting a JIRA, it's not uncommon to get responses within minutes. This same ecosystem, which you would expect to find at a non-profit, core opensource foundation like The Apache Software Foundation isn't exactly the same thing you'd find at other projects.
I certainly am not saying the Spring Integration forums and JIRAs aren't lively, but I perceive there to be more activity from non-project or Company (VMWare/SpringSource) personnel in the Apache Camel project. In my opinion that tends to fosters more creativity, more outsiders to contribute and make the project better, etc, etc.. the core of open-source and open community development. Of course that comes with more arguments :) but the net result is very positive.
In some of the other posts about Spring Integration vs Camel, the authors point to how many more components Camel has than Spring Integration. There is a point to be made there in terms of larger options for connectivity, but I think the more important point is that a lot of those were contributed by non-core Camel developers. Additionally, you can try doing a quick search on github and you'll find people write Camel components that haven't even been brought to the Apache community, but that you can still leverage.
Which one should YOU use?
So I have spewed my opinions in almost never-ending prose here.. but I encourage you to check out BOTH projects and decide for yourself. Others will have different experiences and opinions, and to some degree that will play a large role in deciding which to use for your next integration projects.
A lot of the above was my opinion. But if I've misstated something, or misrepresented something, please comment and correct me!