FuseSource just recently released enterprise versions of their distributions of ActiveMQ and ServiceMix. Some of the top features include incremental patching, custom platform installers, and third party verification. One of the coolest features that comes part of the new enterprise editions is Fuse Fabric and some new tooling that allows you to leverage Fabric to manage your environment. Fabric allows you to deploy, configure, and provision ESB containers and MQ Containers with a few command lines or the click of a button in a slick web-based application. Deploy as many containers as you need to remote boxes or the cloud, push custom configuration, and install or upgrade apps, camel routes, or the container itself all from a central location. Setting up large deployments for integration and managing them is difficult. Fabric and the accompanying tools helps ease this pain. Also, you can feel comfortable using Fabric because its license is the same open, flexible license used by the other Apache projects: Apache License 2.0.
Large Deployment Headache
Let's briefly review some of the issues of deploying and maintaining integration containers across the enterprise, whether in your datacenter or remote locations such as mobile platforms or business-partners:
Installation: Downloading and installing, for example ActiveMQ, is easy enough on one box. But say for example you have a complex configuration of a network of brokers and master-slave pairs. You would have to ssh into each box, untar the distribution, set permissions accordingly, set up any directories or environment variables part of your install and then start the broker on the correct port.
Configuration: Once you've got the broker installed, you need to configure it to participate in the network of brokers with the network connectors set up correctly, point the broker to a shared store, set any flow control or subscription policy details or whatever other broker configuration tweaks that are part of your topology. Throw in the fact that your enterprise might be made up of different operating systems, machines of different resource availability, and purposed for different use cases, and you have a potential nightmare.
Upgrading Containers: With upgrades you have to follow the two steps from above and hope you haven't messed up anything. If for whatever reason you need to roll back, you must make sure you have a custom roll-back strategy in place.
Container Intropsection: What are some tools you use to see basic statistics of OS/Machine usage? Or maybe details of an ActiveMQ or Camel based solution? You probably need to log into each box, or locate the JMX url for each box. Needless to say, doing this for large deployments would be prohibitive.
Fuse Fabric was created with the purpose of solving those maintenance issues. Let's see how. First, you may be interested to get Fuse Fabric or Fuse Management Console. FuseSource recommends using Fabric through the Management Console. Note these two things: 1) Fabric is shipped by default with Fuse ESB Enterprise and Fuse MQ Enterprise (as mentioned above), but 2) the Management Console, which consists of a command-line shell and a web app, requires a subscription for the webapp portion. The web app requires the subscription. The shell console doesn't.
So download the Fuse Management Console (FMC) and fire up the shell with this command line:
./bin/fmc
You'll notice that this is a Karaf container with a lot of things packed into it. Karaf is an OSGI container with a few extra goodies built into it. Let's take a quick look at some of what is distributed with the FMC:
From the shell, type the following which will list all of the bundles with threshold > 0
FMC:karaf@FuseManagementConsole> list -t 0
Some bundles of interest:
- Key Karaf bundles necessary for a running Karaf instance
- Fuse Fabric Zookeeper service
- Fuse Fabric Core
- Jetty bundles
- jclouds bundles
- Fuse Application Bundles
- Fuse Management Console Web Archive
A couple of key things to notice: this idea of a "zookeeper" service, "Fuse Application Bundles" and the Fuse Management Console web archive. As I mentioned, the Management Console requires a subscription to FuseSource, but there are plenty of opportunities to try it out without having a subscription key. There are trial keys that can be used to evaluate it.
Fuse Application Bundles is a new project from FuseSource that makes it even easier to deploy your jar artifacts (or war) as OSGI bundles without having to rely on knowledge of an OSGI build too (bnd) or the OSGI import/export details of the manifest. It allows you to deploy your artifacts and have the framework correctly recognize it as an OSGI bundle with all the correct dependencies configured automatically.
Lastly, the Fuse Zookeeper service is there to provide a distributed registry where configuration and node details are stored. Let's go into that a little bit more.
Fuse Registry
Apache Zookeeper underpins the implementation of Fuse Fabric by providing the distributed "coordination" registry service. What this means is that Zookeeper provides a registry whose purpose is to store information about remote containers that might be deployed in the enterprise. The Fuse Fabric registry stores information about particular integration containers, configuration of those containers, where those containers are, and how to access them. You can run multiple replicated registries in a cluster (ensemble) to provide high availability in the event some registry nodes go down. An odd number of registry servers are recommended to form an ensemble. Once you've fired up the FMC shell, let's create a fabric with the following command which will create a registry:
FMC:karaf@FuseManagementConsole> fabric:create
Let's take a quick step back. What containers are we talking about?
Containers
Fuse Fabric's purpose is to manage, configure, and provision containers. The containers are Karaf containers. That's right, Karaf containers which are OSGI containers. The management, configuration, and provisioning mechanisms are around Karaf, what bundles are installed, what features are enabled, what configurations are available for the features/bundles, etc. Karaf is the core container currently used for Apache ServiceMix and Fuse Enterprise ESB. If you'd like to brush up on Karaf a little bit, see the documentation at the Apache site.
So anything that is deployed through Fabric is Karaf containers, and the location, configurations, and accessibility of these containers is kept in the registry (which is implemented with Zookeeper). So far so good? Let's get into how we specify configurations of these containers.
Profiles
Fuse Fabric has something called profiles which allows you to specify the configuration of the containers. Configuration is probably too bland of a word to convey exactly what a profile does. A profile "breathes life" into an otherwise uninteresting Karaf container. Let's go to the shell really quick. Type the following:
FMC:karaf@FuseManagementConsole> profile-list
You should see profiles with names like "camel", "esb", "mq", "example-mq", and a few others. These are the profiles that are available out of the box. The camel profile turns a container into one that is capable of running camel routes. the mq profile turns a container into an ActiveMQ broker. The esb profile turns a container in which you can build camel routes, activemq clients, and use the NMR. In other words, profiles are "packages" of functionality that you can apply to a container. If you're familiar with OSGI and Karaf features, these profiles are implemented using features and bundles. A profile specifies what features a container should install, what Fuse Application Bundles, OSGI bundles, maven repositories, config-service properties, and system properties should be available and installed into the container. This is a very powerful way of grouping "configuration" to be able to apply them with a click of a button to turn a Karaf container into a much more capable integration container. Multiple profiles can be applied to a container, and a single profile can be deployed to multiple containers (a many-to-many relationship).
The profiles that are available by default in the FMC can be used as starting points for creating your own profiles. Profiles can be inherited from and child profiles can override parent configurations. You can use inherit from multiple "parent profiles" as well. Suppose you have a set of containers that should all have the same camel route deployed but each container will have different configurations for that route. You can start with a parent profile and inherit from it for other profiles that specify more specific configuration.
Profiles can also be versioned. For example, out of the box you'll see that all profiles are considered part of "version 1.0". A version specifies a "set of profiles." If you wanted to create a new version, you create a new set of profiles for version 1.1 or 2.0 or whatever number you give it. Using versions, you can quickly deploy new versions of containers, apps, brokers, configurations, etc. to either select containers or to all containers. The versioning mechanisms is how Fuse Fabric allows you to support rolling upgrades of deployments to containers.
Fuse Agent
All of the profiles for all versions are stored in the Fuse Registry (zookeeper). When you create a new container using Fuse Fabric, it's created with an agent service. The agent or Provisioning Agent is what lives in the remote containers and connects to the registry to figure out what profiles are applied to it. When it sees a profile, or a change to a profile, the agent is in charge of provisioning and installing the required features, bundles and configs.
An implicit restriction on the containers that are deployed with Fuse Fabric is that you don't need to log into the container and locally make changes. All changes should be managed by the registry and Fuse Fabric.
Recap
A distributed registry is used to catalog information about what containers have been deployed, where they are, and how to reach them. A registry also contains all configuration in the form of profiles and all versions of profiles. A container, which is just a Karaf container, gets deployed with Fuse Fabric and connects to the registry to see what profiles have been applied to it. The container's provisioning agent takes care of downloading, installing, and configuring all features and bundles specified by the profile. The sort of profiles that can be applied include those that add broker features, ESB features, camel routes, Fuse Application Bundles, CXF web services, OSGI services, and anything else that can be deployed into an OSGI container.
In this post we took a broad overview of what Fuse Fabric is. In part 2, we'll look at the capabilities of Fabric a little closer, and demonstrate some coolness that allows us to solve the headaches involved with deploying and managing a large number of containers or brokers.