all things open

This being my initial post I thought it would only be appropriate to do something a little daring. I decided to tinker with the idea of building an Open Hybrid PaaS. Before you read on, you should be warned, this is NOT a supported use case and a lot of other problems remain to be solved before this concept becomes a reality. Nonetheless, it’s always fun to see what technology can do, and if we don’t tinker we’ll never drive vision to reality.

Before we begin, here are a list of projects and products that were used in this post.

Upstream / Enterprise Projects

• Katello / CloudForms
• Aeolus / CloudForms
• OpenShift Origin
• oVirt / Red Hat Enterprise Virtualization

Additional Products and Services

• VMware vSphere 5
• Amazon EC2

An Overview on OpenShift Origin

OpenShift Origin Components

The diagram above is from OpenShift Origin’s Build Your Own PaaS page which is an amazing resource for deploying OpenShift Origin and understanding the architecture of OpenShift Origin. OpenShift Origin enables you to create, deploy and manage applications within the cloud. It provides disk space, CPU resources, memory, network connectivity, and an Apache or JBoss server. Depending on the type of application being deployed, a template file system layout is provided (for example, PHP, Python, and Ruby/Rails). It also manages a limited DNS for the application [1].

Why do I need to flexibly provision nodes if my PaaS already scales?
It’s understandable to ask why a provisioning methodology is needed for OpenShift Origin. After all, OpenShift Origin can spin up gears inside a node to meet capacity. The problem is that OpenShift scales gears, but does not scale nodes, which are the container in which the gears run. Essentially, if OpenShift Origin runs out of nodes you are out of luck. Of course, in most cases it’s probably good enough to monitor the PaaS and provision nodes as capacity dictates. We, however, are nerds, and we don’t want to have any manual actions (eventually).

Beyond scaling nodes, the open source technologies of Katello, Aeolus, and OpenShift Origin provide users a means by which they can build a PaaS that spans multiple virtualization providers, including public clouds. This post demonstrates how a PaaS was deployed to VMware vSphere via Aeolus and Katello and then with a few clicks the same deployment was deployed on RHEV. While Amazon EC2 was outside of the scope of this exercise, I don’t see any reason why we couldn’t also run in EC2 with a few more clicks. The portability provided by the combination of these three projects is extremely valuable to organizations who wish to avoid vendor lock-in and choose the infrastructure they run on while utlizing a PaaS. Here is a simple diagram that illustrates the possibilities these projects bring to the table when used together. Note that Katello will utilize Foreman to support bare metal installations and Aeolus can create nodes or brokers.

OpenShift, Aeolus, and Katello Provisioning Capabilities

What would be really useful is if OpenShift Origin were able to request more resources from Aeolus or Katello and Aeolus or Katello could orchestrate the request based on the capacity and importance of other workloads running on either virtualization providers, physical hardware, or public cloud providers.

Nerd Heaven

This is probably a good topic for another post. Let’s move on before I get too far off course. In this scenario I’m demonstrating how Katello and Aeolus can compliment OpenShift Origin to provide a consistent implementation of OpenShift Origin across multiple virtualization providers.

Consistent OpenShift Origin PaaS across RHEV and vSphere provided by Aeolus

An Overview of Aeolus/Katello Terminology
It is probably a good idea to provide an overview of a few of the objects used in Katello and Aeolus.

• System Template/Component Outline – An XML description of a system, including the operating system it is built from and any other packages to be included.
• Image – A binary form of the system template that can be run in a specific virtualization or cloud provider. We take the system template and run it through an installation process to create a virtual machine image for the correct provider (qcow, ami, vmdk).
• Application Blueprint – Once we have images based on system templates built we can create an application blueprint which references one or many images. We can also add services (see below) and apply hardware profiles to the images.
• Services – These are actions we would like to execute on the images when they are launched. They are included in the application blueprint and can take parameters as input.

So, what was needed to build an Open Hybrid PaaS using Aeolus, Katello, and OpenShift Origin? Here we go:

Step 1, Mirror content in Katello and create System Templates
Katello is here to help you take control of your software and your systems in an easy-to-use and scalable manner. Offering a modern web user interface and API, Katello can pull content from remote repositories into isolated environments, make subscriptions management easier and provide provisioning at scale [2]. Basically, its really good systems life-cycle management.

Using Katello, I mirrored the following repositories into my environment.

From Red Hat’s Content Delivery Network

• Red Hat CloudForms Tools for RHEL 6 RPMs x86_64 6Server
• Red Hat Enterprise Linux 6 Server – Optional RPMs x86_64 6Server
• Red Hat Enterprise Linux 6 Server RPMs x86_64 6Server
• Red Hat CloudForms Cloud Engine Beta
• Red Hat CloudForms System Engine Beta

From outside Red Hat

• Extra Packages for Enterprise Linux (EPEL)
• OpenShift Origin
  

  Synchronizing Content

I created two system templates, one for the OpenShift Broker and one for the OpenShift Node. The OpenShift Broker had the following packages:

• mcollective
• mcollective-qpid-plugin
• mongodb
• qpid-cpp-server
• rhc
• rubygem-gearchanger-mcollective-plugin
• rubygem-swingshift-mongo-plugin
• rubygem-trollop
• rubygem-uplift-bind-plugin
• stickshift-broker

OpenShift Broker Template

The OpenShift Node included the following packages:

• cartridge-cron-1.4
• cartridge-nodejs-0.6
• cartridge-php-5.3
• mcollective
• mcollective-client
• mcollective-qpid-plugin
• mongodb
• rubygem-passenger-native
• rubygem-stickshift-node
• stickshift-mcollective-agent

OpenShift Node Template

Step 2, Build provider specific images with Aeolus
Aeolus is a single, consistent set of tools to build and manage organized groups of virtual machines across clouds [3]. Within Aeolus I configured both a Red Hat Enterprise Virtualization Provider as well as VMware vSphere cloud resource provider.

Cloud Resource Providers in Aeolus

It should be noted that I did not have access to delegate my DNS zone with our corporate IT department, so I created a private non-routable network for my virtual machines where I controlled both DNS and DHCP. This is where everything OpenShift Origin existed.

After setting up the cloud resource providers I imported my OpenShift Node and OpenShift Broker system templates and built images for both RHEV and vSphere. The concept of having both descriptions of my builds and then provider specific images is important. It allows operations focused staff to control the content in the build more sustainably. As soon as I update my description Aeolus can take the description and manage the generation of the images for all providers.

Image Building for OpenShift Broker

Since my application blueprints reference images that are maintained by Aeolus I don’t impact my self-service users when making changes to my descriptions. Pretty snazzy.

OpenShift Broker and Node in the Self-Service Catalog

Step 3, Deploy an OpenShift Broker and OpenShift Node, Automate, Test
Once the images were built I created a simple application blueprint which contained the images. I deployed the application blueprint and then began walking through the steps on the Build Your Own wiki page and translating them into a script that could be included as part of a service in the blueprints.

OpenShift Origin Brokers and Nodes in vSphere

It took a lot of tweaking, deploying, testing, and tweaking some more to get everything working properly, but in the end I have application blueprints which I can deploy via Aeolus to VMware vSphere which will provide me with a working OpenShift Origin environment.

Finalzing Parameters for automated launch

Once I had a working environment I could use the OpenShift origin tools (rhc) to create a new domain and then create an application.

rhc Domain Setup

rhc Creating an Application

Voila! My application is running. I can now make changes and git push to update my application.

OpenShift on vSphere

OpenShift Application on vSphere

Step 4, Deploy to a different provider (RHEV)
I had tested, tested, and tested some more across a single provider. Once I was able to deploy an OpenShift Broker and OpenShift Node and was able to run the rhc toolset to create an application I considered my deployment a success. Next it was time to take my deployment from VMware to RHEV. Since Aeolus abstracts the differences between VMware and RHEV, I should be able to deploy the same application blueprint, changing the cloud resource cluster from VMware to RHEV and have the OpenShift Origin PaaS running on RHEV.

It worked!

OpenShift Origin on RHEV

Now I can easily move my PaaS between virtualization providers. Think of the negotiating power I’ll have come renewal time! 😉 I could also continue to deploy the OpenShift Node Application Blueprint to scale my OpenShift Origin installation at the node layer.

Self-Service OpenShift Origin across two virtualization provider

Step 5, Share (of course)
I started a github repository which includes the application blueprints along with instructions on how you can use them. Contributions are welcome and appreciated. I plan on keeping all the work I do around application blueprints, services, and scripts in that repository.

Summary
The combination of Aeolus, Katello, and OpenShift Origin have the potential to let organizations realize an Open Hybrid Cloud. Katello provides re-usability of system templates, ongoing management of deployed systems (updates), and will soon allow for physical system provisioning and configuration management via puppet and foreman. Aeolus gives organizations a way to take those re-usable system templates and create rich application blueprints that reside in a catalog and can be provisioned to multiple providers through a single self-service portal. OpenShift Origin is an amazingly elegant open source PaaS which is built on the proven Red Hat Enterprise Linux operating system. By using these technologies together organization can realize unsurpassed flexibility, agility, and sustainability.

After completing this exercise there are three areas that I believe should be examined to help with further integration:

1. How can OpenShift Origin call an external system to request more nodes?
2. How can I launch an application blueprint in Aeolus from an external system?
3. What orchestrates the scaling? Ideally it should be a system that understands scalability at all levels (physical server, virtual machines/nodes, and gears)? Aeolus and Katello don’t understand scaling gears, OpenShift Origin doesn’t understand scaling physical or virtual machines/nodes.
4. How can services, images, and application blueprints be made easier to share between instances of Aeolus and Katello?

References
[1] https://openshift.redhat.com/community/wiki/architecture-overview
[2]  http://www.katello.org/
[3] http://aeolusproject.org/about.html
[4] https://github.com/jameslabocki/RedHat
[5] https://openshift.redhat.com/community/wiki/build-your-own