Continuous Integration in the cloud

Continuous Integration in the cloud

In our intro on Agile Cloud Development, we articulated why we think this is the future of software development. Today we’ll kick off the Dev in the Cloud series with a look at Continuous Integration (CI) and the cloud’s impact on this popular agile development practice. We’ll explain how CI is the essential building block for Continuous Deployment, the secret sauce for start-ups from Facebook to Netflix. We’ll also highlight the marketplace for the growing number of CI products hosted in the cloud. While some vendors today are offering hosted solutions for just CI, others are including CI as part of their more comprehensive developer Platform as a Service (PaaS) that manages the entire build-test-deploy process.

What is Continuous Integration?

Continuos Integration is the practice of integrating software very often, typically on any change to an artifact in the source repository. Anytime a team members commits a change to the repository, continuous integration ensures their change was safe and the software can be successfully built, tested and deployed.

CI is the software industry’s version of Toyota’s Stop the Line principle, where work is stopped immediately if a quality problem is found. Because code integration is often a manual, error prone effort, especially on large systems or those that build infrequently, the traditional thinking has been to do it less often in order to avoid the cost of integrating (keep costs low). CI runs counter to this and adopts opposite philosophy, “if it’s hard to do, do it more often”. This means you should integrate more often, not less often. Taken to the extreme this means on on every change to the repository.

Doing this without automation would be prohibitively expensive for most teams. But by leveraging automation to detect changes in repositories and subsequently execute a script that downloads, compiles, packages, tests and deploys the software – the cost to complete this cycle dramatically drops and you get a wealth of benefits.

Continuous Integration is the key building block for Continuous Delivery, the practice of keeping your application in a state where it is always possible to deploy into production. This in turn is the building block for Continuous Deployment, the practice of deploying every change into production. These capabilities provide the business agility start-ups like Facebook, Netlflix and Groupon rely on to deploy changes to their products daily to continually improve their customer’s experience.

Benefits

The primary benefits of CI for organizations and teams include:

  • Improved Quality – Quality is primarily improved through reduced time to detect and resolve issues. When CI is in place teams find and fix issues quickly and generally “keep the build green” throughout development. When issues do occur, teams “stop the line” and address integration defects right away. Automation of most all key tasks to build and deploy helps reduce human error and improve repeatability.
  • Faster Release Cycles – Teams using CI can release code faster because the code always stays in a deployable state. Teams using Continuous Deployment take this concept to the extreme, deploying releases to production everyday by automating all tasks from source repository until running in production.
  • Improved Velocity – CI helps teams achieve a higher velocity (throughput) but automating the mundane development tasks so teams can focus more of their time on building new capabilities. When used in conjunction with automated regression testing, CI can also shorten the testing window at the end of each iteration.
  • Scalability – How do you get 20, 50 or 100+ agile developers all actively working on the same codebase? Without CI, this is very difficult because of the rapid pace of change to the codebase. With CI, your teams can scale as your business scales without sacrificing quality and speed.
  • Reduced Costs – This comes as a by-product of the benefits above. CI can lower the development and maintenance costs through automation and improved quality.
  • Improved Culture – Perhaps the biggest impact on agile teams is the improved culture of quality that’s instilled through CI. Quite simply, CI becomes the  heartbeat of development and a wonderful metaphor for the overall health of the project or product. Teams using CI take pride in consistently producing high quality software that always passes the tests and is ready for deployment.

In addition to these, CI in the cloud offers a few more:

  • Faster Startup – No more hassles with provisioning a new build server, downloading software, installing, configuring and managing your own CI environments. Not to mention patches,  upgrades and maintenance. All of these services can be provided by CI products designed to run in cloud either stand-alone or within broader development platform as a service tools.
  • Elasticity – Need more resources to build, test and deploy your applications faster? Why not leverage the cloud’s elastic benefits to provision these on-demand and greatly reduce the time to run large builds or regression test suites.

Cloud Computing Impact

Continuous Integration came from the XP community with Kent Beck and Martin Fowler as early advocates. Today it has grown into one of the most common development practices in agile development with a marketplace of tools for doing continuous integration with a variety of technology platforms. As development and deployment tools have moved toward the cloud, so have CI products.

While most CI products are still designed to download and run in locals environments, the cloud has created a new breed of products that provide CI services on hosted platforms that teams can began using quickly. Now you can avoid the hassles, costs and time of procuring a new environment, downloading, installing and configuring the CI software. Now it’s hosted in the cloud for you, just configure and go. Need more horsepower to speed up build times? Not a problem, some of these products can leverage the cloud’s elasticity to provision extra build resources on-demand and even parallel the build tasks.

Marketplace

The market for cloud-based CI solutions continues to grow as many established CI vendors now offer hosted versions of their products. Many of these started life as downloadable CI products but are now hosted in the cloud as a part of a larger integrated development Platform as a Service (dPaaS) solutions.

These solutions integrate all parts of software development and delivery including requirements and project management, source control, building, testing, release management, issue tracking and collaboration from a single cloud-based product. Larger dPaas solutions that include CI include Bamboo, CloudBees, CloudForge and Electric Commander. These solutions are focused on enterprise customers and their needs whereas focused CI solutions for the Ruby (tddium) and iOS (hostedci) markets are primarily focused on the developer community and their needs.

Popular Continuous Integration in the Cloud Tools
Vendor Product Strengths
 Atlassian  Bamboo Integration with JIRA
Integrated Development Platform as a Service
Elastic Agents for cloud builds
Automatic Branching and Merging
 Cloudbees, Inc  Cloudbees  Java Focus and Support
Integrated Development Platform as a Service
Built on open-source Jenkins
 CollabNet  CloudForge Enterprise Focus and Hybrid Cloud Support
Integrated Development Platform as a Service
Technology platform agnostic
Integration with popular ALM tools
 Electric Cloud  Electric Commander Development Process agnostic
Custom workflow design and visualization
Technology platform agnostic
Integration with popular ALM tools
hostedci hostedci  iOS / OSX / Mac Focus and Support
Built on open-source Jenkins
Solano Labs, inc. tddium  Ruby Focus and Support
Integrated Automated Testing

In addition to these hosted solutions above, downloadable CI products exist for private clouds or internal virtualized environments. Popular ones include JenkinsAnthill Pro, CruiseControl, goVisual Studio TFS 2010, TeamForgeBuildMaster, Mojo and Buildbot. For those teams not ready to move entirely to a hosted CI solution, many are still migrating internal build servers running CI to Amazon EC2 instances for improved management and reduced costs.

Resources

For those wanting to learn more about Continuous Integration, I recommend these resources:

Books Articles Related Blogs Twitter
Continuous Integration by Paul Duvall  Continuous Integration by Martin Fowler Continuous Delivery #ContinuousDelivery
Continuous Delivery by Jez Humble and David Farley  Continuous Delivery Guide by Martin Fowler Stelligent Blog 
 Continuous Delivery vs. Continuous Deployment by Jez Humble Netflix Tech Blog
 Continuous Delivery Cheat Sheet by Paul Duvall Build Doctor
dev2ops

Share this Article:

Share Button
Ryan Shriver (9 Posts)

Ryan Shriver is a Managing Consultant with Dominion Digital, an award-winning process and technology-consulting firm. Based in Richmond, VA he leads their Innovative Products solution that helps people deliver the right products at the right time, quickly with high quality. With services related to experience, value, speed and quality, he helps clients holistically focus on improving their customer's experience.

Working with Internet technologies since 1995 and practicing Agile since 2001, Ryan has deep experience in systems architecture and large-scale agile product development. He's presented internationally on topics including Scaling Agile, Measurable Business Value and Agile Engineering. He's founder and program chair of Innovate Virginia and posts is thoughts, articles and conference presentations to the agile engineer.

Connect with Ryan Shriver:


p5rn7vb

Related Posts:

3 comments for “Continuous Integration in the Cloud

  1. June 27, 2012 at 6:37 AM

    Hi Ryan,

    Just wanted to let you know that there’s another hosted CI service (my company) recently launched – Semaphore (https://semaphoreapp.com). It is made for Ruby developers who use GitHub.

  2. July 9, 2012 at 2:13 PM

    Hey Ryan,

    Another CI service to add to your list: https://circleci.com. We’re made for web apps, and automatically configure your projects.

Leave a Reply

Your email address will not be published. Required fields are marked *


− 3 = four