CERN Accelerating science

Rackspace joined CERN openlab as a contributor in July 2013 to tackle a challenge crucial for both teams: how to get compute clouds cooperating and sharing resources. Together with NASA, Rackspace was one of the companies who founded OpenStack in 2010, an open source project to deliver a massively scalable cloud operating system. OpenStack has now grown to have more than 1300 developers from 200 companies contributing millions of lines of code. This code is licensed under Apache open-source conditions, enabling flexible use by companies and researchers. OpenStack powers Rackspace’s public and private cloud solutions to deliver flexible and dynamic compute resources on demand.

OpenStack is also used extensively at CERN. The CERN IT department runs one OpenStack cloud while two others are hosted at the ATLAS and CMS experiments’ pits; these make use of the compute resources of the high-level trigger farms when the accelerator is not running. Other high-energy physics sites, such as IN2P3 in France and Brookhaven National Laboratory in the USA, also deploy OpenStack.

CERN and Rackspace had the objective of sharing resources between these CERN clouds, the partner labs’ ones and the public cloud resources at Rackspace’s UK-based data centre. In the same way as Twitter or Facebook accounts can be used to access other web sites, physicists need to be able to log in to clouds using their identity from their home institution. Marek Denis, a CERN fellow, started to work on the project in October 2013 to develop the required enhancements to OpenStack to meet these needs.

The OpenStack development process is tightly related to its biannual summits, where thousands of developers, solution providers and users gather to share experiences and to design the next release. At the Hong Kong summit in November 2013, Toby Owen from Rackspace and Tim Bell from CERN presented the CERN openlab hybrid cloud project. Its main objectives are to provide a reference architecture for the federation of OpenStack clouds, blueprints and code to the open-source communities, as well as documentation (presentations and white papers) to enable other members of the OpenStack community to build on the CERN openlab findings.

The design work started immediately with over 50 developers spending several hours debating in the federated identity design room at the summit. It was decided that initial efforts should focus on enabling the users of one cloud to be trusted by another cloud through identity federation. Following the OpenStack development process, a set of blueprints for how the design could be implemented was developed and assigned to Marek for implementation.

Meanwhile, the Rackspace technical support team had started to install the latest release of Rackspace’s private cloud in the CERN data centre. The CERN engineering team was, therefore, able to investigate new ways of deploying applications in a cloud environment thanks for instance to the software-defined networking feature of the latest version of OpenStack. In the future, this cloud will be used to test the interconnection between CERN’s private cloud and Rackspace’s public cloud. The federation code was included in the OpenStack ‘Icehouse’ open source release in April 2014. This will be deployed at CERN during October 2014 and demonstrated in hybrid clouds between CERN and Rackspace’s data centres at the OpenStack summit in Paris during November 2014.

Previous activities for the Project with Cotributor covering 2012 are available here.