Index ¦ Archives ¦ Atom

Ceph for Cinder in TripleO

A wrap up on the status of TripleO's Cinder HA spec. First, a link to the cinder-ha blueprint, where you can find even more links, to the actual spec (under review) and the code changes (again, still under review). Intent of the blueprint is for the TripleO deployments to keep Cinder volumes available and Cinder operational in case of failures of any node.

This said, should $subject sound interesting to you, beware the code still needs reviews, probably polishing and surely more features ... so take this post as a call for help as well! Now some details.

For Cinder to remain operational, in TripleO we deploy the cinder-{api,schduler,volume} services on all controller nodes (as it was even before the proposed spec) but we also customize the host setting so that all instances of cinder-volume will share an identical identifier. This is so that all instances can perform operations on all volumes, otherwise, the particular host which created a volume would be the only one capable of performing further operations on it. This exposed some potential issues with Cinder due to concurrent mutation of same resource from multiple nodes and they already are taken care of, in Cinder, with the state-enforcer blueprint.

Then, for the volumes to remain available, TripleO will deploy a Ceph cluster together with the OpenStack components, to be used as a backend for Cinder. Each and every controller will host a Ceph monitor while an arbitrary number of additional nodes can be configured as Ceph OSDs. The basic idea is that by doing so people will be able to scale out the number of controllers and/or data nodes independently, in an attempt to suit the needs for more space or more CPU resources depending on the use case. In addition to that, network partitioning should also be easy to achieve as traffic from Nova nodes will be directed to the OSDs nodes only for volumes I/O, not to the controllers. Note though that the existing implementation does not yet implement support for the network partitioning but, on a good note, it does use cephx already to secure access to data.

As of today, assuming you checkout by yourself, as linked from the cinder-ha blueprint, the needed TripleO changes, everything should just work by setting some value > 0 for the CEPHSTORAGESCALE env variable, which represents the number of OSD nodes you want to deploy. Should you need it, in the Heat templates it is possible to customize a number of useful Ceph settings, like the number of replicas for the data (needs to be lower than the number of OSD nodes). Now go try it out and come back with some feedback! :)

© Giulio Fidente. Built using Pelican. Theme by Giulio Fidente on github. Member of the Internet Defense League.