Third day at the OpenStack design summit

At eNovance we have a lot of devs and engineers at the current OpenStack summit in Hong Kong and we are striving to share with you the discussion going on here, today we bring you the summit view from Fabien, the discussions he had with the other OpenStackers at the design summit.

My day was almost only focused on Swift related talks so let’s take a tour of that talks.

Profiling Swift

One of the first talks of the day was the one of Edward Zhang about the proposal of a tool that aim to profile a running Swift cluster. The tool is a middleware to insert in all server pipelines in order to collect profiling data. The middleware can be queried to retrieve profiling data. The profiling data can be really useful for developers as it include time spent in function as well as call frequency memory consumption, object throughput from region level to process level. This profiling middleware will be really useful to use with other tools such as ssbench or COSBench. The middleware can be already tested as the code is on gerrit.

https://etherpad.openstack.org/p/icehouse-swift-profiling-middleware-and-tools

Seagate Kinetic drive

After there was a talk from Seagate to introduce new revolutionary Kinetic drive that only handles key/value operations. Traditional SCSI commands are no longer used. This new disk provides Ethernet connector for operation calls and data I/O. The disk API provides classic key/value operations as well as security feature like authentication, client role authentication or even peer to peer replication between disk.

http://www.seagate.com/tech-insights/kinetic-vision-how-seagate-new-developer-tools-meets-the-needs-of-cloud-storage-platforms-master-ti/

https://github.com/Seagate/Kinetic-Preview

This kind of disk is really interesting when considering swift functionalities and design. So just after Clay Gerrard has introduce his work on DiskFile API and especially with Kinetic Disk support that can be found here:

https://github.com/swiftstack/kinetic-swift

Hot content in Swift

One of the main topic of the afternoon was how to handle hot content in Swift. A person from kt.com has explained how to handle this kind of problem. Hot content is a situation when a specific object is requested a huge amount of time in a limited time slice. Due to its design Swift doesn’t handle this efficiently. His proposal and what he uses actually is to move the hot content in the proxy Memcache when this situation is detected. This has some limitations so one of the Clay Gerrard’s proposals is to use Storage Policy to handle that in the future.

Etherpad : https://etherpad.openstack.org/p/swift-kt

Supporting a global online messaging service

The next talk was not typical but very interesting. It was about using Swift as a backend for a Messaging System such WhatsApp, WeChat, or LINE. So we’ve responded to the guy interrogations about the ability for Swift to handle that use case. As the etherpad is pretty empty in terms of responses I’ll explain some of them. About the Swift ability to manage 1 billion users. Swauth seems to fit correctly. Concerning the availability delay in a region of an object put in another region in the case of a multi-region Swift Cluster the availability delay will depends of the delay the proxy took to copy the replica to that region. In case of multi region separated Swift cluster a tool called SwiftSync can be use to replicate an account or part of it in a second cluster. Other questions and responses can be found on the etherpad.

Etherpad : https://etherpad.openstack.org/p/swift-for-messenger

Making Swift More Robust to Handling Failure

One of the last talks was about making Swift more robust in Icehouse by for instance sharing between processes (in memcache) devices that has been detected as failing. The full listing can be found on the etherpad.

Etherpad : https://etherpad.openstack.org/p/icehouse_swift_robust

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">

  1. Pingback: Third day at the OpenStack design summit | doc.cloudgear.io

  2. Pingback: OpenStack Community Weekly Newsletter (Nov 1 – Nov 15) » The OpenStack Blog