View More
View Less
System Message
An unknown error has occurred and your request could not be completed. Please contact support.
Wait Listed
Personal Calendar
Conference Event
There aren't any available sessions at this time.
Conflict Found
This session is already scheduled at another time. Would you like to...
Please enter a maximum of {0} characters.
{0} remaining of {1} character maximum.
Please enter a maximum of {0} words.
{0} remaining of {1} word maximum.
must be 50 characters or less.
must be 40 characters or less.
Session summary
We were unable to load the map image.
This has not yet been assigned to a map.
Search Catalog
Replies ()
New Post
Microblog Thread
Post Reply
Your session timed out.
This web page is not optimized for viewing on a mobile device. Visit this site in a desktop browser to access the full set of features.
Red Hat Summit 2017

S104851 - On the path to building an event-monitoring data pipeline for storage microservices

Session Speakers
Session description

Over the past few years, telecommunications companies have invested significantly in large, multi-tenant cloud deployments and aspire to operate with high efficieny by running and orchestrating a suite of infrastructure microservices at scale. These cloud deployments and services are backed by systems that are typically shared by tenants and as these environments scale, the need to streamline operational control by managing these systems with a higher degree of predictability becomes imperative. In this talk, we will present a production, multi-tenant (containerized) compute and software-defined (Red Hat Ceph and Gluster) storage microservices architecture, discuss the typical workloads that are imposed on the system and examine key events that are generated by discrete components that comprise the overall system. Subsequently, we will propose a set of open-source technologies utilized to build a pipeline that encompasses collection of these events, appropriate storage of these events and a Kappa (Analytics) architecture that leverages an in-memory analytics engine - Apache Spark - to correlate these events in order to extract operational insight with predictability. As a result, we advocate that event data is voluminous, has numerous properties and, in some cases, may require a grouping of multiple events to create a more complex event. Furthermore, the characterization and evaluation of complex events to predictively recognize and avert potential issues is highly dependent on the temporal relevance of the event data of concern. We hope to demonstrate how simple machine learning algorithms may be applied to extract typical (affinity) and atypical (interference) patterns of resource behavior within the shared, distributed system and how building a parametric model with suitable weightings for different features may be useful in making these predictions with increased accuracy and in taking the appropriate remedial action.

Additional information
Cloud & hybrid IT, Big data
Breakout session
45 minutes
Session schedule