By Ryan Kirk, PhD., Principal Data Scientist, CenturyLink Cloud
A previous post discussed our approach to real-time anomaly detection. Recalling that for every unique stream of data, we predicted what the actual value should be. Then, if an actual value was different than our expectations, we flagged it as anomalous. Combining this anomaly detection approach with our real-time, faceted dashboard allows analysts to easily pinpoint areas of the cloud infrastructure that behave peculiarly.
This article expands on anomaly detection and dives into further detail on how to use data to detect and prevent customer-impacting events from taking place. Additionally, we’ll discuss how we further automate the analysts’ search process, determine the presence of customer impact, and integrate our predictions within the rest of the business to facilitate resolution. This post will cover:
- The similarities and differences in the lifecycle of data science and that of system architects
- How we detect the presence of unknown events
- How to determine whether an event is potentially impacting
- How to integrate this information into business’ operations
The Lifecycle of Data Science
There are at least as many definitions of data science as there are data scientists. At its core, data science requires both data and science. This means that these professionals are intimately familiar with extracting, cleaning, processing, and analyzing data....