Making Supervised Learning Work: Threat Stack’s Unique Data Labeling Experience For Security

Do you have those moments when suddenly the light goes on in your head, and you “get it”? Most of us do. Usually, it’s because I’m overthinking something, OR I’m not following a logic thread through to its conclusion.

Often, it happens when I’m discussing a topic I’m somewhat familiar with, with someone who is an expert, a “guru” (or at least a wise person) who has traveled farther down the path than I’ve yet wandered. One such experience happened to me recently, and my learning was deep.

The Flow of Cloud-Native Security

To set the stage, yes, it had to do with cloud security. Threat Stack, part of F5, is experienced in cloud-native security. F5 focuses primarily on securing applications and APIs, while Threat Stack supports by securing the cloud infrastructure that these applications run on. The concept of “Applications and APIs are only as Secure as the Infrastructure They Operate On” is part of the company’s DNA. The engineers and researchers at Threat Stack are constantly working on new and innovative solutions to make their clients’ infrastructure and systems safer and more secure.

A big part of Threat Stack’s cloud security process is to create and operate “rules”. Think of rules as an ever-expanding initial triage filter that takes literally billions of data points daily and sorts them, weeding out what you tell us is okay, then alerting you on what you tell us is “not okay”.

At Threat Stack, we started by creating rules to help look for and weed out what the rules say is okay, and then focus and alert on threats in cloud-native operations. The process also sorts, categorizes and classifies the data. As a result of this extensive library of rules, Threat Stack provided customers with known threats.

But we knew that wasn’t enough, so we added unsupervised learning. There, if something is not covered by rules, an anomaly detection alert is triggered.. It says: “You didn’t tell us about this event, so we’re alerting you to look at it.” It examines and infers the structure of a data set that you give it, making no judgement on “good” or bad”, but just focusing on: “it’s new and different.” It’s just finding anomalies, without human intervention

As you can imagine, triaging billions of data points still leaves organizations with thousands of anomalies and potential cloud security vulnerabilities to sift through. Is a particular event really worthy of a security team’s attention? That sifting can become very labor- and resource-intensive. DevSecOps professionals have even coined a term for having to analyze and deal with a large number of potential problems: “Alert Fatigue”.

Both solutions, even when combined, seek to reduce false positives. However, that also causes them to miss “normal” behaviors that still have extremely high risk. Threat Stack discovered that neither anomaly detection nor rules alone are enough.

Anomaly Detection Isn’t Quite Enough

Unsupervised learning, even when coupled with rules, still focuses on giving a result of anomaly detection. While unsupervised learning solves part of the problem, it ignores so-called “normal” behavior that contains risks and makes systems vulnerable.

The engineers at Threat Stack saw this problem and wondered: Is there something more? Something combining the best of all systems? They sought to answer the challenge: How do you identify threats when behavior looks normal, but is actually malicious? The final step is Supervised Machine Learning. While supervised learning (SL) is being used – slightly – in cloud-native security, the problem is that SL cannot label, group, or classify data. As a result, it hasn’t yet reached its full potential in providing cloud-native security … yet.

Deep Learning About … Deep Learning

That idea of supervised learning, or “deep learning” was my understanding of how it worked – at least until today. And here’s where my “aha” moment happened, as I was talking to Chris Ford, RVP of Product and Engineering at Threat Stack.

The Threat Stack engineering team – as always – works on figuring out how to make Threat Stack security even more powerful. It is not enough to weed out a few potential threats or false positives. Chris and the rest of the team knew that there was more potential, more opportunities, more growth in the cloud-native security field.

Chris pointed out that, at first glance, supervised learning seems to have a drawback. It doesn’t classify, organize, label, or group data. It is defined – and limited — by its use of labeled datasets to train algorithms that classify data or predict outcomes accurately.

Unlike unsupervised learning models, supervised learning cannot cluster or classify data on its own. And for it to function well, for supervised learning to reach its full potential, to really advance deep learning in cloud-native security, it has to deal with data that IS organized and labeled.

If that’s the case, I wondered, then what good is supervised learning? It’s just sitting there, running unclassified data, crunching away, trying to make sense of billions of bits of unorganized chaos.

Why doesn’t everyone use supervised learning? Simple.
o SL requires lots of data (Threat Stack can check that box; it deals with more than 60 BILLION pieces of data daily!)
o SL requires LABELED data (Check that box, too. That’s what the rules do.)
o SL requires lots of LABELED data (Ditto: For nearly seven years, Threat Stack has been collecting, classifying and labeling data.)

Rules Feeds Data to Supervised Learning Aha!

That’s when my aha moment happened. Remember the rules that Threat Stack has been running for years? Each of those rules keeps getting added to, expanded, broadened and deepened. As a result, Threat Stack has one of the most comprehensive libraries of cloud-native security rules in the business.

Part of the rules data analysis process, Chris explained, is that, as those billions of bits of data run through the rules, the ever-growing, ever-focusing rules process labels, categorizes and classifies the data into neat, defined groups. Threat Stack has been doing that classification for years. As a result, it has a depth of data analysis and classification that is industry-leading in the cloud security world.

Out of Overwhelming Chaos: Data Crunching, Order, and Deep Supervised Learning

Threat Stack discovered an ideal way to uncover all relevant threats. They realized that a combination of intrusion detection techniques is required: This is “Detection-In-Depth.” Any intrusion detection technique on its own is necessary, but insufficient. Threat Stack is using supervised learning to do behavioral detection that can predict behaviors and deliver high-efficacy threat detection – which is a novel way to leverage supervised learning in cloud security.

Most important, now that Threat Stack’s ever-expanding rule sets have classified that data, and the classified data is labeled, supervised machine learning can do more with it. It can do more than merely highlight and alert about anomalies.

This supervised learning functionality can learn from the data – especially how it is classified, labeled, organized, and prioritized — to create high-efficacy alerts, with context, that represent real risk. It can learn and create models that do prediction. These high-efficacy alerts can be acted on immediately, for the protection and security of an organization’s customer, client and operational data and processes.

Not only is all of Threat Stack’s learning and data available out-of-the-box, ThreatML also offers customers the ability to have a security system that is highly-tuned to their environment, with little operational burden. In that way, Threat Stack customers can focus in on the models they want to study and learn about.

The promise of machine learning – especially supervised learning – is that it can reduce work, (specifically human toil,) increase operational efficiency, and be more focused and active on creating secure environments, by delivering high-efficacy alerts. The more supervised learning learns, the more tightly the rules become focused, and the more effective the alerts become. No more “alert fatigue”!

In this way, supervised learning leads cloud-native security to be a continuous process analysis / continuous process improvement function, which takes away operational burden (cost, overhead, personnel, resources, and time). And because Threat Stack created this solution to work across multiple platforms, while still being transparent, both Threat Stack and our customers can continue to learn and adapt. There is no hidden “black box.” Instead, Threat Stack shows its work. In fact, customers are encouraged to “look inside the box,” to see what is going on, why alerts are generated. In this way, customers can continue to adapt and improve their own security positions.

Customers Told Threat Stack About Their Security Needs

As always, the growth and evolution of Threat Stack is customer-centric. Supervised learning is no different. We heard customer pain points, especially around vulnerabilities and threat detection. The various solutions offered in the market fell into one of two camps: There was either too much information, too many alerts, OR there was an arbitrary limit on alerts, which meant that alerts and vulnerabilities and threats were being missed.

The solution? Create a system that would solve for both issues, yet NOT be labor intensive. The resulting “Detection-in-Depth” covers both the known universe of threats and vulnerabilities, and the unknown, yet-to-be-discovered (but predictable). Using both approaches, in concert, means that organizations find what they need to, yet the burden on cloud security teams and organizations is reduced.

Webinar About Supervised Machine Learning’s Expanding Role In Cloud-Native Security

To learn more about how Threat Stack’s Rules + Supervised Machine Learning represents a new step in threat detection and cloud-native infrastructure security, view a DataBreachToday.com webinar featuring Chris Ford: “Machine Learning Done Right: Secure App Infrastructure with High-Efficacy Alerts.”