Changing Security Rules in Real Time with Machine Learning

Visibility - or lack thereof - is a common complaint across all operational and development domains today. Developers lament the lack of visibility into the performance of apps once deployed into the nether that is "production" - whether on-premises or in a public cloud. Operations of all ilk - from infrastructure to network to security - bemoan the inability to peer into the cracks of application traffic and leech from its flow signs of an attack.

Visibility, as it is used in security, is necessarily achieved by the deployment of network and application services that are logically in front of the application. This practice has arisen for many reasons, but the truth is that an application lacks the context necessary to recognize many forms of attack. This is because the application stacks restrict developers to information available on a per-connection basis. This is a security measure in and of itself, as the isolation prevents a bad actor on one connection from having access to a good actor on another connection.

But this restriction limits the ability of the application - and its platform - from understanding connections and application traffic in context. By moving that responsibility upstream - in front of - the application, network and application services have the opportunity to evaluate each connection and its payload in the context of all the connections and payloads exchanged with that application.

This is important to recognize when we look at how to address the growing threat of application layer DoS attacks. 

It is disconcerting to note that these application layer attacks are increasing in volume and frequency. It is even more distressing to know that 43% of application layer DoS attacks are able to evade signature, reputation-based detection. Static signatures are quickly made obsolete by the rapid evolution of bad actors' ability to customize and target specific applications.

Static signatures are like flu vaccines. Flu vaccines are developed to protect against a particular strain of the flu, based on prior experience with the virus. It is essentially a biological signature that identifies a specific virus. It is not able to identify a new strain of the same virus, leaving us vulnerable to suffering through yet another bout of flu.

Bad actors are as adept as a flu virus at adapting and evolving to avoid detection and elimination. To meet this challenge, we need to be able to identify bad actors based on their behavior rather than a specific signature. To do that, we need visibility and the context it can give us. 

L7 Behavioral DoS Protection

Essentially, L7 Behavioral DoS protection is a flu vaccine that's capable of detecting the virus responsible based on its behavior rather than its actual composition. This capability is especially important at layer 7 (application) because such attacks are not merely volumetric. Some are designed merely to tie up connections and drain the application's capacity to serve legitimate users, thereby denying them service. Others are subtler, attempting to craft requests such that the application crashes and causes an outage.

To detect the various strains of L7 DoS attacks requires the ability to evaluate individual requests against what is considered normal. But merely defining "normal" behavior is akin to crafting a signature. What is normal for a hyperscale, API-based application is not normal for a web-based application used by a single business unit or department.

Thus, determining "normal" is the first step. Behavioral DoS accomplishes this by collecting specific bits of information from every packet that passes through the system. These bits are taken from the network, transport, and application data. A baseline is established from things like normal header attributes or sizes, what is normal stress on the server, or normal response times and latency. Using a statistical model, the system calculates a threshold of normal for those values from thousands of observations each second.

When the application shows sign of stress with degraded performance or heavy load, it triggers the system to go into "attack mode". In this mode, several things happen: 

• Rate limiting begins while a dynamic signature is built based on anomaly discovery.
• Depending on your settings, the new custom signature can await your approval or can automatically activate mitigation.
• You can further customize how attack mitigation performs when traffic matches the new signature.
  • Conservative – block only known bad
  • Standard – block only known bad unless server stress is too high then rate limit all traffic during the attack
  • Aggressive – automatically rate limit all traffic as necessary to ensure good server health
• Once dynamic signature runs long enough to identify the source of the attack, mitigation shifts from rate limiting to layer 3 based blocking.

This process includes ensuring that there is less than 5% overlap of characteristics with 'good flows’ to prevent false negatives. Once application health returns to normal, the system can relax.

The advantage of L7 Behavioral DoS is that it focuses on constantly learning what is normal and identifying what is not. That allows it to detect L7 attacks that are new or evolved from earlier attacks that static signature-based systems won't notice until they are updated.

Behavioral analysis is possible thanks to very focused machine learning models. These models are enabling application protection to evolve in real-time, offering the agility the security domain so desperately needs to keep up today. Bad actors are constantly tweaking and modifying attacks to evade detection. To better protect applications, security services need to change their assumptions and operate in the context of the environment they are in. It's simply not reasonable to ask security professionals to craft thresholds for each and every application in the business' portfolio.

As the load on security and operations grow from the addition of new applications and environments, it is important to shift that burden from people to technology whenever possible. Behavioral DoS as part of an Advanced WAF is one of the ways to put machines to work for you, instead of against you.