The security of machine learning

Artificial intelligence and machine learning are persistently in the headlines with rich debate over its next advances. Will cybercriminals further leverage machine learning to craft attacks? Can defenders build a machine learning model capable of detecting all malware?

We believe machine learning is an essential and critical piece of cybersecurity, but it must be only one part of a broader solution to be effective.

It’s unwise for any security product to rely solely on machine learning as its primary or singular layer of defense. An all-eggs-in-one-basket approach leaves attackers with a single door to break down. A product with a true multi-technology approach, such as Sophos Intercept X, presents a complementary and reinforcing set of obstacles that must all be overcome at the same time for an attack to succeed.

Machine learning is one of an ensemble of protection technologies in Intercept X designed to identify malware and potentially unwanted applications, including those that have never been seen before. We go further in our application of machine learning, beyond simply making predictions on files. We also include “advisors” in our Endpoint Detection and Response product to provide additional information to aid in decision making. These advisors have the effect of providing the intuitions of expert malware analysts to the handling of suspicious (as compared to categorically malicious) events or artifacts.

Machine learning: A target

Cybercriminals have always sought out new and easy ways to break into systems and maneuver around networks. It is reasonable to assume they would look for ways to target a machine learning model and trick it into thinking an attack is “safe”. Sophos has prepared for such an event and other types of potential and evolving attack techniques.

To illustrate, the video below from our Chief Scientist, Josh Saxe, explores the ways in which machine learning models can be manipulated.

Our strategy to remain resilient to these attacks has been to conduct diligent industry-leading research into neural networks and their architecture, as well as to consolidate next-generation and traditional security technologies into a single solution. This includes layers of analytics, behavioral detections, static detections, heuristics, machine learning models, anti-exploitation techniques, anti-ransomware technology, and more. We call this “The Power of the Plus”.

Only through careful architecture alongside independent, agnostic, and complementary protection technologies can the power of machine learning be safely utilized.

Sophos Offensive Research

Sophos has been conducting offensive security research of machine learning models for over two and a half years as part of our internal research into the security of machine learning and improving product resilience.

As far back as February 2017, we built an advanced, product-agnostic proof-of-concept that could trick most, if not all, machine learning models by mutating a known-good file into a malicious file. The vast majority of models would believe the file was the original, known-good file. Even today, this proof-of-concept deceives the majority of machine learning models in endpoint security solutions. Note: We chose against making this research widely available to prevent it getting into adversaries’ hands.

By preempting our adversaries and understanding the approaches they could take, we underscore the importance of our strategy to provide multiple defensive technologies covering a broad spectrum of techniques and capabilities.

Cybersecurity by Sophos

The SophosLabs Data Science team are major contributors to the field of artificial intelligence research, as evidenced by the technical papers on our website or listings on Google Scholar. Extensive and continuous research and publication of our defensive innovations is a pillar of our ethos. The results of this research influences and determines the composition and orchestration of our layers of protection technologies in Intercept X and all other Sophos solutions.

Our research influences ongoing optimizations in our products to make them less susceptible to attempted adversarial attacks. In addition, our layers of defense are designed to make convictions independently so that, should one layer miss a threat, another layer stands strong to protect the system.

Digital security and physical security have many parallels. Think of a building and how it could be protected. If you were to build nothing but a giant wall, it may prove difficult to climb over but eventually someone will find a way to get over it (or under it).

Now consider a fortress. Armed guards, attack dogs, CCTV, tripwires, barbed wire, motion sensors. It may be possible to hop the wall, but you still have many additional hurdles ahead of you.

Single layers are simple to build but are also simple to bypass. Our goal has always been to build fortresses.