I think we can all agree that information security problems are getting worse over time, not better. Both the frequency and the severity of incidents are increasing, and the situation is deteriorating across the spectrum, from data breaches at large retailers to sophisticated campaigns subverting national telecom infrastructure.
It's not simply a matter of trying harder. These failures are not due to incompetence, and despite what high-profile security vendors might say, these problems are not primarily due to users ignoring alerts generated by existing systems. I believe that the approaches we rely on today are inherently incomplete and fragile, and a profoundly new way of looking at things is needed. Endpoint modeling represents exactly such a promising new approach to IT security.
There are other well-understood areas that show the effectiveness of modeling as a technique. Consider insurance. How would, say, auto insurance work without models? By maintaining models of drivers, which incorporate dimensions like age, gender, location, daily driving distance, recent driving history, and type of car, underwriters can price insurance rationally.
Insurance is one example of what modeling enables. How would Web advertising, video games, investing and trading, and credit card fraud detection work without models? Observable Networks has brought this approach to bear on IT security.
Introducing endpoint modeling
Endpoint modeling is a way to maintain a software-based model, a kind of simulation, of each device that is active on your network; there's a one-to-one mapping between devices on your network and their corresponding endpoint models. Observable Networks automatically discovers the devices on your network, then identifies and tracks the role of each device through time. When a device changes its role -- say, when a domain controller runs a telnet client or when a medical imaging device participates in off-site Web traffic -- endpoint modeling can ask: Would this role transition be of interest to a security analyst?
Importantly, we do this without installing agents on end hosts, and we do it without deep-packet inspection. How so? We instrument the network, collecting metadata from switches, and we watch the network traffic generated by each device, both to and from the Internet and between local devices. This metadata is compressed, encrypted, and pulled back to our cloud-based modeling and reporting infrastructure. Wth endpoint modeling, Observable Networks tracks the role of each device on your network through passive observation of network metadata, and it does so with a cloud-delivered service.
Endpoint modeling makes some formerly impossible things very easy. You can know when a device accesses the Internet for the first time. You can know when a domain controller makes use of a Google Form. You can know when a local printer becomes a Web server, serving remote clients.
Of course, there have been many attempts over the years to provide anomaly detection based on network logs of various kinds. In the 1990s, people used terms such as “network behavioral analysis” to describe this. Today, phrases like "analytics," "big data analytics," and "machine learning" are commonly used. For better or worse, these methods are more or less the same, and they are all difficult to use for the same reason: Ultimately, they all apply statistical methods to log data. These approaches, when applied on any reasonable scale, will lead to insurmountable problems with noise and false positives.
]How is endpoint modeling different? It uses the same type of network and endpoint log data not to draw statistical inferences, but as an input to modeling and simulation.
To illustrate the key difference, let’s consider a simple example. Suppose a multifunction printer has been on your network for many months, and in all of that time has communicated only with other local devices. Suppose further that one morning, for the first time ever, it interacts with a remote host on the Internet, sending and receiving a few packets totaling no more than a few kilobytes of payload data.
Because modern multifunction printers feature latent services like Web, FTP, and SMTP, all with default passwords, this activity would be of interest to a competent security or IT analyst who wants to ensure that the printed or scanned materials are not exfiltrated. Endpoint modeling detects this role change trivially; this long-present printer has never before interacted with a remote host. Log analysis approaches would somehow need to detect those packets as being anomalous, but recall that these packets are anomalous only because they come from a device that has been on the network for a long time and does not usually interact with the Internet.
In other words, the anomaly can only be seen when this network behavior is contrasted with this device’s typical behavior over a reasonably long time period. Endpoint modeling maintains the device-specific state and context to recognize precisely these types of role changes. If a statistical analysis does not incorporate device-specific modeling, this type of behavior change will never be surfaced.
Of course, this printer example is simple, but if you think through how, say, your own laptop interacts with other devices and device types on your corporate network, you can begin to see how endpoint modeling might easily detect when, for example, your laptop suddenly starts interacting with database servers or building automation systems. (See my blog post if you’re interested in a closer look at the differences between endpoint modeling and log analysis.)
Advantages of endpoint modeling
Endpoint modeling is not only different, but profoundly different. In an automated fashion, Observable Networks maintains a device-specific software model of each endpoint in your environment and tracks its role, which includes how it uses the network and who it connects with, among other factors. What are the advantages of endpoint modeling compared to traditional security approaches?
I can describe two types of new, significant value that endpoint modeling brings beyond traditional network security approaches. First, endpoint modeling can detect risky and suspicious traffic that cannot be seen in your environment today. This is because the service monitors internal, local-to-local traffic that is typically not actively secured, and because any alert of the form “this device has done x for the first time ever” can only be achieved with some form of endpoint modeling.
Second, and I think more important, endpoint modeling finally brings a durable information security advantage to the defender. You are likely aware of the asymmetry that prevails in information security, in which the bad guys always have the upper hand because they have built-in advantages, such as the ability to verify that their new malware is not detected by existing AV and IDS systems before deploying into your environment. Endpoint modeling turns the tables, creating an asymmetry in your favor. No internal or external adversary has the unobstructed, 24/7 view of your network and its devices. This is precisely the advantage that endpoint modeling establishes and maintains.
The unique mix of devices and networks in your environment all of a sudden become an advantage for you. All those quirks in your network and your infrastructure that make it difficult to deploy off-the-shelf tools or to integrate networks following a merger or an acquisition -- those are simply side effects of how unique your environment is. Endpoint modeling turns this chronic frustration into a sustainable advantage. Endpoint modeling comes to learn what is typical in your environment, so the more unique and distinctive your environment is, the better.
Further, unlike so many other security mechanisms, endpoint modeling gets better and appreciates in value over time. This is true even in the face of technology trends that are rapidly eroding the effectiveness of traditional methods of information security. Let’s consider three technology trends and their impact on current best practices.
Trend No. 1: The limitations of signature-based malware detection. Most of us can clearly see security mechanisms that rely on malware signatures, whether they are in the network or on the endpoint, have become severely limited in effectiveness. Perhaps most notably, malware authors rely on tools and techniques that ensure malicious payloads are one of a kind. This is why we see major vendors publicly stepping back from traditionally lucrative product lines.
Put another way, traditional signature-matching security mechanisms depend on enumerating and documenting the possible signatures; this is no longer possible for a growing fraction of threats. By contrast, endpoint modeling derives its effectiveness from changes in well-understood device behaviors based on past behavior. With endpoint modeling, we do not need to anticipate the shape a threat may take; we simply need to spot the resulting change in behavior.
Trend No. 2: The rising use of genuine end-to-end encryption. You don't need me to tell you that the world has become far more privacy conscious in the past year. Individuals and companies alike are moving aggressively toward properly encrypted end-to-end communications. I ask you: What happens to your SSL proxies, your next-gen firewalls, your payload detonation systems, your IDS, and your full-packet capture systems when the network traffic you care most about is properly encrypted end-to-end? In my view, end-to-end encryption turns the lights off for network security analysts. This is why endpoint modeling is driven by network metadata and why it does not rely on deep-packet inspection.
Trend No. 3: Device proliferation. Most of us are living through this right now. The BYOD trend means that people are bringing personal handsets and tablets onto the network. More significantly, all forms of modern electronics need to connect to the network in order to function properly -- thermostats, appliances, TVs, webcams, security systems, and building automation systems, to name a few. All of these devices have embedded computers inside, and very few of them accept end-host agent software or participate in multifactor authentication schemes. This is why endpoint modeling is based on network traffic rather than end-host agents.
These three trends -- the limitations of signatures, the rise of end-to-end encryption, and device proliferation -- substantially reduce the effectiveness of enterprise security. There are other trends that further erode current best practices, but I’ll leave it at these for today. These trends motivated us to find a new way to look at IT security, and endpoint modeling was the conclusion that we came to.
Implementing endpoint modeling
What might it take to achieve endpoint modeling in your environment? At Observable, we've worked hard to make it easy to get started and to get our costs down to support what we consider to be a disruptive price point. But first, it’s reasonable to ask: Are you already realizing the benefits of endpoint modeling in your environment?
If you have already deployed SIEMs or log management solutions, you may well wonder whether you are already getting the benefits of endpoint modeling. However, while it is possible to build an endpoint modeling solution on top of a SIEM, I don't think anyone has done it yet.
There are three limitations. First, such systems are limited by the logs that they receive; not all data is available in log form. Second, I mentioned earlier that Observable Networks’ solution is cloud-based; that’s because endpoint modeling is both compute and storage intensive, and cloud resources make that sustainable and cost-effective. Third, for most organizations, SIEMs and log management systems are a central repository of a growing volume of enterprise security and operations data. And while centralization and a “single pane of glass” can help with making information discoverable, they fall far short of yielding the benefits of endpoint modeling.
In other words, SIEMs make it possible for a growing volume of machine-generated logs to be funneled toward a finite number of humans for processing, but it falls to human analysts to keep up with this growing volume of data. That simply does not work today, and the trends are strong enough that in a couple of years people will stop pretending it works. SIEMs and log management systems work best when they include indexable, high-value intelligence and concise alerts.
In fact, most of our customers use our endpoint modeling service to produce precisely these artifacts for their SIEMs. We have a uniquely concise approach to generating alerts, based on typed data objects we call observations, that permits thousands or tens of thousands of individual network facts to provide supporting evidence for a single alert.