Threat intelligence: Sharing is caring

This post appeared originally in our sysadvent series and has been moved here following the discontinuation of the sysadvent microsite

In today’s threat landscape, with botnets, ransomware, and unpatched and unprotected IoT toasters and garbage bins in every home, the ability to quickly and easily identify suspicious activities and artifacts is probably more important than ever before. If your company’s web server is being probed from a particularly malicious IP network, you might want some alarms to go off; if a zip file known to contain a variant of a cryptovirus crosses your network perimeters, someone should be told; and if a workstation in your office is communicating with a known Command and Control (C&C) IP address, someone will need to take a look at that.

This is where threat intelligence enters the scene. TechTarget describes threat intelligence as “organized, analyzed and refined information about potential or current attacks that threaten an organization”. In other words, any indicator that, in your context, can be useful in preventing or identifying an attack or other unwanted activity may be considered threat intelligence. These indicators are often found and disclosed by an organization that has been attacked, by security companies on a malware hunt or after having assisted in cleaning up after a breach or an attack, or they can be found from spam traps or from variants of honeypots. Just like someone will always be exposed to new viruses before antivirus signatures can be constructed, someone who gets burned will still publish what they know so that others might protect themselves.

In threat intelligence these indicators are commonly referred to as “Indicators of Compromise” (IoC). The term hints that there’s a confirmed knowledge of malicious behaviour associated with the indicator. The term is slightly diluted, since e.g. a confirmed phisher’s originating IP address or malware captured in a honeypot don’t necessarily stem from compromises per se. As long as there’s some kind of classification for the IoC, the name in itself shouldn’t matter much.

Some IoC types

There are many different types of IoCs. Mentioned below are some of the different types and what they may be used for. Remember, the “intelligence” part of threat intelligence is knowing how and where to use the IoCs in a manner that benefits your organization.

IPv4 and IPv6 addresses

Depending on how they are classified, you could add IP addresses associated with known malicious activity to either ingress or egress block lists in firewalls and other filtering network components.

If related to spam or phishing, rejecting email from those sources could be worthwhile.

If you don’t want to outright block IP addresses, IP addresses could be added to IP sets in an IDS and you could get an alert if your network sees activity to or from any of these addresses.

Domains

Domain type IoCs can be used in several contexts. Domains associated with malicious activity, can be added to an RPZ in your company’s DNS resolvers. If someone accidentally launches some malware, or if a vulnerable web server is coaxed into downloading a root kit, your DNS can block the attempt and even raise an alarm.

Another approach is blocking domains in a forwarding web proxy, e.g. Squid. Contributing software like SquidGuard or e2guardian makes it easier to manage blacklists and whitelists in Squid.

A mail server can either outright reject mail from such domains, or the domains can be used for spam/phishing scoring. A significant domain attribute is the domain’s age, where very fresh domains are often used by spammers and phishers. Spam Eating Monkey offers multiple RBLs for domain age lookup, and a recent blog post describes how to track fresh domains for non-RBL availability.

URLs

URL indicators often point to compromised components on a web site, like unpatched WordPress plugins or some long forgotten exploited PHP file. Sites like VirusTotal track (through various software security vendors) harmful URLs. Static lists of URLs can be used by spam/virus software as well as IDSes.

A company running web proxies for outbound traffic may add such URLs to filtering blacklists.

File hashes

Malware binaries or libraries as well as simple downloader shell scripts can easily be quite uniquely identified through hashing. “Easy” hashes like MD5 can be quickly computed but with a lower confidence of uniqueness, while more complicated algorithms like SHA1 or SHA256 take more computational power but reduces the chances of two or more files end up with the same resulting hash.

Hashes are usually available from threat intelligence platforms, in which case they can be downloaded and fed into various pieces of software. An IDS can often extract hashes from files it observes in network traffic, and create appropriate alerts when it finds a match.

In Amavis, mail attachment hashes can be checked against VirusTotal with the AmavisVT plugin. The plugin calculates hashes of various attachments, and checks the hashes against VirusTotal’s HTTPS API. If more than a predefined number of VirusTotal’s registered malware/virus detection engines (default is 5) identifies the hash as malicious, Amavis will react accordingly.

Email addresses

Email addresses associated with undesired activities can be blacklisted in a mail server. As mentioned in an earlier sysadvent article, recipient addresses for testing exploitable mail relays should be on the watch or block list.

Getting your feet wet

An extensive collection of threat intelligence sources, as well as threat intelligence platforms, can be found at https://github.com/hslatman/awesome-threat-intelligence. Most sources are available to anyone, some after registration, so it will be up to you to organize them and use the information in a way that makes sense to you.

Note that the compilation also includes known benign resources, which should always be factored in when processing threat intelligence data. You should also maintain your own whitelist as part of the sanitation process. For instance, whitelisting can prevents certain domains from being considered harmful even if accidentally included in other feeds.

A freely available entry level threat exchange platform is the Open Threat Exchange (OTX). Since the platform is open to anyone and without editorial control, the responsibility for sanitizing the data is certainly on the consumer (i.e. you). OTX provides turnkey APIs for various security software, as well as Java, Python, and GoLang libraries for easy integration into your own projects.