Considerations for Leveraging Cyber Threat Feeds Effectively
My lessons learned and recommendations from multiple threat feed and Threat Intelligence Platform (TIP) assessments.
There are a few different approaches that I have seen organizations take when it comes to consuming threat intelligence. Teams that have been in the threat intel space for a long time probably started with informal sharing distros (We don’t talk about fight club!) with unorganized TXT files serving as their community standard. These teams likely evolved their sharing efforts into using CSVs, maybe even with some basic structure, before moving on to helping build out projects like STIX, openIOC, etc.. These analysts then started looking at open-source and commercial feeds with automation and integrations — or even starting their own Threat Intelligence Platform (TIP) companies.
On the other side of the spectrum, you have organizations building threat intelligence capabilities from scratch and attempting to go from maturity level 0 to 5 in a single project. These organizations usually get a Threat Intelligence Platform (TIP) with a flashy demo- maybe even a #pewpew map!- and they turn on every threat feed available. In a few short months, they blame their TIP for feeding them too many false positives and their threat feed providers for get tagged with “too little; too late”. Many of the problems these organizations face is that they have not gone through the crawl and walk phase of capability building and attempted to purchase their way straight to the run phase of operations.
The good news is that many of us have experienced the hard fought battles of building an intelligence capability and we are often more than willing to share our lessons-learned. Usually this exchange is over a beer at a post-event happy hour, but I’ve decided to share my experiences a little bit broader than my personal circle- just offer to grab a round if you find this helpful in your organization. This post will cover my thoughts on managing threat feeds within an organization. I plan to follow up later with my thoughts on Threat Intelligence Platforms (TIPs) and cyber threat information sharing standards.
Without further ado, here are my recommendations for implementing a basic cyber threat intelligence capability.
Threat Feed Evaluations
As with all projects, organizations must have a serious conversation about threat feed requirements, processes, and standards before turning on commercial and open source feeds. Once your team has decided to ingest feeds automatically, it is recommended that organizations only turn on 3–5 feeds at a time and run them for at least 60–90 days before turning on additional feeds. During this period, analysts and managers should be talking regularly about the amount of false positives in the evaluated feeds and the quality of the context in them.
When do you turn on feeds? If you have a security appliance that receives indicator and signature feeds directly into it, that is a separate discussion that must occur between threat intelligence leadership, detection and monitoring leaders, and security engineering. I generally recommend turning these feeds on and working with the vendor to tune out issues. However, I have heard some mature organizations argue that they only use their own custom signatures. There are pros and cons in this approach that must be discussed internally at each organization.
The absolute worst thing you can do to your organization is catch em all — this isn’t Pokemon and your analysts may quit when they end up with an 85% false positive rate with thousands of unreviewed alerts pending in their queue. (That 85% FP rate isn’t made up either; it is rounded from actual analysis on a previous project.) Remember, one of the goals of threat intelligence is to prevent alert fatigue- not cause it.
One method to evaluate how your feeds are impacting your organization is to assess the actions that they lead to for your team in a short 60–90 day project. For example, have an analyst review all of the hits (i.e. IOC in feed matches traffic in environment) and tag them as False Positive, True Positive Escalated, True Positive Mitigated. The definition of Escalated here is that the existing security stack did not block the event so action was required to scope and mitigate the activity. The definition of Mitigated here is that the existing security controls detected and mitigated the event. These metrics will help you better understand how much good work vs. busy work your feeds are causing within your environment.
A second method is to give your analysts a rating questionnaire and have them run evaluate the feeds on a 1–5 scale with criteria like the quick list below. Your team ranks each individual feed, then someone runs the averages to see how the feeds compare to each other.
A | Context
- Is there enough context around the IOCs to understand how they are used in an event/attack?
B | Timeliness
- Is there a big gap between Date Sighted and the day you receive them?
C | Ease of use
- Does the feed provide all malware hashes and your organization has to run these manually?
- Does the feed provide mostly IPs and your analysts have to run additional analysis against them before safely deploying?
- Is the feed auto-fed into your TIP or does your team have to manually download them from a site?
Threat Feed Considerations
I recommend staying away from threat feed vendors where the primary business line is IOC feeds. Traditionally, these companies focus too much on quantity over quality and you’ll end up with a giant dump of IPs with minimal context. The other challenge here is that they often deduplicate and anonymize their feeds before pushing them to you- you’ll see this same challenge with TIP vendor’s curated feeds. This is a huge hindrance to your analysts since they will not be able to tell if IOCs are being widely observed across multiple sectors or if other elements of context is being lost in the deduplicate process. Worse still, if you cannot do basic supply chain analysis of your IOCs, you may duplicate feeds on your end that your vendor is also pushing you. This costs you money in paying for feeds needlessly and it creates an echo effect for those IOCs. In a world where algorithms are now telling us what’s important, this could mislead your system to calculate higher confidence scores.
As an analyst, I have to say the most egregious crime in anonymized threat feeds is not being able to see the true source of the intel. How can analysts effectively assess their sources for confidence and accuracy if the data is coming from mixed sources that are also anonymized? Trust grows in the light.
When it comes to cyber threat indicators, context is king. If a vendor has a feed of indicators and it doesn’t include context for each indicator in a Description field, you are not buying indicators- you are buying Observables (more on that in a later article on cyber threat information sharing standards).
Each organization will set their own standards for their threat intel providers. Here are a few key aspects to consider:
1 | Is the feed provider the source of the intelligence or are they repacking IOCs from other sources?
- Original source is always preferred since the company will likely stand behind their analysis or be available for an RFI around their info
- Bundled feeds can make it harder to evaluate the quality of IOCs because of deduplication processes and anonymization (more on this later)
2 | Do the indicators include enough context to be quickly actionable?
- A Description like “This email delivered this URL which led to…”
- What type of malware is this malicious hash?
- What were the domains that were observed with these malicious IPs?
3 | Is the feed causing unacceptable false-positive rates for your organization?
- Each org has to determine “acceptable” rates for their teams
- Food for thought: one team I worked with in private sector once told me that they would turn off my organization’s threat feed after three strikes of wasted time- even though our feed was 100% free, it still could cost them money in wasted resources
4 | Is the feed providing too many IOCs that are mitigated by other security appliances?
- Example: the majority of IOC matched in your logs are all in blocked events because IPS detected it already
- Note: this is okay to have the intel still enter the TIP, but you may consider tuning them out from entering your SIEM.
5 | Does your feed include properly marked benign indicators?
- Example: malware calls out to Google DNS to see if it can reach the internet
- While 8.8.8.8 is a horrible IOC for detection, this benign IOC may be a critical piece of intelligence to have in the TIP
- Remember- IOCs!=malicious code (more on that in a later post)
Commercial, Community, and Open-source
What about paid feeds vs open-source? I believe that most organizations are best suited with a blended approach for their threat feeds. Commercial feeds can benefit an organization because they are often backed by intelligence teams that are available to discuss their analysis with your team when support is needed. Generally, these feeds are often more mature and will follow established intelligence standards in their analysis. Community feeds, such as Information Sharing & Analysis Centers (ISACs) can provide your organization with intelligence and IOCs directly relevant to your business sectors. They also provide a venue to discuss best practices for security of sector-specific issues. Open-source can be an invaluable resource for timely intelligence on emerging threats. Major campaigns often first emerge publicly on Twitter long before a blog or threat report is written. Organizations should invest the time to evaluate which open-source feeds, blogs, and personalities they should follow for the most up-to-date information available.
Commercial Vendors — I won’t pump one vendor over the other, but I will provide some basic advice here. As budgets allow, I recommend purchasing at least one premium intelligence service from vendors like CrowdStrike or FireEye- both if the budget allows. The quality of the context from these companies is unmatched and these companies do an amazing job enforcing traditional intelligence principles in their analysis and production cycles.
Other commercial vendors worth noting are companies like Flashpoint that conduct business, risk, and intelligence analysis for companies. These types of services often include active monitoring of closed forums and the dark web that can help companies identify data breaches.
Community based feeds — Since your team is looking to maximize their own value and the value of your feeds, the best bet is to focus efforts on collecting high context IOCs from established teams like sector appropriate ISACs and DHS’s Cyber Information Sharing & Collaboration Program Communities like ISACs and CISCP offer automated indicator sharing, as well as in person analyst exchanges where teams can learn from each other’s experiences.
Open-source feeds — More is not always best… So now you have an awesome threat feed or two coming in from a commercial vendor, you’re connected to a community channel or two, and you would like to bring in open-source feeds to round out your threat feeds. AlienVault’s Open Threat Exchange, Cymon.io, and Abuse.ch are all honorable mentions in the opens-source feed discussion. There are so many good options to consider that it is best to point at collections like the SANS’ Threat Feed Map and Herman Slatman’s Awesome Threat Intelligence page for full lists of amazing resources. (full links below for all references)
PRO TIP: if a new report comes out and it’s not in your threat feeds, AlienVault’s OTX will usually have a pulse of the IOCs within a few hours of publication. This is an awesome resource to use for large reports where the IOCS aren’t provided in a consumable format. OTX lets you choose the download format and I believe they also have a STIX field available for auto-ingest to your TIP.
There are numerous analysts in the field that Tweet out solid threat indicators and campaign analysis regularly. Some worth noting: @ItsReallyNick @DrunkBinary @cyb3rops @QW5kcmV3 (and probably a solid dozen more you’ll find from following these folks). You can run TweetDeck with keywords of interest so your analysts can monitor for breaking events too.
PRO TIP: TweetDeck also looks good on a SOC wall during those infamous tours, so bonus points for having something actually useful to show off on the tour.
Enrichment — Separate from your threat feeds, you should also invest in enrichment sources like paid VirusTotal accounts and DomainTools accounts for whois and pDNS information. These aren’t feeds but they will add more value to your threat feeds and TIP than any single feed ever will. For example, a good TIP has the ability for analysts to click a button to have the system call out to VirusTotal and pull in the SHAs, detection scores, and even detection names. The same can be done for domains and URLs to collect the whois and pDNS information. The amount of time saved for your analysts will likely cover the costs of the accounts.
In closing…
There is a lot I can say about this topic and I know that most of us have at least 20 Chrome tabs ready to launch at any moment when research is calling. My intent with this post was to pass on my thoughts for evaluating threat feeds and resources rather than simply pass on a list of resources.
If anyone has any questions, I am more than happy to discuss and can be found on Twitter @klrgrz and on LinkedIn at https://www.linkedin.com/in/andypiazza/.
Threat Intelligence Resources — in no particular order —
And of course… “Threat Intelligence and Me: A Book for Children and Analysts” (I literally have a copy on my desk to spark conversations with non-threat intel team coworkers) https://www.amazon.com/dp/1541148819/ref=cm_sw_r_tw_dp_U_x_3UiECbF4KTQC
