Automating Cyber Threat Intelligence with SkyATP: Part One

By cdods posted 10-17-2016 09:55


Part Two is now available here.


Each year, the economics of "fighting back" against Hacktivism, CyberCrime, and the occasional State-Sponsored attack become more and more untenable for the typical Enterprise. It's nearly impossible for the average Security Team to stay up to date with the latest emerging threats while also being tasked with their regular duties. Given the current economic climate, the luxury of having a dedicated team to perform Cyber Threat Intelligence (CTI) is generally out of reach for all but the largest of Enterprises. While automated identification, curation, and enforcement of CTI cannot truly replace human Security Analysts (yet), it has been shown to go a long way towards increasing the effectiveness and agility of your Security infrastructure. 


For anyone not immediately familiar with the concept of CTI, its goal is simple; To identify and, ideally, mitigate possible threats in your environment as early in the Cyber-Kill-Chain as possible. The methods for acquiring CTI can vary drastically, each with varying degrees of skill and effort required (Threat Feeds, Malware analysis, Social Media, IRC, DarkNet forums, Social Engineering, etc). Once acquired, the proverbial Security Analyst is then responsible for both converting the CTI into actionable intelligence and then converting those intelligence reports into tangible Firewall Policies, IPS signatures, Reputation Blacklists, SIEM rules, etc. The actual cost of prevention, whether it be man-hours to implement the required changes, performance limitations of the infrastructure, or the dynamic nature of the threat itself, simply exceeds the capacity for most organizations today.


This is where open and extensible Threat platforms like Juniper's SkyATP can assist. For those who weren't aware, SkyATP has the ability to provide advanced anti-malware protection via Static/Dynamic analysis (Sandboxing), while also being a fully-fledged threat-intelligence platform capable of storing up to 1,000,000 unique objects (IP/URL/Domain). We’ll go through a simple example in Part One, migrating to more complex and interesting use cases such as SIEM-generated Blacklists, STIX/TAXII/CyBox integration, and Twitter+DarkNet scraping with Python’s beautifulsoup library in future posts.


Use Case 1 - Identifying, Tracking, and Blocking traffic from Tor Exit Nodes 

In our first example, Security Analyst Steve Stevenburger has identified a possible Anonymous campaign targeting his corporation while monitoring #AnonOps. Anonymous has been instructed to perform targeted attacks against Web Servers in his DMZ while concealing their identities via Tor. Steve knows that Tor exit nodes move frequently, and that keeping an up-to-date list of all 1000+ exit nodes within his Firewall policy will be almost impossible. 


Thankfully for Steve, he knows a little bit about automation and has SkyATP enabled on his Perimeter SRX NGFW. 


He writes a quick script that:
a) Polls the official TorProject's exit-node list via cURL and extracts legitimate IP information via grep

b) Utilizes SkyATP's open API to install and propagate 3rd-partyThreat Intelligence to all his SRX

c) Runs on an hourly basis via Cron to ensure that the active Tor Relays are always being blocked



# Define Application Token (Paste in your value between the "")

# Define the name of the feed you wish to create

#Define temporary file to store address list

# cURL fetches Tor Relay list from
# grep identifies and extracts valid IP addresses from the list

curl -k | grep -E -o '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)' > $TorList

#Remove old Feed information before uploading new list
curl -k -v -XDELETE -H "Authorization: Bearer $APPToken" -F server='*'${FeedName}

# Wait for 5 seconds before uploading new list
sleep 5

#Upload List to SkyATP as Feed Tor_Exit_Nodes
curl -k -v -XPOST -H "Authorization: Bearer $APPToken" -F file=@${TorList}${FeedName}

# Cleanup
rm $TorList

# Exit

Once the script has been run successfully, we can see that the latest Tor Nodes are being blocked during an ICMP request below (feed-name=Tor_Exit_Nodes)

<14>1 2016-10-17T15:18:11.618Z SRX-1500 RT_SECINTEL - SECINTEL_ACTION_LOG [junos@2636. category="secintel" sub-category="Blacklist" action="BLOCK" action-detail="DROP" http-host="N/A" threat-severity="0" source-address="" source-port="1" destination-address="" destination-port="24039" protocol-id="1" application="N/A" nested-application="N/A" feed-name="Tor_Exit_Nodes" policy-name="cc_policy" profile-name="Blacklist" username="N/A" roles="N/A" session-id-32="572564" source-zone-name="Outside" destination-zone-name="DMZ"] category=secintel sub-category=Blacklist action=BLOCK action-detail=DROP http-host=N/A threat-severity=0 source-address= source-port=1 destination-address= destination-port=24039 protocol-id=1 application=N/A nested-application=N/A feed-name=Tor_Exit_Nodes policy-name=cc_policy profile-name=Blacklist username=N/A roles=N/A session-id-32=572564 source-zone-name=Outside destination-zone-name=DMZ


It's important to note that this example is only scratching the surface of what open source tools and SkyATP are able to accomplish.


I've placed some additional information and recommended reading below:

Validating your new feed has been populated. We can see that we have 1051 unique objects stored:

root@SRX-1500> show services security-intelligence category summary 

Category name     :Blacklist
  Status          :Enable               
  Description     :Blacklist data
  Update interval :3600s
  TTL             :3456000s
  Feed name       :Tor_Exit_Nodes
    Version       :20161017.0
    Objects number:1051
    Create time   :2016-10-17 14:06:04 UTC
    Update time   :2016-10-17 15:06:37 UTC
    Update status :Store succeeded
    Expired       :No
    Options       :N/A

Validating the actual contents of the feed:

root@SRX-1500> show security dynamic-address feed-name Tor_Exit_Nodes 

No.      IP-start        IP-end          Feed             Address           
1    Blacklist/1      ID-21300041      
2     Blacklist/1      ID-21300041      
3   Blacklist/1      ID-21300041      
4      Blacklist/1      ID-21300041      
1049  Blacklist/1      ID-21300041      
1050 Blacklist/1      ID-21300041      
1051 Blacklist/1      ID-21300041    


Recommended Reading:

  1. SkyATP's Threat Intelligence Open API Setup Guide
  2. SkyATP's Full Open API Guide
  3. Aviram's latest blog post about sharing Threat Intelligence
  4. SANS institute whitepaper: Who's Using Cyberthreat Intelligence and How?
  5. SkyATP Technical Documentation

An additional piece of information for those who aren't explicitly familiar with how Tor behaves: Identifying traffic that is sourced from Tor Relays is relatively difficult without an up-to-date list of their addressing. It's somewhat trivial to block egress Tor traffic from your network with DPI/AppID as the protocol has distinct characteristics. However, when the traffic leaves a relay destined to your network, like in the example above, the encapsulation that Tor adds is stripped and the traffic will no longer trigger "Tor Specific" signatures. 




10-19-2017 07:14

Great Article Craig !!!

10-18-2016 05:23

Great article Craig,


Keep them coming 🙂