Cloud Spotlight: Cloud Scanning
We’ll keep exploring scanning techniques, but now we'll look at how attackers scan cloud systems. Scanning cloud systems has its own challenges and chances. Attackers can’t easily scan many IP addresses in the cloud with usual methods. Even if they get scan results, they need to figure out which IP address is their target and not just another customer's.
Attackers often scan cloud assets because these networks are less monitored and logged. They might bypass firewalls if scanning from the same cloud provider. Cloud systems can also expose information about non-cloud infrastructure, increasing attack opportunities. This module covers techniques to scan large ranges of cloud IP addresses, which can also apply to large internal networks where tools like Nmap are too slow.
JQ and JSON Data
No matter your role, if you work with cloud systems, you'll use JSON. It's a common format for handling data in the cloud, used by AWS, Google Cloud, Azure, and others.
JSON is text-based and can be processed with tools like grep, cut, awk, and sed, but this can be tricky due to inconsistent line endings. A better option is to use JQ, a tool designed for handling JSON data.
JQ is a simple tool and programming language for handling JSON data, created by Stephen Dolan (find it at https://stedolan.github.io/jq/). It’s easy to start with and helps format complex JSON data for better readability.
Cloud Scanning Process
Before diving into cloud scanning, note that it's just one way attackers find cloud assets. They also use reconnaissance and OSINT to discover a target's cloud provider, checking DNS info, website content, and public sites that reveal hosting details. The BuiltWith site helps by showing tech trends and cloud providers for any domain. For example, searching "sans.org" on BuiltWith reveals that it uses multiple cloud providers, including Amazon AWS Route 53.
An attacker that uses reconnaissance and OSINT resources can determine which cloud providers are in use at a target organization. Once the attacker identifies the cloud providers, they can dive into a deeper scan to identify even unlinked systems.
Cloud Provider IP Address Enumeration
Scanning involves checking a set of IP addresses or host names. In cloud scanning, attackers first find the cloud provider, then determine the associated IP addresses to scan.
Cloud providers offer IP address lists in JSON format, which companies use for network security and responding to attacks in their continuous integration (CI) build process.
Let's see how we can retrieve cloud provider IP addresses using the hosted lists using the Wget and JQ tools (note that for Azure, we must download the IP address list manually from https://www.microsoft.com/en-us/download/details.aspx?id=56519 then parse the extracted file with JQ). The short JQ programs will extract only the IP address and Classless Inter-Domain Routing (CIDR) mask information:
Google Cloud Platform
wget -qO- https://www.gstatic.com/ipranges/cloud.json | jq '.prefixes[] | .ipv4Prefix' -r
Amazon AWS
wget -qO- https://ip-ranges.amazonaws.com/ip-ranges.json | jq '.prefixes[] | .ip_prefix' -r
Azure (download first)
jq < ~/Downloads/ServiceTags_Public_*.json '.values | .[] | .properties.addressPrefixes | .[]' -r
The IP address ranges cover all the provider's IPs. Attackers might narrow their focus to active regions using reconnaissance and add a condition in the JQ program:
Scanning Large Ranges: Masscan
After finding the IP range to scan, attackers can use a tool like Nmap. Nmap is strong with many scan types and vulnerability checks, but it struggles with very large networks. While it can scan hundreds or thousands of IPs, it becomes slow for networks with hundreds of thousands or millions of hosts.
Masscan, by Robert David Graham (https://github.com/robertdavidgraham/masscan), is a network scanning tool that focuses on speed. Unlike Nmap, it doesn't have many assessment features. Masscan sends SYN packets quickly by separating sending and receiving functions, which helps it scan for open ports much faster.
Masscan sends 100 packets per second by default, but it can send 50,000 packets per second using the --rate 50000 option, using about 20 Mbps of bandwidth. With this speed and scanning 10 common ports, it can scan 16 million IPs in an hour.
Masscan Scan of AWS us-east-1
Masscan can scan many IP addresses, including all of a cloud provider's, if the system is fast and the scan takes hours or days to finish.
Masscan can scan large IP ranges, like all IPs of a cloud provider, in hours or days with a fast connection. In this example, we use Wget and JQ to get AWS IP addresses from the us-east-1 region and save them in a file. We then scan those IPs for TCP port 443 using the list, and save the results.
The us-east-1 AWS region has about 32.7 million IP addresses. Scanning a single TCP port with Masscan at 100,000 packets per second (PPS) takes about 5.5 hours. However, scanning too fast can cause false-negatives. Slowing the rate to 10,000 PPS makes the scan 10 times slower but improves accuracy.
After the scan, the output shows open ports, IP addresses, and the time they were found. We can use awk to extract the IPs and save them in a new file for the next step.
Attributing Hosts
The Masscan results show IP addresses using TCP port 443 but don’t tell us who owns them. To find out the owner, we can connect to the port and get the server’s certificate, which reveals the owner’s identity. Attackers can also use this certificate to figure out who owns the cloud service.
We identify the organization for the AWS cloud VM at IP address 18.207.73.1 from the Masscan results. We use OpenSSL to connect to the server and get certificate information. By running OpenSSL with the s_client and -connect options, we retrieve the server certificate. Then, we use OpenSSL again to decode this certificate into readable text. We pipe the decoded certificate to grep to get details like the country, state, locality, organization, unit, and server name.
OpenSSL connects to one server at a time, but we can use another tool to speed up data collection.
TLS-Scan
TLS-Scan by Binu Ramakrishnan (https://github.com/prbinu/tls-scan) is a tool for Linux and macOS. It reads IP addresses from a list and gets certificate info from TLS servers on port 443 (or other ports).
TLS-Scan reads IP addresses from input, checks each IP’s certificate details, and saves them in JSON format. We get the IPs from TCP port 443, then use TLS-Scan with port 443, the CA certificate bundle, and specify the output file.
TLS-Scan is quick and doesn’t stop if a server is slow or unresponsive. It still takes a few minutes to gather certificate info from many systems, but it shows progress updates while scanning.
Interpreting TLS-Scan Results
TLS-Scan saves results as a JSON file. we can use JQ to extract and format this data into CSV for other tools.
We use a small JQ script to get results from a TLS scan, focusing on the ip and subjectCN fields in the certificateChain array. We combine these fields into comma-separated values and save them to a CSV file using JQ’s raw mode to avoid extra quotes.
The certificateChain in TLS-Scan results shows each certificate's identity in the chain. There are usually at least two certificates: the server's and the CA's.
Other Scanners: EyeWitness
An attacker can use Masscan, TLS-Scan, JQ, and Grep to find servers in a cloud network that are listening on TCP/443 for a target organization. For a large organization with many servers, this could create a long list of targets. To focus their attack, the attacker will want to narrow down this list to find the most useful systems.
EyeWitness by Chris Truncer (https://github.com/FortyNorthSecurity/EyeWitness) helps attackers quickly gather info on cloud systems or internal servers. It takes screenshots of detected web servers and creates an HTML report with these images. This lets attackers easily see the purpose of various websites, recognize default and management pages, and spot indexed directories. EyeWitness can also find default credentials for some web apps and servers, aiding attackers in focusing on the most useful targets.
To use EyeWitness, we need to provide URLs. From the results of Masscan and TLS-Scan, where TCP/443 is open, we can create a list of IP addresses and host names of web servers to check with EyeWitness.
While we can use just IP addresses with EyeWitness, this might not give the best results. Many web servers show different content when accessed by IP address compared to using the host name due to virtual host setups. For the best results, check both the IP address and any associated host names.
In the TLS-Scan JSON results, attackers can find the IP addresses of servers responding with TLS and identify server host names from the Common Name (CN) and Subject Alt Name fields. The script extract-tlsscan-hostnames (https://urls.sec504.org/lro6s) reads this JSON file and lists all identified host names, one per line.
Cloud Scanning Defense
In this module, we learned how attackers scan many IP addresses to find cloud systems linked to a target organization. Defending against this is tough because many systems don't log half-open SYN scans (from tools like Nmap and Masscan) or TLS-Scan access (which ends the connection after getting the certificate). Although we can use packet capture tools, firewalls, or network flow analysis to monitor scanning, it’s hard to distinguish between malicious scans and regular server activity.
When checking cloud security, defenders should ask which systems need to connect to the server. For a cloud web server, consider if it needs to be publicly available or if access can be limited using firewalls or a Virtual Private Cloud (VPC). Often, API and web servers meant to be accessed only through a Web Application Firewall (WAF) don’t need to be open to everyone and can be restricted to a few specific IP addresses.
As defenders, we focus on what happens after a scan. Web server logs can show if an attacker is browsing all the server pages or sending strange requests. We'll keep examining these attack patterns and find the best ways to protect both on-premises and cloud systems.
Lab 2.3: Cloud Scanning
Let's run gomasscannet to set up the target, then use Masscan to scan the 10.200.0.0/16 network for servers on TCP port 443.
In this output we see the remote network representing the cloud environment (10.200.0.0/16) is accessible through the router (gateway) at 192.168.200.10.
Let's run masscan to enumerate cloud targets with hosts listening on TCP/443 (HTTPS).
Let's break down the command:
-p 443: Find systems listening on TCP port 443 using a SYN scan.--rate 10000: Send 10,000 packets each second.-oL simcloud.txt: Save results in a file.
Let's see the scan result file's contents.
Masscan found systems listening on TCP/443. We need a file with just the IP addresses, one per line, for use with other tools.
Now we have a list of IP addresses (14 target), and we can use tools to check these hosts. Next we'll use TLS-Scan to identify details about the targeted hosts.
Let's simplify the command like this:
--cacert=/opt/tls-scan/ca-bundle.crt: Specifies the path to the root CA certificate bundle for verifying certificates.-o simcloud-tlsinfo.json: Save the scan results to this file.< simcloud-targets.txt: Use shell redirection to read the IP addresses fromsimcloud-targets.txtand get the list of targets to assess.
TLS-Scan connects to 14 target hosts to gather certificate info and saves it in a JSON file. This info can often show who owns the cloud system, helping an attacker identify the target.
The TLS-Scan JSON file has details like certificate subject, common name, and subject alt name for attribution. We can use JQ to extract this info.
Let's run JQ to get IP addresses and subject common names from the certificate chain for each TLS server.
We can break down the command as follows:
.ip + " " + .certificateChain[].subjectCN: JQ selects the.ipand.certificateChain[].subjectCNfields from each JSON record and joins them with spaces.< simcloud-tlsinfo.json : Get the JSON data using shell redirection.
The JQ command will show IP addresses and certificate names for multiple hosts. Let's use grep to find the IP address and hostname of the Falsimentis system.
Create an EyeWitness Report
EyeWitness is a helpful tool for making visual reports of target systems, showing screenshots of websites or graphical interfaces like RDP and VNC. It creates an HTML report, unlike Masscan and Nmap which don't focus on visuals.
Let's create an EyeWitness report for all of the cloud systems identied in the Masscan results.
Now let's open the html report.
In this lab, we learned how attackers use Masscan to scan many systems quickly. We scanned 65,535 hosts, but Masscan can check millions. The scan shows which systems are open on certain ports but doesn't reveal who owns them. However, TLS-Scan can quickly get certificate details to help identify the server owner.
Bonus Lab
Let's visit the new Falsimentis target system at https://10.200.74.2. The server is also called downloads.falsimentis.com, according to its certificate.
1) What is the hidden directory specied in the robots exclusion file?
Let's download the robots.txt file.
Answer: 918cd9e790b13972faa1034157a11982
2) What file is disclosed in the Falsimentis download server in the hidden directory?
The robots.txt file stops search engines from looking at certain web server directories. Let's use curl to check out what's in that directory.
Answer: Falsimentis Board Meeting Minutes 4Q.docx, Falsimentis Board Meeting Minutes 4Q.pdf
3) Download the identied files. Use the Metadata in the files to identify the author, editor, application used to compose the document, and document producer.
Let's download the files from https://10.200.74.2/918cd9e790b13972faa1034157a11982/
Let's examine the two files using exiftool .
We can use grep also to get the author, editor, application, and document producer.
Let's break down the command:
-i: Perform case-insensitive matching-E: Treat the search term as a regular expression (regex)
Answer: Florina Schieicher (author), Roseann/Rose Bycraft (editor), Microsoft Office word (application), macOS 11.2.3 (document producer).
Last updated
