Part Three
Practical NSM Issues
Before diving deeper into NSM, we need to consider some key practical aspects:
Server/sensor design
How to sniff
NTP
Network Security Monitoring needs the right technology and design to be effective.
NSM Sensors and Servers
A basic design uses sensors with a central server. Some data like NIDS alerts goes to the server, while other data like full packet captures stay on the sensors. Security Onion 2.0 calls these manager nodes and forward nodes.
Full packet capture data is kept local because centralizing it needs the same bandwidth as the captured traffic. Analysts can access it on local sensors when needed, with authorization based on data sensitivity.
High-volume data stays on the sensor, while low-volume and summary data is sent to a central server.
Practical Issues: How to Sniff
You need a spot to monitor all network traffic. It's easy on virtual networks but can be a bit tricky on physical ones. However, affordable taps and switches with mirror ports make this easier now.
Don't use a hub for sniffing. Hubs are old tech (Layer 1) and only support half-duplex at 10 or 100 Mbps, which slows down network traffic and reduces performance. TCP/IP needs full duplex.
A small hub can be used for incident handling if a single system is acting oddly and no tap or span port is available. Connecting the system to the hub and the hub to the switch lets you monitor traffic. The downside of slower speeds is minimal for just one PC.
Mirror Ports
Mirror ports have some drawbacks, like not forwarding bad frames or VLAN tags, but they work well if managed switches with mirror port support are already in place.
Managed switches with span ports are now much cheaper. It's not built for heavy use, but it shows how affordable this feature has become.
Cisco calls it a "span" port, but most others call it a "mirror" port. They mean the same thing, and we'll use "mirror port."
Network Taps
The small "throwing star" tap needs two cables to monitor both sides. More advanced taps with dual power and buffers can cost hundreds to thousands of dollars. Overall, adding network taps to production environments isn't usually hard or expensive, except for fiber or very high-speed connections (10 gig+).
Port Overload
Mirror ports and taps can get overloaded. If you send seven 100-megabit streams to a 100-megabit mirror port, it can drop frames. Some advanced taps have buffers to store frames during overload, but these buffers only work for short bursts of traffic. Prolonged overload will fill the buffer.
Regularly check your mirror ports and taps. Adjust as needed by reducing traffic or adding more taps or mirror ports.
Sniffing Virtual Traffic
Sniffing virtual traffic is easy. Just set up a virtual NSM sensor like Security Onion on the hypervisor, select the virtual network, and start sniffing. One sensor can monitor multiple networks, but make sure it has enough virtual RAM, CPU, and disk space.
Practical Issues: NTP, Time Zones, and Daylight Saving Time
Using NTP for synchronization is a basic best practice and is usually free with internet access, using minimal bandwidth.
If your organization operates in different time zones, use Coordinated Universal Time (UTC) for consistency. It’s also better to skip daylight saving time since it varies by location, even within the same country. For example, most of the U.S. observes daylight saving time, but Hawaii does not.
Entropy and freq.py
Malware tries to hide by changing its MS-DOS headers to escape detection. Let’s find these changes!
Cornerstone NSM: Detecting Entropy
Entropy means disorder:
Strong encryption provides a ciphertext with high entropy
Random string: high entropy
Strings like “download” or “files”: Lower entropy
Malware often creates random strings to evade detection. For example, if malware is named "evil.exe," it would be easy to spot through pattern matching.
Programmatic Entropy Analysis
We need a way to automatically detect this, even if analysts have to reduce false positives later. Luckily, there are free tools for entropy analysis, like ent, which can analyze input to measure its entropy. However, using these tools can be complicated.
Get Your freq.py on
A tool called freq.py detect randomness in small strings. It uses Natural Language Processing to analyze how often character pairs appear, improving the chances of finding unusual strings that are unlikely to occur naturally.
The file freq.py includes ready-made frequency tables in lowercase and mixed case, created from a lot of public English text. You can make your own tables with it, but you'll need a large amount of good, normal text to create accurate tables.
Domain Generation Algorithms DGAs
A key use of freq.py is to find automatically generated DNS names. Malware often uses algorithms called Domain Generation Algorithms (DGAs) to create future DNS names without needing a complete list. Analysts usually identify DGAs by reverse engineering malware. While this is helpful, it's even better to spot unknown malware by looking for unusual DNS requests, which freq.py can help with.
While you'll need to overlook some domains and services eventually, and this method won't always perfectly show malicious activity, the benefits of this single approach are significant. Since malware often uses DNS, any techniques that enhance our ability to detect it are valuable.
DGA++ – Beyond Domain Generation Algorithms
DGA detection is a key use for freq.py, but attackers create many other random artifacts to evade simple detection methods. As you learn about adversary tactics, look for items you can identify and use with freq.py.
Despite needing to ignore some domains and services, DNS-based techniques are valuable because malware often uses DNS, offering an extra advantage in detecting threats.
freq_server.py – for freq-ing at Scale
To scale freq.py, you can use a cron job or scheduled task. For advanced use, freq.py offers a server mode with a simple web API.
Tracking .EXE
Malware can bypass signature-based detection. Start using behavioral detection by tracking .EXE transfers. Don't dismiss it just because encryption is harder to catch.
We can detect encryption. Stage 1 malware usually just downloads the stage 2 executable, which then starts encrypting communications. We'll focus on finding these stage 2 downloads along with the other methods.
Stage 2 .EXE
We used the strings command on /pcaps/meterpreter.pcap with a minimum length of 14 to display the .EXE header and encryption functions like "OpenSSL," "AES," and "CAMELLIA."
Camellia is a block cipher similar to AES, with comparable speed and efficient hardware design, making it ideal for low-power TLS applications.
Tracking .EXEs
The listed strings aren't essential to the DOS header and can be changed by malware to avoid detection. Old non-PE files may reverse the magic bytes to "ZM" on older systems like XP.
Identifying Windows .EXEs
"This Program Must Be Run under Win32"
Spot the Anomaly
Many EXEs with obvious anomalies, like this one, often appear "clean" to antivirus at first. High ASCII characters 9d and 9a replace the low ASCII word "be."
Predictable Transfer of .EXEs
EXEs entering an organization should follow a trust model, ranked from most trusted to least trusted, like this: download.microsoft.com → internal WSUS server → desktop. Unfortunately, many organizations don’t use this secure design and let .EXEs be downloaded from almost anywhere, as long as they pass antivirus checks. This approach is unsafe and likely to fail.
Defensible Executable Transfers
The diagram shows an important network security idea: Executables should only come from trusted sources. Many organizations allow users to download any executable that antivirus software approves, which is risky because antivirus can miss some threats. Using malware detection tools like FireEye improves safety, but it's still not completely safe.
How .EXEs Should Not Move
A hallmark of malware is transferring .EXEs from client to client. This behavior is easy to detect, assuming basic network segmentation is in place, and a sensor is able to see the traffic.
Non-Defensible and Suspicious Executable Flow
The diagrams illustrate how malware enters and spreads in an organization. Anomaly-based detection makes it easy to spot these cases. The first example needs a strong network setup with clear sources for all .EXE files, which can be challenging for many companies.
To easily detect the second example, set up basic network segmentation at Layer 3, putting clients on their own network. Then, just create an alert for any executable transfer between clients. If clients and servers share the same network at Layers 2 and 3, you need to redesign your network to fix this issue.
Detecting Stage 2 Downloads
Tracking .EXE transfers on a network is easy, but getting a clear view beyond a basic IDS is the real challenge. However, switches with mirror ports are now cheap, making it affordable to add IDS sensors to important servers and client networks, like those with VP laptops and desktops.
Many have stopped using anomaly-based methods because they produce too many false alarms. Monitoring .EXEs shared between clients is a straightforward and effective way to detect anomalies. A strong network setup and some easy adjustments to our detection systems let us create a practical form of anomaly detection that we can act on right away.
Targeted Anomaly-Based .EXE Rule
Rules like this are often turned off because they create too many alerts. These alerts aren't false positives; they're real alerts from harmless traffic, like downloads from microsoft.com. We should also look for UDP transfers.
Lab 3.3 - Wireshark Merlin Analysis
Objectives:
Analyze artifacts of an attack using the Merlin post-exploitation framework.
Carve Microsoft EXEs from a packet capture using Wireshark.
Gain experience using Wireshark.
Challenges:
We previously investigated an exploit the Merlin post-exploitation framework. We will now dig deeper into this exploit with Wireshark. We used Zeek to carve EXEs from capture.pcap: we will now use Wireshark to do the same thing.
Open /labs/merlin/capture.pcap with Wireshark
View the TCP connection that sent the most data
Extract all EXEs and DLLs
Locate the cmd.exe connection
Let's start by opening /labs/merlin/capture.pcap with Wireshark:
Now, let's check which TCP connection transferred the most data:
In Wireshark: go to Statistics -> Conversations.
Next, we'll click the "TCP" tab and sort by the number of bytes, from highest to lowest.
The connection that sent the most TCP data was from 10.5.11.11:53813 to 192.0.2.160:445.
Now let's right-click on either IP in the TCP stream and select "Apply as Filter" → "Selected" → "A ↔ B".
Next, let's close the "Conversations" window, go back to the main Wireshark window, click on a packet, and select Follow -> TCP Stream.
We previously extracted EXEs from capture.pcap using Zeek. We will now use Wireshark to do the same thing.
We might wonder why we do this step twice with different tools. It's because some tools can find data that others miss. Plus, it's important for an analyst to be independent of any single tool.
In Wireshark, let's go to File > Export Objects > SMB, then click Save All.
Then press Open.
Now, let's identify the sending and receiving IP addresses, filename, and MIME type used to transfer the EXE files. We previously analyzed capture.pcap with Zeek and found a cmd.exe connection. Let's use what we learned from the Zeek analysis to find and track that connection.
We earlier looked for "Microsoft Corporation. All rights reserved" to find the cmd.exe connection. Let's copy that text, then go to Edit -> Find Packet.
Let's open the "Find" bar at the top of Wireshark. Paste "Microsoft Corporation. All rights reserved" into it. The background will stay red. Next, switch "Packet list" to "Packet bytes" and change "Display filter" to "String." The background will then turn green. Finally, let's click "Find."
Then we'll right-click on the highlighted packet and choose Follow -> TCP Stream:
This connection displays data traveling from the sender (port 8443) to the receiver (port 53664). In Wireshark, data sent from the sender is shown in red, while data sent back from the receiver appears in blue. The command typed is shown twice: once when sent and once when echoed back. To view only the traffic from sender to receiver, let's change "Entire conversation" to 10.5.11.11:53722 -> 192.0.2.160:8443.
Last updated
