Part Two
NIDS Design
NIDS like Snort and Sourcefire can operate in three modes. Snort can be set to use only anomaly-based rules, but most setups, including default ones, mainly use signature-based rules.
Signature Matching
Here is a signature-based rule from Emerging Threats:
The signature activates when the string "/etc/shadow" appears in TCP traffic from outside hosts to HTTP servers.
Protocol Behavior
Attackers distort packets, which a protocol behavior IDS can catch. However, some developers also distort packets because they often skip reading the RFCs that explain protocols like TCP. They create applications that function but don't always follow the official design rules.
A protocol behavior IDS will warn about bad traffic, but it might also alert for badly designed apps that send network data.
Anomaly Detection
Anomaly detection is often seen negatively due to bad design and implementation. It's most effective in small, well-structured networks and specific high-risk situations. Knowing the systems and networks being protected is crucial for success.
Most anomaly-based detection systems have struggled to work well in large and complex networks, leading to disappointing results and a bad reputation for their effectiveness.
Anomaly-based detection checks for unusual activities by comparing them to what is considered normal. An IDPS using this method creates profiles that show normal behavior for users, devices, network connections, or applications by watching their usual activities over time.
Detecting Specific Anomalies
The "classic" anomaly-based design hasn't been popular because of many false positives. However, focusing on quality shows that certain unusual network traffic patterns are valuable:
Random names for .EXEs, DLLs, directories, usernames, DNS names, and function calls
One-character Windows .EXE names
Client-client .EXE flow
Large data in ICMP echo request/response payloads.
Purists might say this isn't classic anomaly-based IDS. It's better described as targeted anomaly detection using rules made by people, not machines.
There Is No Easy Button
The right person in the right job with enough power is essential, and they work best in teams with similar professionals.
Large organizations spend on products and services but often neglect to invest in their own employees. No outside company understands your important data and network as well as your employees.
NSM Data Sources
More data is better than less, and slow data is better than none. Centralizing lots of data in a SIEM like ArcSight is helpful, but these systems are often too small and perform poorly.
Packet Data
Most modern packet tools can import and export pcap data. A new format, PcapNg (PCAP Next Generation), aims to fix the limits of the old libpcap format. PcapNg offers features like dropped packet counts, comments, local IP addresses, interface and direction info, and a hostname-to-IP address database.
Wireshark can use PcapNg files, but many other tools can’t. It's better to save your pcap files in the pcap format unless you need PcapNg features.
Full Packet Capture Tools
Some tools, like tcpdump, can capture and analyze packets. However, it's best to use a specific tool for long-term packet capture.
Three well-known open-source tools for full packet capture are daemonlogger, dumpcap, and netsniff-ng.
Stenographer: https://github.com/google/stenographer
Dumpcap is included with Wireshark: http://www.wireshark.org/download.html
Netsniff-ng: http://netsniff-ng.org/
Extracted Data
Zeek can pull files from packet data, which we'll talk about soon.
Carving Files with Wireshark Step 1: Identify the File
Wireshark can automatically export some files through Edit -> Export -> Objects, but it only works with HTTP, DICOM, and SMB. A future lab will cover this. The next slides will explain how to do this manually, which is useful if Wireshark's export doesn't work.
Let's click on frame 395, right-click, and select "Follow TCP Stream."
Step 2: Choose the Conversation and Save As
Next, we find the download with the most data transferred and save it in raw format.
Edit the File
The file command identifies file types using "magic numbers" at the start of the file. For example, DOS .EXEs start with MZ.
Use the Bless hex editor to edit it:
Highlight the bytes before "MZ," delete them, then save the file as /tmp/carved.exe.
Save the .EXE, Check the File Type, Hash, and Scan with Antivirus
We have a new .EXE file. The file command shows it's a "PE32 executable (GUI) Intel 80386 for MS Windows."
Using sha1sum gives a hash that helps search other services like VirusTotal or Cymru's Malware Hash Registry without needing to upload the file.
Or Use Zeek
We talked about file carving in Wireshark to understand the process, not just run tools. We can also carve files with Zeek using this command in terminal:
The "extract-all-files" Zeek script saves transferred files it detects in a folder called "extract_files" in the current directory.
Pcap Strings Example
This command uses ngrep (network grep) to search for network traffic patterns in a packet capture file:
ngrep: A tool for searching network packet data for specific patterns.-q: Runs in quiet mode, reducing output noise by suppressing non-matching packets.-I /pcaps/virut-worm.pcap: Specifies the input PCAP file (virut-worm.pcap) to analyze."JOIN": Searches for packets containing the string "JOIN".
Flow Data
Flow data is useful for analyzing traffic, especially encrypted traffic. It includes Cisco's NetFlow (version 9) and other general tools.
Pcap Flow Example Using Tshark
This tshark command analyzes a packet capture (pcap) file and provides a summary of TCP conversations. Here's a brief explanation:
tshark: The command-line version of Wireshark, used for capturing and analyzing network traffic.-n: Disables name resolution (no DNS or service name lookups), displaying raw IP addresses and port numbers.-r /pcaps/virut-worm.pcap: Reads and analyzes the specifiedvirut-worm.pcapfile.-q: Quiet mode, suppresses packet-by-packet details, focusing on summary output.-z conv,tcp: Produces a TCP conversation statistics summary, showing details such as the endpoints, bytes transferred, and packets exchanged.
Transaction Data
Pure flow data shows IP addresses, ports, and bytes transferred (Layers 3 and 4). Transaction data adds Layer 7 info, like HTTP/FTP commands or DNS requests.
Statistical Data
Statistical data numerically analyzes network traffic and helps detect anomalies.
Then go to Statistics -> Protocol Hierarchy.
Alert Data
Relying only on IDS alerts is like "seeing the world from the bottom of a well." IDS captures only part of the attack data, making it hard to get the full picture. Full packet capture or tagged data can help fix this.
Example Security Onion IDS Alert
Here is a Suricata alert shown in the Security Onion interface for an "ETPRO MALWARE WIN32/KOVTER.B Checkin 2 M1"
Cloud NSM: Network Visibility
CSPs have improved cloud network visibility enough to now support effective NSM, which was previously limited and forced.
AWS and Azure lead in providing network flow data and packet capture for cloud instances. This area is evolving quickly, but we'll explore current methods to extract cloud network data.
AWS: VPC Flow Logging
VPC flow logs give Netflow-like visibility, not full packet capture. They lack L7 details but still boost network visibility in AWS. AWS Traffic Mirroring came later and offers deeper insights.
VPC flow logs can be set up for the entire VPC, specific subnets, or individual network interfaces. This lets you monitor traffic from a single instance or the whole VPC. New instances in a VPC are automatically logged if the VPC-wide option is used.
AWS: VPC Flow Log
To create flow logs, set the scope (interface, subnet, or VPC), choose traffic type (all, allowed, or dropped), and select the log destination (S3 or CloudWatch). Flow logs don’t impact network speed or latency.
Publishing flow logs to CloudWatch allows searching and filtering, while S3 only stores the logs and needs extra tools for analysis.
AWS: VPC Flow Logging Architecture
AWS VPC Flow Logs can be created for VPCs, subnets, or interfaces and sent to S3 or CloudWatch.
Flow Log Sources
VPC
Subnet
Interface
Flow Log Destinations
Amazon Simple Storage Service (S3)
Amazon CloudWatch
AWS: Traffic Mirroring
The absence of L7 data limits network analysis. While L3/L4 data is useful, some L7 data is often needed. AWS offers Traffic Mirroring, similar to on-premise span ports or taps.
Traffic mirroring works at the interface level, not for entire VPCs or subnets. The destination is a network interface with software to handle and analyze the traffic, not CloudWatch or S3. The traffic uses VXLAN, so receiving tools must support decapsulation.
Traffic mirroring only works on newer Nitro-based EC2 instances, but since more types are using Nitro, this might not be an issue for you.
AWS: Traffic Mirror Sessions
Traffic mirror sessions have a source, target, and filter. The source is a network interface, and the target can be a network interface or load balancer. The filter is what makes the traffic mirror session powerful.
The mirror filter sets rules to decide which traffic gets mirrored. This helps reduce the amount of mirrored traffic if needed. It uses basic details like source and destination IPs and ports for filtering. The rule specifies if traffic will be mirrored (accept) or not (reject). Multiple rules can exist, and each has a rule number for prioritization.
Rule processing works like regular firewalls: the first rule that matches the traffic decides how it’s handled.
AWS: Traffic Mirroring Illustrated
The illustration shows key parts of AWS VPC Traffic Mirroring. The source and destination are network interfaces, and a traffic filter decides which traffic gets mirrored. Any solution processing the traffic needs to support VXLAN decapsulation.
Traffic mirroring captures data from both source and destination network interfaces, unlike VPC flow logs. However, it only works with newer Nitro-based instances, so make sure your instances are compatible.
Azure: Network Watcher
The Network Watcher service lets you monitor your network in Azure. When you create or update a Virtual Network, Network Watcher activates in that region without affecting your resources or costs. However, using Network Watcher's features will result in charges based on the data stored and processed.
We'll look at Network Watcher's tools like IP Flow Verify, NSG Flow Logs, and Packet Capture. Network security groups (NSGs) control what traffic can enter or leave a VM. NSG flow logs record details like source and destination IP addresses, ports, protocols, and whether the traffic was allowed or denied.
Traffic Analytics is a cloud tool that shows user and app activity in cloud networks. It analyzes NSG flow logs to give insights into traffic flow in Azure.
Azure Network Watcher: IP Flow Verify
The IP Flow Verify tool in Network Watcher is very useful for checking if traffic is allowed or blocked by Network Security Group settings. It saves you from having to connect to an instance just to test connectivity.
NSG misconfigurations can cause connectivity problems and this tool helps spot unexpected traffic. It's simple and very useful.
Azure: Network Security Groups
Azure Network Watcher's flow logging works with Network Security Groups (NSGs) linked to virtual subnets and network interface cards (NICs). NSGs act like firewalls and have security rules that include priority (100 to 4096), source and destination information (IP address and port, service tags, or application groups), protocol, rule name, and an optional description.
An NSG acts like a basic firewall, where the first matching rule controls traffic. The order of checking NSGs and their rules matters. Each virtual subnet and NIC can have one NSG. For incoming traffic, the subnet's NSG is checked first. If it blocks the traffic, the NIC's NSG won't be checked. For outgoing traffic, the NIC's NSG is checked first, then the subnet's NSG.
Azure: NSG Flow Logs
NSG Flow Logs track network activity at the Network Security Group level and are stored in JSON format in an Azure storage account. They don't require changes to virtual machines and don't affect performance since they connect directly to the virtual network. Costs vary based on the amount of flow logs processed and storage fees.
NSG flow logs contain a lot of data up to Layer 3/Layer 4 and some Azure metadata to link records to specific cloud assets. Version 2 of these logs includes data on the number of packets and bytes sent in each direction.
Azure Network Watcher: Traffic Analytics
Organizations using Azure's Traffic Analytics can:
See network activity in your Azure subscriptions and find busy areas.
Find and protect your network from security threats by checking open ports, apps trying to access the internet, and virtual machines connecting to suspicious networks.
Analyze traffic patterns in Azure regions and the internet to improve your network setup for better performance and capacity.
Identify network mistakes causing connection failures.
Azure Network Watcher Traffic Analytics: ATT&CK Coverage
The photo shows part of the MITRE Engenuity CTID project, focusing on how Azure security controls relate to MITRE ATT&CK. It lists 14 key tactics from MITRE ATT&CK and highlights relevant techniques for Azure security.
Azure: Network Watcher - Packet Capture
We like flow data, but L7 info is often where security insights are found. Azure Network Watcher lets us capture packets to go deeper than L3/L4 data. First, we need to set up the instances for packet capture, which can be a hassle, but using Infrastructure as Code with Azure Resource Manager makes it easier. The captured packet data can be saved locally or sent to Azure blob storage.
Accessing packet data is great, but it was meant for temporary captures, not constant monitoring. Captures must be stopped to publish logs. Microsoft offers some automation with Azure Functions, but it seems security monitoring wasn't the main focus.
Azure: Virtual Network TAP
Azure offers a better way to access packet data through Virtual Network TAP (vTAP), which can mirror data from one interface to a collector. However, vTAP isn't available in all regions yet, and it requires an approved partner solution as the destination.
Traffic is sent using VXLAN over UDP port 4789. The vTAP works like AWS Traffic Mirroring, which also uses VXLAN. When fully released, it should support sending data to systems like Zeek or Suricata that can decapsulate VXLAN traffic.
You may wonder why we mention vTAP when Azure offers Packet Capture. Both capture packet data, but vTAP is better because it doesn’t require changes to the instance to do so. This is important since adding the agent with Azure Resource Manager can impact normal operations.
Lab 3.2 - Security Onion Service-Side Analysis
Challenges
The following questions are based on a service-side exploit that occurred on July 7 th, 2023.
An attacker stole a domain admin's credentials from a remote office and used PsExec to launch an attack on the 10.5.11.0/24 network from a 10.0.0.0/8 subnet.
The attacker successfully compromised a host on the 10.5.11.0/24 network and then pivoted, successfully compromising other hosts on the 10.5.11.0/24 subnet.
What is the IP address of the attacker and the first victim? The attacker address is on the 10.0.0.0/8 network, and is on a different subnet than 10.5.11.0/24. The victim address is on the 10.5.11.0/24 subnet.
What is the hostname/workstation name of the attacker?
What is the domain admin account that was used to successfully authenticate in these attacks? Answer in domain\username form
A standard Windows binary is executed via the successful SMB authentications via PsExec. What is the full path and name of that executable? Note that files shown in Wireshark that are executed via SMB are often shown without the leading "\". For example, "c:\windows\system32\cmd.exe" may be listed as "windows\system32\cmd.exe". The answers will omit the leading "\", but either form is correct.
An encrypted C2 channel is created seconds after the initial service-side compromise. What is the socket pair of this encrypted C2 channel?
The attacker used the same stolen domain admin username and password to attempt to compromise five other systems via PsExec. Three attacks were successful, and two failed. Which systems were attacked, and which attacks were successful?
Successful attacks generated alerts logged in Security Onion, but failed attacks did not trigger any alerts. While the Hunt menu in Security Onion displays all packets, it might be easier and quicker to check the full packet capture file at /nsm/import/9fd707b68716d7d8c2eb43e296bb943b/pcap/data.pcap to analyze the failed attacks.
Q1) What is the IP address of the attacker and the first victim?
First, let's go to the Security Onion Alerts menu, click the clock icon to switch it to a calendar, and we'll enter this date.
Now let's identify the involved IP addresses by grouping the data as "Sensor, Source IP/Port, Destination IP/Port, Name".
There are many alerts for the IP address 10.5.11.52, so let's focus on it. Let's go to the Hunt menu, click the clock icon to set a specific date, and enter the following:
Then let's search by this:
The remote IP address 10.99.99.43 connects to 10.5.11.52 on port 445, causing multiple NETBIOS SMB-DS alerts. This is followed by an "ET POLICY Command Shell Activity Using Comspec Variable Over SMB" alert, indicating likely lateral movement.
Answer: the attacker ip address is 10.99.99.43, the victim ip address is 10.5.11.52
Q2) What is the hostname/workstation name of the attacker?
Let's click on the rule named "GPL NETBIOS SMB-DS ADMIN$ share access," then select Actions and choose PCAP.
"GPL NETBIOS SMB-DS ADMIN$ share access" refers to a IDS rule that detects access to the ADMIN$ share over the NetBIOS/SMB-DS (Server Message Block-Direct Server) protocol.
Next, let's download the pcap file and open it in Wireshark.
Next, let's click on frame 9.
Answer: vqvtPkdVqiOr3JsP
Q3) What is the domain admin account that was used to successfully authenticate in the SMB attacks? Answer in domain\username form.
Answer: 12colonies\administrator
Q4) A standard Windows binary is executed via the successful SMB authentications via PsExec. What is the full path and name of that executable?
Answer: System32\WindowsPowerShell\v1.0\powershell.exe
Q5) An encrypted C2 channel is created seconds after the initial service-side compromise. What is the socket pair of this encrypted C2 channel?
Let's look for traffic that's not going to port 445. Let's go to the Hunt menu and type in this search:
The traffic to port 51515 is noteworthy. Let's click on the first entry, go to Actions, and select PCAP to download the packet. Then we'll open it in Wireshark, right-click on any frame, and choose Follow, then TCP Stream.
The message "This program cannot be run in DOS mode" shows that a Windows executable was transferred. This happened just seconds after the compromise, so the C2 connection is: 10.5.11.52:49487 -> 10.99.99.43:51515.
Answer: 10.5.11.52:49487 -> 10.99.99.43:51515
Q6) The attacker used the same stolen domain admin username and password to attempt to compromise five other systems via PsExec. Three attacks were successful, and two failed. Which systems were attacked, and which attacks were successful ?
Successful attacks triggered alerts recorded in Security Onion, while failed attacks did not generate any alerts. Although the Hunt menu in Security Onion contains all the packets, it's quicker and easier to open the entire pcap file in Wireshark.
Let's open /nsm/import/9fd707b68716d7d8c2eb43e296bb943b/pcap/data.pcap to determine in Wireshark
Let's filter the traffic to show only TCP port 445 going to or from 10.5.11.52.
We can narrow down the traffic in several ways. Since the attacker is using the "12colonies\administrator" account, we can use two useful filters in Wireshark: ntlmssp.auth.domain and ntlmssp.auth.username. The username filter is more reliable, so let's look for "administrator."
We know about the initial victim at IP address 10.5.11.52. Now, we also see these IP addresses: 10.5.11.10, 10.5.11.44, 10.5.11.85, 10.5.11.173, and 10.5.11.191.
A user from 10.99.99.43 logged into 10.5.11.52 and then tried to log into several other addresses: 10.5.11.10, 10.5.11.44, 10.5.11.85, 10.5.11.173, and 10.5.11.191. The next question is whether these logins were successful.
Let's right-click on the SMB connection between 10.5.11.52 and 10.5.11.10, and Follow-TCP Stream.
There is a lot of SMB traffic, suggesting the login was likely successful. To check each SMB connection from 10.5.11.52, we'll follow the TCP stream. Let's repeat this process again, but this time with 10.5.11.44.
Then with 10.5.11.85.
Then with 10.5.11.173
And the last one 10.5.11.191.
10.5.11.10, 10.5.11.44 and 10.5.11.85 sent a lot of SMB traffic, while 10.5.11.173 and 10.5.11.191 sent much less.
The "Info" column in Wireshark shows "STATUS_LOGON_FAILURE" for both 10.5.11.173 and 10.5.11.191, meaning both login attempts were unsuccessful.
We can support this theory by checking the traffic between the original attacker and each victim they targeted. For example, let's look for traffic between 10.5.11.10 and 10.99.99.43 using this display filter:
This is the same Command and Control (C2) server used by the first victim. We should pay attention to the source port: 51516 (which is one higher than the port used by the first victim). If we search for the IP addresses 10.5.11.44, 10.5.11.85, 10.5.11.173, and 10.5.11.191. This will show that 10.5.11.10, 10.5.11.44, and 10.5.11.85 were compromised, while the others were not.
Answer: 10.5.11.10, 10.5.11.44, and 10.5.11.85 were compromised, 10.5.11.173, and 10.5.11.191 were not
Last updated
