Bootcamp: Linux Olympics
The Linux Olympics is an interactive, self-paced event with integrated hints to help us build our skills at completing common tasks in the Linux terminal. In this exercise, we will have the chance to build our skills and complete command-line tasks including using common Linux commands, navigating the Linux file system, working with file and directory permissions, managing Linux processes, working with Linux networking, and leveraging powerful commands for text file processing.
Let's start the Bootcamp Exercise.
Training: Common Commands
1) The ls command is used to list files. List the files in your home directory.
2) The 'cat' command is used to display the contents of a file. Display the contents of the fencing.txt file.
3) In Linux, files that begin with a . are hidden by default. List the hidden and non-hidden files in your home directory using 'ls -a'.
4) Display the content of .fencingfact1.txt file.
5) The 'pwd' command prints the name of the working directory. Identify the current directory name.
6) The 'cd' command is used to change to a new directory. Change to the epee directory.
7) Display the fencing fact in the epee directory.
8) The 'mkdir' command is used to create a new directory. Create a directory called bout, then change to the new directory.
9) Running 'cd' with no arguments returns to the home directory. Return to the home directory.
10) The 'whoami' command will display your user name. What is your user name?
11) The 'id' command displays your user name, user ID, and group information. What is your user ID?
12) Display the fencing fact in the foil directory.
13) Display the final fencing fact in the sabre directory.
14) The 'rm' command is used to remove files. Remove the fencingfact4.txt file.
15) The 'mv' command is used to move a file to a new directory. Change to the foil directory. Move the fencingfact3.txt file to /tmp.
16) The 'mv' command is also used to rename files. Change to the /tmp directory, then rename the fencingfact3.txt file to fact.txt
Training: File System
1) The Linux file system uses absolute and relative path references. Absolute paths always start with a '/'. Change to the /var/tmp/badminton directory using an absolute.
2) Display the badminton fact in the /var/tmp/badminton directory.
3) Relative paths can also be used in Linux without a leading '/'. Change to the ../badminton2 directory using a relative path.
4) Display the badminton fact in the /var/tmp/badminton2 directory.
5) Relative paths can use multiple '..' indicators to travel up several directories. Change to the ../../../tmp directory using a relative path.
6) Display the badminton fact in the /tmp directory.
7) The find command uses the basic syntax 'find [starting directory] -name "wildcard"'. Use the find command to list all files matching *.txt in /home.
8) Repeat the 'find' command, this time displaying all files ending in .TXT in /home.
8) Repeat the 'find' command, this time displaying all files ending in any case combination of .TXT in /home or in /tmp.
9) Display the new badminton fact in the /tmp directory.
10) The find command can display output similar to the ls command using the -ls argument. Display the file list information for all badminton facts in /home.
11) The find command can also search for files based on size. Identify the badminton fact file in /home that is 72 bytes in size (-size 72c).
12) Display all of the badminton facts in /var, /tmp, and /home using find's -exec feature and cat.
Training: Permissions
1) Linux uses discretionary permissions on all files for the owner, the group, and other. Identify the owner of the racewalkingfact1.txt file.
2) Attempt to examine the contents of the racewalkingfact1.txt file.
3) You cannot examine the racewalkingfact1.txt file since you don't have permissions to read the file. Change the file permissions by running 'chmod 600 racewalkingfact1.txt'.
4) The permission 600 indicates read/write for owner (6), no permission for group (0) and no permission for other (0).
Re-examine the permissions of the racewalkingfact1.txt file.
5) Notice how the permission information for the file has changed from ---------- to -rw-------. Display the contents of the racewalkingfact1.txt file.
6) Linux supports octal notation for permissions where 4=read, 2=write, 1=execute. Adding the values together produces the desired permission. Change the permission for racewalkingfact2.txt so it is owner and group readable.
7) Display the contents of the racewalkingfact2.txt file.
8) Using octal notation (4=read, 2=write, 1=execute), change the permission of racewalkingfact3.txt so it is read/write by owner, read by group, and no permissions for other.
9) Display the contents of the racewalkingfact3.txt file.
10) Using octal notation (4=read, 2=write, 1=execute), change the permission of racewalkingfact4.txt so it is read/write by owner, read/write by group, and read for other.
11) Display the contents of the racewalkingfact4.txt file.
12) The racewalkingfact5.py file is a Python script. Display the contents of the racewalkingfact5.py file.
13) Using octal notation (4=read, 2=write, 1=execute), change the permission of racewalkingfact5.py so it is read/write/execute by owner, read/execute by group, and read/execute for other.
14) When a file has the execute bit set, it can be run as a program. Execute the racewalkingfact5.py script: ./racewalkingfact5.py
15) The execute permission is also used for directories. Attempt to change to the racewalkingfact6 directory.
16) Change the permission of racewalkingfact6 to 755.
17) Change to the racewalkingfact6 directory, display the race walking fact, then change back to your home directory using a relative path.
18) Display the permissions for the racewalkingfact7.txt file.
19) File ownership is controlled by the chown command. Attempt to change the ownership of the racewalkingfact7.txt file to the olympian user with the chown command.
20) Only the root user can change the ownership of a file. Repeat the last chown command, this time adding the sudo command to the beginning of the command.
21) Display the contents of the racewalkingfact7.txt file.
22) Unlike ownership, a user can change a group assignment to any other group where they are a member. Change the group of racewalkingfact8.txt to athlete using the chgrp command.
23) Display the contents of the racewalkingfact8.txt file.
24) The racewalkingfact9 file is a compiled program. Change the file so that it is read/write/executable for the owner, read/executable for group and other.
25) Execute the racewalkingfact9 program.
26) The permission 755 implies a leading zero: 0755. When you set the leading 0 to 4, the file is SETUID, and will run with the permissions of the owner, not the user running the program. Change the owner of racewalkingfact9 to root.
27) SETUID files can be dangerous on the file system, since they grant privileges of the file owner. Change the permission of racewalkingfact9 to 4755 to make it SETUID.
28) Execute the racewalkingfact9 program as a normal user.
29) An attacker with root access can use SETUID permissions to create a backdoor. Copy the /bin/bash file to /tmp/backdoor.
30) An attacker with root access can use SETUID permissions to create a backdoor. Change the ownership of /tmp/backdoor to root.
31) An attacker with root access can use SETUID permissions to create a backdoor. Change the SETUID of /tmp/backdoor to 4755.
32) An attacker with root access can use SETUID permissions to create a backdoor. We have revoked your sudo access, but you can still get root access by running the backdoor command with the -p argument.
Training: Processes
1) The 'ps' command is used to list running process information. List the processes for your user account.
2) Running 'ps' only lists processes for your current session. Adding the '-f' flag to the 'ps' command displays additional process detail including command line arguments. List the command line arguments and other details associated with your processes.
3) Adding the '-w' flag to the 'ps -f' command displays wide output. Obtain the wide output for the trampolinefact1 process.
4) Adding two '-w' flags to the 'ps -f' command displays wide output without truncation. Obtain the full command line argument output for the trampolinefact1 process.
5) Adding the '-e' flag to the 'ps -fww' command displays all processes, not just those for your current session. Identify the full command line for the trampolinefact2 process.
6) Examine the file associated with the trampolinefact3 command line argument.
7) Terminating a Process. Start the trampolinefact4 program.
8) Terminating a Process. Since the trampolinefact4 program does not exit by itself, your prompt does not return. Send an interrupt signal (SIGINT) message to the process by pressing CTRL+C to terminate the process.
9) Terminating a Process. Programs can choose to ignore the SIGINIT message. Start the trampolinefact5 program.
10) Terminating a Process. Attempt to terminate the process by pressing CTRL+C (SIGINT).
11) Terminating a Process. This process ignores the CTRL+C SIGINT message. Send the quit signal (SIGQUIT) by pressing CTRL+\.
12) A process that automatically runs in the background is a daemon process. Launch the trampolinefact6 program.
13) Identify the process ID of the trampolinefact6 process.
14) When a process runs in the background, you send a signal using the 'kill' tool. Send the default signal (terminate signal/SIGTERM) to the trampolinefact6 process.
15) Terminate the other trampolinefact processes.
16) Daemon processes can also ignore signals. Launch the trampolinefact7 program.
17) Identify the process ID of the trampolinefact7 process.
18) When a process runs in the background, you send a signal using the 'kill' tool. Send the default signal (terminate signal/SIGTERM) to the trampolinefact7 process.
19) Identify the process ID of the trampolinefact7 process.
20) Notice how the trampolinefact7 process refuses to terminate. Terminate the trampolinefact7 process using the kill signal/SIGKILL by adding '-9' to the kill command (before the process ID).
21) Start the trampolinefact8 process.
You can stop a running process by pressing CTRL+Z. Stop the trampolinefact8 process.
22) The 'jobs' command shows you the status of processes in your current terminal. Run the 'jobs' command.
23) You can resume a stopped process in the background by running 'bg'. Resume the stopped trampolinefact8 process.
24) Run the 'jobs' command again to see the changed status of the trampolinefact8 process.
25) You can return a background process to run in the foreground by running 'fg' Return the trampolinefact8 process to the foreground.
26) Processes can also be run in the background by adding a trailing ampersand to the end of the command line. Start the trampolinefact9 process as a background task.
27) Examine the running status of the trampolinefact9 process.
28) Return the trampolinefact9 process to the foreground.
Training: Networking
1) The modern Linux 'ip' command has replaced several other networking tools. Run the 'ip' command with no argument to see the basic usage information.
2) The 'ip' command can be confusing, so we'll examine several use examples. Run 'ip addr show' to display interface and address information.
3) Notice the interface names and addresses assigned to interfaces. The 'ip addr show' command can also be shortened to 'ip a'; run this shortened command.
4) To examine information for a specific interface, add 'dev ' to the 'ip addr show' command. Examine the 'ip addr show' detail for the eth0 device.
5) The 'ping' tool is used for basic connectivity tests. Use 'ping' to test the connection to the 172.30.0.35 host.
6) You can specify the number of ping packets to send with '-c'. Ping the 172.30.0.35 target four times.
7) You can set a static IP address using 'ip addr add <ip/mask> dev eth0' Add the IP address 10.10.10.10/24 to the eth0 interface using sudo.
8) Test the 10.10.10.10 IP address set on the eth0 interface using ping.
9) You can remove a static IP address using 'ip addr del <ip/mask> dev eth0' Remove the 10.10.10.10/24 IP address from the eth0 interface using sudo.
10) You can examine the status of an interface as up or down using 'ip link show dev eth0'. Examine the status of the eth0 interface.
11) The 'ip link set dev eth0 down' command will place the interface in the down state. This command requires root access. Change the eth0 interface to the down state.
12) Change the eth0 interface back to the up state.
13) The 'ip route' command will display the host routing table. Display the host routing table.
14) The 'ip neigh' command will display the host ARP table. Display the host ARP table.
15) The 'netstat -na' command will list all network connections (inbound and outbound), as well as UNIX sockets. Use the netstat command to identify the outbound connection.
16) Adding the '-t' argument to the netstat command will limit the display to TCP connection information. Re-run the netstat command, this time with the argument '-nat'.
17) To focus on UDP information, change the '-t' argument to '-u'.
18) In the prior examples, the netstat command revealed a mostly idle host. Run the 'netstat -nat' command again, identifying an outbound SSH session with a state of ESTABLISHED.
19) In this output you can see an SSH connection to 172.30.0.35. Use the netstat command again to identify a new TCP port that is listening (accepting connections) on the host.
20) The prior output identifies listening ports, but does not indicate which process is listening on the ports. Add the '-p' argument to identify the PID and process name information (you must be root).
21) A modern replacement for netstat is the ss command, which accepts the same arguments. Examine the outbound and listening ports using the ss command.
Training: File Processing
1) Linux includes several tools for working with plain-text files. A simple editor for creating files is nano. Run 'nano tabletennisfact1.txt' to edit the file.
2) Use your arrow keys to navigate your cursor. Make any change to the file, then press CTRL+X to exit, saving your changes.
3) You can also create files using echo and shell redirection: echo "Any Content" > file Create a file named paddle using echo with the message 'ball'.
4) Another option is to use 'cat'. You can use cat to send any input to a file using standard redirection, including several lines of text. Create a file called 'offensivehits' using cat: cat > offensivehits' . Once you've created the file, you can add anything you want. Type the word 'hit' and press Enter.
5) You can keep adding more lines, typing or pasting text to add to the file. Type the word smash to add to the file, then press Enter. To close the file press CTRL+D.
6) Using any technique, create a file called 'defensive' with four lines: push, chop, block, and lob.
7) Now you know three options for creating files: nano, echo, and cat. Next we'll look at retrieving data from files. The 'head' command will retrieve 10 lines from the beginning of a file. Run 'head /etc/passwd' to read from the beginning of the file.
8) You can specify a number of lines other than 10 using the -n argument. Display only the first two lines from the /etc/passwd file.
9) The 'tail' command will retrieve 10 lines at the end of a file. Examine the last 10 lines from the /etc/passwd file.
10) Like head, the tail command accepts a line count with -n. Display the last 3 lines of the /etc/passwd file.
11) Tail can skip one or more lines with -n +2 (or any other number). This is useful if you want to skip the first or last line in the file. Read the /etc/passwd file, starting at the 2nd line from the beginning of the file.
12) The 'grep' command is a powerful tool, allowing you to search for one or more lines in a file or any stream of data: grep searchterm filename Use grep to search for the string 'players' in all .txt files in your home directory.
13) Adding the '-C2' argument to grep adds two lines of context to the matched output; by default, this is one line before the matched term, and one line after. Add the -C2 argument to the previous command to display added context in the search results.
14) By default, grep performs a case-sensitive search. Add the -i argument to perform a case-insensitive search. Identify the two files with the word 'minute' in the table tennis fact .txt files with 2 lines of context.
15) Working with text files, we often want to collect basic stats on the file. This is possible through the 'wc' (word count) tool. Use the wc tool to collect basic information for the tabletennisfact4.txt file.
16) In this output, wc tells us that the file has 2 lines, 23 words, and 116 characters (including new line markers). Add the -l argument to obtain just a line count from the file.
17) Linux also excels at sorting and removing duplicates lines from basic files. Start by examining the first several lines of the tabletennisfact5.txt file.
18) This file lists several variations of table tennis paddle rubbers. Count the number of lines in the file.
19) The 'sort' tool sorts the contents of a file alphabetically and numerically: sort filename. Sort the file contents alphabetically using the 'sort' tool.
20) Pipelining is when we take the output of one tool and send it to another tool using a pipe: | Repeat the tail command and count the number of lines.
21) Instead of counting the number of lines, send the tail output to the sort command to sort the output.
22) The sorted output shows some duplicate entries. Add the -u argument to sort to make the output unique (eliminating duplicate lines).
23) Count the number of sorted, unique paddle types by adding another pipeline with 'wc -l'.
24) Adding the -n argument to sort will cause it to treat input as numbers and sort numerically. Sort the tabletennisfact5.txt file using the numeric sort option.
25) The 'cut' command allows you to cut selected portions of text. This is often valuable when working with delimited files. Examine the first several lines of the tabletennisfact6.csv file.
26) The first line of tabletennisfact6.csv indicates the row titles for this CSV file. We can use cut to specify a comma delimiter '-d ,' to access a field by the column number ('-f 1' for the first column). Access the name column of the tabletennisfact6.csv file using cut.
27) Use a pipeline to count the number of unique Olympic cities in the tabletennisfact6.csv file (removing the initial header line).
28) The field specifier can accept a comma-separated list of fields to retrieve multiple columns from the data: -f 1,2,4 Retrieve the Olympic city and the player name from the data using 'cut'.
29) The 'uniq' command performs the 'sort -u' function, but has an extra feature: count the number of unique lines. Count the number of table tennis medals allocated for each Olympic year.
30) Which two table tennis athletes have the most Olympic medals (tied)?
Training: JSON File Processing
1) JSON is the JavaScript Object Notation, a common data format that you will use often when working with cloud assets. Display the contents of the JSON file fact1.json.
2) JQ is a powerful tool and scripting platform to process and manipulate JSON data. In the simplest form, JQ takes JSON data and converts it into a human-readable format. Run 'jq "." fact1.json' to pretty-print the JSON file.
3) JQ can also read the JSON data from standard input. This is often used with cat to read the file and pipe the contents to JQ. Display the contents of the fact1.json file with cat, then pipe the results to jq.
4) JQ also validates the data content and will display an error if the content is not valid JSON. Display the contents of the file fact2.json with cat, then attempt to parse the file with JQ.
5) Notice how the fact2.json file does not validate when processed by JQ, due to a missing quotation mark at the end of the line. Edit the file to add a quotation mark at the end of the string, inside the closing square bracket, then run JQ again to parse the file.
6) JQ's error messages are not always as intuitive as one would like. Display the contents of the fact3.json file using cat, then attempt to decode the fact3.json file using JQ.
7) JSON requires all strings to be denoted in double quotes, while JavaScript permits single quotes for strings. This can lead to inconsistencies where JQ will refuse to parse data but we can fix this with a simple script. Display the contents of the fixquotes.sh script using cat.
8) The script uses sed to convert sinqle quotes to double quotes, correcting the problematic JSON formatting. Run fixquotes.sh specifying the fact3.json file as the command line argument, sending the converted file content to JQ.
9) JSON files can encode multiple data elements. Examine the file DATATYPES.json to learn more about the data types. Read all of the information contained in this file!
10) JSON does not support some other data types. Examine the file DATATYPES-UNSUPPORTED.json to learn more about unsupported data types. Read all of the information contained in this file!
11) JSON arrays are a common data type representing multiple data elements of all the same type. Examine the file array1.json to see an example of an array of strings.
12) Arrays can contain any data type, as long as all of the elements are the same type. Examine the file array2.json for an example of an array of numbers.
13) JSON objects are a data type that can contain any type of data using the "attribute":value syntax with multiple elements separated by commas. Examine the file object1.json to see an example of this syntax.
14) Here we see two attributes (year and city) with values that are different types (number and string) in a single object. Objects are often used in an array to denote multiple data elements for a given record. Examine the file object2.json to see an example of this syntax.
15) Here we see an array of objects, each denoting a year and a city value. Object elements can also be nested, where the data elements can contain multiple levels of arrays and objects. Examine the events.json file to see an example of this syntax. (Tip: pipe the results to more to see one screen of data at a time).
16) JQ supports several built-in functions to perform operations on the JSON data. By specifying a function name and optional arguments, we can get additional information about the JSON data. Decode the events.json file with JQ, adding the argument 'length' to count the number of objects in the top-level JSON array.
17) JQ allows us to extract specific information from the JSON results using dot notation. Start by accessing the first array element by adding '.[0]' (including the single quotes) to the end of the prior JQ command reading from events.json.
18) If JQ cannot retrieve the information from a specified data object, it will return null. Attempt to access the 7th array element (one more than what is in the events.json array) to see this behavior.
19) JQ can work with a whole collections of objects instead of a single array element. This is useful when we want to work with data collectively instead of accessing individual elements. Access the collection of objects in the array, removing the index number from the prior command (".[]"). This is known as the array selector, and is often used to process data within an array of elements.
20) JQ supports the concept of pipelining within the program syntax. Pipelining is similar to Linux and Windows command prompts where one command sends output to another command using the pipe character. With JQ, the pipeline is included within the quotation marks that delimit the JQ program. Repeat the last command, adding a pipe character after the array dot notation, followed by the length built-in function.
21) Here, length operates on all of the received objects, counting the number of elements for each object (the 1st through 5th objects all have three fields: year, city, and medalists; the last object only has two fields). Dot notation also allows us to access individual fields within the objects by specifing the field name. In the prior command, replace the length function with dot notation to access the year field in raw format by adding the -r argument to jq.
22) Repeat the prior command again, this time requesting the .city object instead of the year.
23) Using JQ you can access multiple elements instead of the single .city field by supplying a comma-seperated list of fields. Re-run the prior command, requesting the .year and .city elements in the JQ pipeline. Retain -r argument for raw output.
24) Here we see that we can request multiple fields in the JQ output using a comma-seperated list. JQ will display the fields one per line. The JQ command line argument -j changes this behavior, and will join all of the fields together. Re-run the prior command, adding the -j argument.
25) This output is less than useful since there is no delimeter between fields. In addition to requesting fields in the comma-seperated list, JQ also allows you to request static strings denoted in quotation marks. For example, .field1, ",", .field2 will display field1 followed by a comma, then field2. Request the year and city fields, seperated by a comma.
26) This is closer to being useful output, but the end of the record is still immediately appended to the prior record. Add another comma-seperated string field, this time using the \n indicator to add a new line to the end of each record.
27) Adding static strings in JQ output is useful but awkward. JQ can simplify this process, but first we need to look at the forced type conversion feature. Within the pipeline for any single element, we can convert multiple fields to another data type such as an array by surrounding the fields with square brackets. Request the .year and .city fields in a comma-seperated list again (without the string seperators), this time putting the field list inside of square brackets.
28) In this output JQ has converted the year and city fields from the raw data to arrays of data, one per record. Since the data is in array format, we can use other JQ functions to manipulate the data. The JQ join function joins the elements of an array using a delimiter. Add the join(",") command to the end of the JQ pipeline; remove the raw and join command-line arguments.
29) This is useful output, and simpler to design where we don't have to add the string elements to manually seperate each field. Re-run the command, this time adding -r to produce the output in raw format, omitting the quotation marks
30) Converting data to CSV format is a common task for JQ. To simplify this process, JQ has a format string modifier @csv. Run the previous command again, this time removing the join(",") command and replacing it with @csv. Notice how the city string is included in quotes to preserve any string including those with spaces.
31) We'll continue to use this command to request an additional field of information: the gold medal winner for each year. Examine the pretty JSON data from JQ to refamiliarize yourself with the data structure, accessing the array objects using .[] as the only pipeline request.
32) This JSON structure includes medalist winner information nested in each of the yearly object elements. Nested objects are common in JSON, and easily accessible using dot notation. To access a nested object, specify the top-level object (.medalists) immediately followed by the nested element name (.gold or .silver or .bronze in this example). Modify the previous JQ command where we retrieved multiple fields, adding the gold medalist to the CSV output.
33) Repeat the prior comand, adding the silver and bronze medalists.
34) Next you will apply what you've learned to a new data file. Use the file topathletes.json to answer the remaining questions in this event. Use JQ to pretty print the data in the topathletes.json file.
35) Use the length function to count the number of elements in the top-level array.
36) Extract the fourth data element in the top-level array.
37) Extract the list of countries represented by the top modern pentathalon athletes in raw format.
38) Extract a list of all athlete names in the order First Last, seperated by a space, with a newline at the end of each record, in raw format.
39) Create a CSV data set of two colums: country followed by total medals won, in raw format. Include quotation marks around the country name.
Final Olympic Event
1) Identify the username of your logged-in account.
2) Identify your user ID, group ID, and group assignments.
3) List your sudo permissions.
4) List the files in your home directory.
5) Identify the listening TCP port numbers.
6) Continue your analysis, focusing on the system login file: /var/log/auth.log Copy the /var/log/auth.log file to your home directory.
7) Change the owner and group for the /home/admin/auth.log file so that admin is the owner and admin is the group.
8) Change the permissions on the /home/admin/auth.log file so that it is readable and writable by owner with no permissions for the group and no permissions for other.
9) Examine the first several lines of the auth.log file.
10) How many times does the case-insensitive string 'failed' appear in the auth.log file?
11) Examine the failed password login attempts in the auth.log file for a user other then root.
12) Use a tool to count the number of lines in auth.log where a password is accepted.
13) The auth.log doesn't necessarily indicate a successful compromise. Wardrobe99 network admins have prepared a web server access log extract for you to evaluate as well. Examine the file size of the /var/log/nginx/access.csv file.
14) Begin to assess the web server activity. Change to the web server log directory in /var/log/nginx.
15) Examine the first few lines in the access.csv log file.
16) How many lines are there in access.csv file?
17) How many times does phpmyadmin as a case-insensitive string appear in the logged requests?
18) How many times was phpmyadmin as a case-insensitive string NOT part of the logged requests?
19) Identify the largest HTTP response length in the access.csv file.
20) Which IP address made the request to generate the longest HTTP response length?
21) Identify any other HTTP requests that include the string shelleyz.
22) The web server log indicates a possible attack involving a process called linkd. Search the process list to identify a process called linkd.
23) Wardrobe99 management is concerned about the attacker's access for this breach and have asked you to disable network access for this server. Place the eth0 interface into the down state.
24) Terminate the linkd process.
25) Identify the fully-qualified location of the linkd executable.
26) Finally, make the linkd process non-executable.
Last updated
