This is the 10th post of a series for teaching cyber-security in a coding-club. Read [part 9].

Pipe and count

Disk Usage

In our mission we may have to inspect large files. For example creating a file full of possible passwords to try. if you had a huge file called passwords_file and did:

cat passwords_file.txt

it might spend a long time printing the contents to the shell. However there is a way to know how big a file is in advance. Find your shell (or Terminal) and type:

ls -l

You should see something like this:

drwxr-xr-x 1 user group   30 Oct  5 18:13 config
drwxr-xr-x 1 user group   16 Mar 15  2017 images
drwxr-xr-x 1 user group   14 Mar 15  2017 js
drwxrwxr-x 1 user group   20 Sep 28 18:27 mystuff
drwxrwxr-x 1 user group   28 Sep 28 18:26 nested-directories
-rw-r--r-- 1 user group 1704 Mar 15  2017 README.md
-rw-r--r-- 1 user group   35 Mar 15  2017 style.css
drwxr-xr-x 1 user group   52 Oct  5 18:18 test-website

The column after group has numbers:
30, 16, 14, 20, 28, 1704, etc ...
Those are the sizes of the files in bytes. If you want it in a more "Human" readable form (with kilobytes k, Megabytes M or Gigabytes G) you can instead type:

ls -lh

Note the letter h which means human, as in easier to read. But what if we have entered a server and don't know where the large files are. We need to find that Megabyte file which is full of secret data! We can then use:

du -h

Which stands for Disk Usage displayed in human format.
Try it out! You should have gotten a long list of all the folders (or directories) in this computer with all their sizes.

Once you find the largest directory you could go inside to find more information. But for now, let's learn one of the most powerful tools of the shell: PIPES!

Pipes

Using the output of a command

So far we have been typing commands like ls or du or cat and reading the output in the shell. But what if I want to do something with it?

for example, after running du -h we saw something like this:

36K     ./.guides/bashtests
80K     ./.guides/content
728K    ./.guides/img
864K    ./.guides
0       ./js
0       ./nested-directories/nested-level-1/nested-level-2/nested-level-3
0       ./nested-directories/nested-level-1/nested-level-2
0       ./nested-directories/nested-level-1
0       ./nested-directories
0       ./test-website/images
4.0K    ./test-website
0       ./images
0       ./mystuff
0       ./config
1.5M    .

What if I need to know how many folders and sub-folders I have? Well, I could count it by hand, or I could ask a counting program to do it for me. This counting program is called wc:

Command: `wc`

Definition:
The wc command or word count command counts how many words, lines and characters are in a file. If you use the option wc -l it will only display the number of lines.

If you list your files with ls you should see a file called README.md. Run the command:

wc README.md

The output should look like this:

22 233 1704 README.md

Which means that the "README.md" file has 22 lines, 233 words and 1704 bytes. Ok. Fine. But what does this have to do with pipes?

Well, I said the pipe command is very powerful. Let's see how.

Command: `<input> | <output>`

Definition:
The | command or pipe command takes the output of what is left of it and passes it as input to whatever is on the right of it. The symbol you need to type is a vertical line (often found above the "\" symbol).

So my original question was to find out how many sub-directories I have. I printed du -h which showed me a really long list of all the sub-directories. But who has time for counting them all??? So here's what I want to do:

Take the output of the command du -h and pipe it to the counting command. Try it out!

du -h | wc -l

Now instead of looking at the mess that comes out of du -h we can see straight away that there are an exact number of lines in the output, and therefore that same number of directories.

Real hackers will use pipes all the time. for example:

ps -e | grep sshd | wc -l

which means:

ps -e: list all the processes in this computer
|: pipe the output of ps to the program grep
grep sshd: grep looks for the word sshd, and prints it as many times as it's found
|: pipe the output of grep into wc
wc -l Count the lines