Linux tips for web developers: Working with text files

By
Ryan Robinson

As developers, we often have to go to the command line to take a look at text files. Have you ever run cat on a giant text file and and waited five minutes for the whole thing to display? Have you ever sifted for a specific line in a log that is 99.9 percent full of stuff you don’t care about? Ever wanted to pull specific information out of multiple lines and put it in a new file easily? Well, I have some cool stuff to show you.

Cat, head, and tail

We've probably all used cat before. Running..


cat filename.txt

..will simply display the specified file, line by line, right in your terminal. Sometimes, these files are enormous, and we just want the first or last few lines.

To just display the firrst part of a file, why not use head?


head filename.txt

This will display the first 10 lines of your text file.

Conversely, running..


tail finename.txt

..will display the last 10 lines.

But, what if what you need isn't in the first or last 10 lines? Maybe it's in the last 150? We can add -x to head or tail, where x is the number of lines to display, to get what we need!


tail -150 finename.txt

This will display the last 150 lines.

Simple stuff, right? Let's dig deeper.

GREP

Say you want to search for a specific word or phrase in a file, without looking at it line by line manually. That's where grep comes in. Grep allows you to search any text you send to it and will only display what you're looking for. To use it this way, we'll use the pipe operator, |.


cat filename.txt|grep "word or phrase here"

We are piping the display of filename.txt to grep, which searches for our string. Despite the use of cat (or head, or tail for that matter), it will only display lines of text that contain your string.

You can even get the lines of text surrounding the line that contains your desired string! Placing -x next to grep, where x is the number of surrounding lines to display, will show you x number of lines before and after the line containing your string.


tail filename.txt|grep -5 "word or phrase here"

This is useful when searching for an event in a log file, and wanting to seeing all of the errors surrounding the event.

The pipe operator can also be used more than once. Let's say you want to search for a phrase that occurs only in lines (or it's surrounding lines) that contain another phrase. You can do it like this:


head filename.txt|grep -20 "phrase one"|grep "phrase two"

Finally, you may find it useful to output the results of your search to another file to reference later. To accomplish this, simply add a > and the desired filename.


cat filename.txt|grep "something" > newfile.txt

If you run into permission problems, you may need to try it as root.

And there you have it! For further command line text manipulation goodness, check out Linux commands sed and awk.