In this module, you were introduced to some tools and commands that you can use on the UNIX shell to transform your data from one format into another.

You were first introduced to the commands headtailwc, and nl. Using these commands will help you to do quick inspections of your text files and get a better understanding of the raw data. The next command you were introduced to is cat. This command allows you to peek into the contents of your files. You can also use it to create new files by appending or concatenating files’ contents.

Then, you learned the basics of regular expressions, also known as regex. Regular expressions are useful for pattern matching and can be used in IDEs and on the command line with other filters and commands.

You were then introduced to sed and grep, which can be used to search for and replace patterns in text streams and files.

Finally, you learned about pipes and output redirection. Using pipes allows you to put together all the commands and filters you learned in the previous units to build data processing pipelines.

You should now have a good understanding of what these commands do and how to use them to transform data efficiently from the UNIX shell.