Introduction

Back to: Linux command line and the Shell free course beginner to expert by ProGuide365.

Have you ever needed to transform data from one format to another? Transforming data is a common task for developers. Whether you’re trying to extract some information from some logs or incoming streams, you’ll often need to deal with raw data and transform it before it’s usable.

In this module, you’ll learn how to wrangle (transform) data by using the UNIX shell. You’ll be introduced to some commonly used tools and also to the basics of regular expressions.

Learning objectives

In this module, you’ll learn how to:

• Do basic file inspection with commands such as head, tail, wc , nl, and sort.
• Use cat to create, append, display, and concatenate files.
• Write and use simple regex (regular expressions) for text pattern matching. (This introduction to regex isn’t in depth.)
• Use grep to search files or stdin for pattern matching along with regex.
• Do basic text transformations on input streams by using sed and regex.
• Use pipes and filters for data wrangling.

Prerequisites

• Basic knowledge of the command line and basic shell commands