6 Linux command Examples and One liners Every Programmer Should Remember

Hello guys, Linux is an important skill for any developer or DevOps engineer becuase most of the server-side applications run on Linux boxes. At least you should be familiar with essential Linux commands like ls, cat, ps, top, find, grep, awk, sort, uniq, kill, xargs, curl, lsof, etc. I have shared a lot of useful Linux tutorials in this blog and you can take a look at them to learn about those commands in detail. Today, I am going to share some of the useful Linux one-liners you can use in your day-to-day life. These one-liners are either example of one single command or multiple commands used together to do a certain task like finding duplicate rows in a text file or counting how many times each IP address appear in a text file, much like group by clause of SQL.

These commands will help you during troubleshooting, debugging, and data analysis. For example, I have extensively used grep, cut, sort, and uniq combinations to find out which clients are connecting to our server-side Java application and then pro-actively inform them to move off before we decommission our server and move to the cloud.

The client really appreciated our reports which proactively identified connections from their hosts, which even didn't know. That's the power of Linux commands, they are really a must-know tool not just for Java developers but for any programmers.

While there is no end to creativity and how you use a combination of Linux command to solve a problem or extract a piece of information you know, I will share 6 frequently used Linux one-liners which range from finding duplicate rows in a text file to removing ctrl+ M characters from a file which often creep in when you copy a file from Windows to UNIX.

Basic knowledge of Linux commands is necessary to understand these one-liners but if you are a complete beginner then you should first go through a Linux fundamental course like Linux Command line basics, which will teach you each of these commands used here together to do a meaningful task. It's a very comprehensive yet affordable course and you can buy in just $10 on Udemy flash sales which happen every now and then.

Linux Command One-Liners Examples for Programmers

Without wasting any more of your time, here is my list of some of the useful Linux command one-liners for programmers, software engineers, cloud engineers, and system admins working in the Linux environment.

1. How to Find Duplicate Rows in a Text File 

If you have a text file, you can find lines that are duplicated by running

$ sort file | uniq -d

By default, the uniq command takes a sorted list and prints each line once. If you add the -d option, it only prints lines that occur more than once.

For example,

file1
-------
hello
bonjour
bonjour 
hello
nihao
namaste

If you run uniq -d file1, you get:

$ uniq -d file 1
bonjour

This is because the only bonjour occurs more than once in a row or consequently.

So, to get all duplicates, you need to first sort the file so that all duplicate will come one after other and then you can run sort file1 | uniq -d, which returns:

$ sort data.txt | uniq -d
hello
bonjour

This one was a very simple example of how you can combine two Linux commands to do some meaningful tasks. If you want to learn more about the sort and uniq command then I suggest you join  The Linux Mastery: Master the Linux Command Line in the 11.5 Hour course on Udemy. This is a project-based course where you will do many such tasks to understand not just these two commands but others like find, grep, awk, and sed.

2. Linux command to remove Ctrl M characters from a file

Strip the CTRL-M (^M), ASCII octal 015, from line-endings from a file. This is useful when DOS (Microsoft Windows etc) originated files are moved to UNIX.

$ tr -d '\015' < filename > outputfilename

If you want to learn more about Ctrl-M characters and how they creep into your file while copying from Windows and Linux then check out my earlier post about how to remove Ctrl+M characters from a file in Linux.

3. How to search the jar files in a CLASSPATH for a given Java package name (or class name):

This is a very useful one-liner for Java developers working in the Linux machine. You can use this command to troubleshoot classpath related issues and solving those ClassNotFoundException and NoClassDefFoundError:

$ printenv CLASSPATH | tr : '\n' | grep -i .jar 
                     | xargs -tI {} jar -tf {} | grep org.springframework

To search for something in the path, like all occurences of "dev":

$ printenv PATH | awk 'BEGIN {RS=":"} /dev/ {print}'

Two ways to look at the PATH (or CLASSPATH etc) with one item on each line:

$ echo $PATH | tr ":" "\n" | grep dev
$ printenv PATH | awk 'BEGIN {RS=":"} {print}'

4. Selective removal of files

I wanted to remove a large number of files from a directory. However, I did not want to descend into subdirectories, nor did I want to remove any .pdf or .chm files. Some of the files had non-standard characters (such as quotes and spaces) in them.

The solution? A simple one-liner using find and xargs

$ find . \( ! -name ".pdf" -a ! -name ".chm" \) 
       -maxdepth 1 -type f -print0 | xargs -0 rm

If you have trouble understanding these commands becuase of several find command options, then I also suggest you check out Linux Command Line Interface (CLI) Fundamentals course on Pluralsight,  a great course to learn essential Linux commands for programmers and developers.

By the way, you would need a Pluralsight membership to join this course which costs around $29 per month or $299 per year (14% discount). I highly recommend this subscription to all programmers as it provides instant access to more than 7000+ online courses to learn any tech skill. Alternatively, you can also use their 10-day-free-pass to watch this course for FREE.

5. Linux command to remove empty files

It's very common to remove empty lines from a file. If you are also looking for the solution to that same problem here is one simple one-liner to remove empty files older than 7 days

$ find . -type f -mtime +7 -ls | awk '$7 == 0 { print $NF }' | xargs rm

6. Doing group by and count using uniq -c command

If you have a text file, you can find how many times each word is repeated just like the SQL group by clause using the uniq - c command.  For example, you have run a web application and you want to find out which hosts are connecting to it.

You can grep hostnames or IP address from logs and then dump them into a text file and then you can use the sort and uniq command to find out which host is sending the most connection and other numbers. This is very useful while troubleshooting the DOS attack or when your application has a burst of a client connecting to it.


For example,

file1
-------
10.11.255.19
10.11.255.19
10.11.255.19
10.11.255.20

if you run uniq -d file1, you get:

$ sort file | uniq -c
10.11.255.19  3
10.11.255.20  1

That's all about these useful Linux command examples and one-liners. They are worth bookmarking and you will find them need every now and then. As I have said, Linux command line knowledge is essential for any Programmer, system admins, and software professionals. If you are new to a tech career, I suggest you spend some time learning command-line tools like bash and other fundamental Linux commands. 



Related UNIX Command Tutorials
If you are a Java developer often working in Linux or UNIX environment then you will also find the following tutorials useful:

• 10 examples of find command in UNIX (examples)
• How to create, update and delete soft link in UNIX (command)
• 10 examples of grep command in UNIX (examples)
• How to get an IP address from the hostname and vice-versa in Linux (command)
• 10 examples of xargs command in Linux (examples)
• How to delete empty files and directory in UNIX (solution)
• 10 examples of date command in Linux (examples)
• How to make a directory tree in one command? (example)
• 10 examples of tar command in UNIX (examples)
• UNIX command to find out how long a process is running? (answer)
• 10 examples of Vim in UNIX (examples)
• How to how long argument of a process in Solaris (command)
• 10 examples of chmod command in UNIX (examples)
• UNIX command to find the size of the file and directory? (command)
• 10 tips for working fast in UNIX? (tips)
• 5 examples of kill command in Linux (examples)

Thanks for reading this article. If you find these UNIX command examples and one-liner useful then please share with your friends and colleagues. If you have any other interesting UNIX or Linux one-liner then please share with us. 

P. S. - If you want to learn Linux and looking for some free resources like books and online courses, then you can also check out this list of free Linux courses for Programmers and IT Professionals. This list contains some of the best free courses from Udemy, Pluralsight, Coursera, CodeCademy, and other online platforms.