Split a line into lines - AWK or SED

We can use "jot" to print A-Z like below:

$ jot -c 26 A | tr '\n' ' '
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

The required Output:

A B C D
E F G H
I J K L
M N O P
Q R S T
U V W X
Y Z

i.e in one line, 4 alphabets.

Solution:

4 alphabets means including spaces, its 8 characters in one line.

$ jot -c 26 A | tr '\n' ' ' | sed -e "s/.\{8\}/&\n/g"

One solution using AWK:

$ jot -c 26 A | tr '\n' ' ' | awk 'BEGIN{n=1}{while(substr($0,n,8)){print substr($0,n,8);n+=8}}'
A B C D
E F G H
I J K L
M N O P
Q R S T
U V W X
Y Z

How to check folder size in linux - Linux Newbie

Question: What command do I need to execute to know the size of a folder, including its subdirectory?

Ans: To calculate the total folder size of folder "bin"

$ du -sh bin/
280K bin/

-s, --summarize (display only a total for each argument)
-h, --human-readable ( print sizes in human readable format (e.g., 1K 234M 2G))

Time between two dates - using BASH

This was posted in http://www.unix.com/shell-programming-scripting/

---------------------------------------------------------
Hi,

Please help me out in creating a script which will subtract Tue, Feb 26, 2008 01:38:25 AM from Mon, Feb 25, 2008 09:30:03 PM and will give me out put in the format hh:mm:ss.

I tried datecalc.It doesnt work for this format.
----------------------------------------------

And here is my solution: The idea is to convert both the times to epoch and get the diff in seconds and then convert the seconds to hh:mm:ss format using awk.


$ cat datediff.sh
#!/bin/sh

D1=`date +%s -d "Feb 26, 2008 01:38:25 AM"`
D2=`date +%s -d "Feb 25, 2008 09:30:03 PM"`
((diff_sec=D1-D2))
echo - | awk '{printf "%d:%d:%d","'"$diff_sec"'"/(60*60),"'"$diff_sec"'"%(60*60)/60,"'"$diff_sec"'"%60}'

$ sh datediff.sh
4:8:22

Sequence subtration, one liner

The file myfile.out contains a series of digits. The purpose is subtract 1st number from 2nd number(2nd-1st),3rd number minus 2nd number and so on

$ cat myfile.out
10
30
5
802
96

i.e
$ sed -e 's/[0-9]*/&\n&/' myfile.out | sed -e '1d' -e '$d' | tac | paste -sd"- " | tr ' ' '\n' | tac | sed -e 's/-/ & /' -e 's/[0-9].* - .[0-9]*/echo `expr &`/'
echo `expr 30 - 10`
echo `expr 5 - 30`
echo `expr 802 - 5`
echo `expr 96 - 802`

The breakdown in each of the pipes:

Adding a duplicate of each digit
$ sed -e 's/[0-9]*/&\n&/' myfile.out
10
10
30
30
5
5
802
802
96
96

Deleting 1st and last line
$ sed -e 's/[0-9]*/&\n&/' myfile.out | sed -e '1d' -e '$d'
10
30
30
5
5
802
802
96

Reversing
$ sed -e 's/[0-9]*/&\n&/' myfile.out | sed -e '1d' -e '$d' | tac
96
802
802
5
5
30
30
10

Making the format using "paste" command
$ sed -e 's/[0-9]*/&\n&/' myfile.out | sed -e '1d' -e '$d' | tac | paste -sd"- "
96-802 802-5 5-30 30-10

Replacing each space with newline
$ sed -e 's/[0-9]*/&\n&/' myfile.out | sed -e '1d' -e '$d' | tac | paste -sd"- " | tr ' ' '\n'
96-802
802-5
5-30
30-10

Again reversing
$ sed -e 's/[0-9]*/&\n&/' myfile.out | sed -e '1d' -e '$d' | tac | paste -sd"- " | tr ' ' '\n' | tac
30-10
5-30
802-5
96-802

Formating
$ sed -e 's/[0-9]*/&\n&/' myfile.out | sed -e '1d' -e '$d' | tac | paste -sd"- " | tr ' ' '\n' | tac | sed -e 's/-/ & /' -e 's/[0-9].* - .[0-9]*/echo `expr &`/'
echo `expr 30 - 10`
echo `expr 5 - 30`
echo `expr 802 - 5`
echo `expr 96 - 802`

Now

$ sed -e 's/[0-9]*/&\n&/' myfile.out | sed -e '1d' -e '$d' | tac | paste -sd"- " | tr ' ' '\n' | tac | sed -e 's/-/ & /' -e 's/[0-9].* - .[0-9]*/echo `expr &`/' > myfile1.out

Executing myfile1.out
$ sh myfile1
20
-25
797
-706

ohh! it was a long one liner, good to learn the uses of different sed operations. The same can be achieved in a very traditional way:

FILE=./myfile.out
N=1
total=`sed -n '$=' $FILE`
until [ "$N" -eq $total ]
do
S1=$N
S2=`expr $N + 1`
N=$S2
VAL1=`sed -n "$S1 p" $FILE`
VAL2=`sed -n "$S2 p" $FILE`
expr $VAL2 - $VAL1
done

Number of files modified in each month - AWK

I had a log dir where one of my applications used to dump logs. The number of files in the dir is more than 1000's and I had to count the number of files been modified in each month. This is a one liner for the same.

$ ls -l | sed 1d | awk 'BEGIN{print "Month (Num Files)"}{count[$6]++}END{for(j in count) print j,"("count[j]")"}'

Output:

Month (Num Files)
Feb (100)
Apr (341)
Jan (921)
Mar (343)

The same logic can be used to count the number of occurrences of a particular field in a file. e.g.

$ cat myfile
1
1
3
3
3
3
5
6
6
6

If one had to calculate how many times each of the digits occurred.

$ awk 'BEGIN{print "Number (count)"}{count[$1]++}END{for(j in count) print j,"("count[j]")"}' myfile

Output:
Number (count)
5 (1)
6 (3)
1 (2)
3 (4)

grep and put the SL no - AWK Newbie

The purpose is very simple, I had to find all people names(1st field) from location(3rd field) "Ind", with a SL No in beginning. i.e. if the input files are like this:

$ cat file1.txt
Hanr:3232:Ind:/tmp/da02.out
Ajith:2211:Ind:/opt/ds.out
Rein:2221:Jpn:/tmp/ls.out
Leela:1198:Ind:/

$ cat file2.txt
Olea:5454:Swd:/opt/ert.tmp
Lola:2129:Chn:/home/Lola
Nina:8784:Ind:/opt/ls.out
Rahim:6551:Ind:/tmp/as.out

The output required is :
1 Hanr
2 Ajith
3 Leela
4 Nina
5 Rahim

Here is the solution:
$ awk -F ":" '$3=="Ind" {c++;print c,$1}' file1.txt file2.txt

Finding common numbers between two files - BASH

Simple, but very^2 important.

$ cat file1
123
123
321
342
546
523
652

$ cat file2
143
767
984
123
342

Sort the files in ascending order:

$ sort -o file1.srt file1
$ sort -o file2.srt file2

----------------------------------
$ comm --help
Usage: comm [OPTION]... LEFT_FILE RIGHT_FILE
Compare sorted files LEFT_FILE and RIGHT_FILE line by line.

-1 suppress lines unique to left file
-2 suppress lines unique to right file
-3 suppress lines unique to both files
--help display this help and exit
--version output version information and exit

Report bugs to .
----------------------------------

$ comm -12 file1.srt file2.srt
123
342

or

Without sorting, we have an easy option with "grep"

$ grep -f file1 file2
123
342

Renaming dos file names to unix format - BASH newbie

Thought of sharing this with all of you, a simple way of renaming dos format file names, replacing all spaces in file name with "_".

$ ls -lrt
total 1572
-rw-r--r-- 1 jsaikia sta 529402 Feb 11 12:33 dos file1.txt
-rw-r--r-- 1 jsaikia sta 529402 Feb 11 13:38 dosfile 2.txt
-rw-r--r-- 1 jsaikia sta 529402 Feb 11 13:38 dos file 3.txt

$ ls | while read file; do mv "$file" `echo "$file" | tr ' ' '_'`; done

$ ls -lrt
total 1572
-rw-r--r-- 1 jsaikia sta 529402 Feb 11 12:33 dos_file1.txt
-rw-r--r-- 1 jsaikia sta 529402 Feb 11 13:38 dosfile_2.txt
-rw-r--r-- 1 jsaikia sta 529402 Feb 11 13:38 dos_file_3.txt

Why the "" with $file is important ?

See this small example:
$ ls -l
total 8
-rw-r--r-- 1 jsaikia sta 23 Feb 11 13:49 dos file 2.txt
-rw-r--r-- 1 jsaikia sta 23 Feb 11 13:49 dos file1.txt

$ cat dos\ file1.txt
this is a dos file (1)

$ cat dos\ file\ 2.txt
this is a dos file (2)

Now if I write a small one liner to cat the contents of all the files.

$ ls | while read file; do
> cat $file
> done
cat: dos: No such file or directory
cat: file: No such file or directory
cat: 2.txt: No such file or directory
cat: dos: No such file or directory
cat: file1.txt: No such file or directory

Now adding "" to $file variable.

$ ls | while read file; do cat "$file"; done
this is a dos file (2)
this is a dos file (1)