Remove leading zero from line - awk, sed, bash

Input file:

$ cat ids.txt
00009
01902
34390
00190
00001

Required: Remove the leading zero's from each of the lines in the above file.

Using sed:

$ sed 's/^[0]*//' ids.txt

Output:
9
1902
34390
190
1

So in vi ex mode, the command for the same will be:

:1,$ s/^[0]*//

Other alternatives:

$ awk '{printf "%d\n",$0}' ids.txt
$ awk '{print $1 + 0}' ids.txt
$ cat ids.txt  | bc

Using bash parameter substitution:

$ shopt | grep extglob
extglob         off

# Set this option on
$ shopt -s extglob
$ i=000100
$ echo ${i##+(0)} 
100

extglob shell option in bash:
If set, the extended pattern matching features are enabled

Now if you are thinking how we can add zero's at beginning of a bash variable, here is the way using printf:

$ printf "%010d\n" 00005
0000000005

You might look at this post which shows how we can make some numbers equal width by padding the number with leading zeroes.

Some related posts:
Replace leading zero's with blank using sed
Remove white space in vi editor

Simple bash script to parse log file

A simple bash script; can be useful for bash newbies.

Input file 'log.txt' is a log file and is of the following format:

$ cat log.txt
2010-04-06 08:06:01 INFO Start ....
2010-04-06 08:06:02 INFO Consuming file trp.1270540317.0.in
2010-04-06 08:06:02 INFO Consuming file trp.1270540326.2.in
2010-04-06 08:06:03 INFO Consuming file trp.1270540341.6.in
2010-04-06 08:06:03 INFO Consuming file trp.1270540367.0.in
2010-04-06 08:06:04 INFO End ....

Required: For each file (trp.<epoch>.<id>.in) entry in the above file, calculate the difference in seconds between this "epoch" time-stamp and the time the file is processed.
e.g.
for this entry:

2010-04-06 08:06:02 INFO Consuming file trp.1270540317.0.in

Calculate this:

(2010-04-06 08:06:02)-1270540317

The bash script:

#!/bin/sh

QFILE=log.txt

grep "INFO Consuming file" $QFILE | while read line
    do
        filename=$(echo "$line" | awk '{print $NF}')
        PT=$(date +%s -u -d "${line%INFO*}")
        AT=$(echo "$line" | awk '{print $(NF-2)}' FS=\.)
        ((diff_sec=PT-AT))
        echo "$filename $diff_sec"
done

Output:

$ ./show-diff.sh
trp.1270540317.0.in 845
trp.1270540326.2.in 836
trp.1270540341.6.in 822
trp.1270540367.0.in 796

Some related posts:

• Parse log file and calculate query response time using awk in bash
• Bash while loop sum issue explained
• Ways of writing bash for loop
• Extract sub string from variable in Bash
  

Vi editor copy and paste block example

"file.txt" is of the following format:


File "dummy.txt" contains the following lines:

$ cat dummy.txt
item2
item5
item9
item1

Required: In Vi, Copy the above lines from "dummy.txt" and paste to "file.txt" in the following format:






The solution:

In vim:

• In "dummy.txt", press 'Ctrl v' to go to the "Visual Block" mode.
• Select the required block from "dummy.txt" using the keyboard arrow keys and then press "y" for copy or "x" to cut the selected block to buffer.
• In "file.txt" move cursor to the location you want the paste the copied block (in this example its Row 3 Column 23) and in command mode press P (capital P)
• Done !

Related posts:

• Vim copy paste indentation issue solution
• Vi editor tip : Pull word under cursor
• Start a new shell from vi editor