Print last section using awk RS variable

This post is mainly intended for the awk newbies.
Input file:

$ cat grn.log
Sat Jun 27 20:56:36 IST 2009 : Init
Sat Jun 27 20:56:39 IST 2009 : Phase1,mval=45
Sat Jun 27 20:56:46 IST 2009 : End
---
Sat Jun 27 21:06:15 IST 2009 : Phase1.4
Sat Jun 27 21:06:39 IST 2009 : Phase4,kval=23
Sat Jun 27 21:06:59 IST 2009 : Phase5,kval=29
Sat Jun 27 21:07:36 IST 2009 : End
---
Sat Jun 27 21:15:36 IST 2009 : Init
Sat Jun 27 21:16:29 IST 2009 : Phase1,mval=42
Sat Jun 27 21:16:46 IST 2009 : Cont

Output Required: Print the last section of the above file.

In general AWK reads one line at a time, and breaks up the line into fields.
We can print the last line this way:

$ awk 'END{print}' grn.log

o/p:
Sat Jun 27 21:16:46 IST 2009 : Cont

Now setting awk RS variable to "---", we can tell awk to treat a whole section as a line and then we can print the last section in the same way as above.

$ awk 'BEGIN{RS="---"}END{print}' grn.log

o/p:
Sat Jun 27 21:15:36 IST 2009 : Init
Sat Jun 27 21:16:29 IST 2009 : Phase1,mval=42
Sat Jun 27 21:16:46 IST 2009 : Cont

How to reuse a command in Bash command prompt

Suppose you have executed the following series of scripts(commands) for an argument say 12.34.

$ ./vali.sh 12.34 && ./sub.py 12.34 && rm /tmp/12.34.lck

Now if you need to repeat the same command, but this time for argument 11.3, then you can always go to command history (basically last command using your arrow keys), edit the 12.34 to 11.3 and you are done.

But, sometime if you don't want to take the risk of making any typo, you can change the argument in all the places using this:

$ !!:gs/12.34/11.3/

Which is basically:

./vali.sh 11.3 && ./sub.py 11.3 && rm /tmp/11.3.lck

Related posts:
- Accessing last argument of previous command
- Recall last argument of last command in bash

Use of bash until loop - shell script

Requirement:

- I had to execute a script "my.sh" 10 times.
- my.sh basically outputs loglines that to be redirected to log file.
- Once that log file reaches 5MB, kill my.sh and again execute my.sh (continue this for 10 times)

The script:

#!/bin/sh

for((i=1;i<=10;i+=1))
  do
      lfile=mylog.$i
      sh my.sh >> $lfile &
      Pid=$!

      sleep 2
      until [ $(ls -l $lfile | awk '{print $5}') -gt 5000 ]
          do
              echo "Sleeping for next 5 seconds"
              sleep 5
          done

      echo "Executed $i times"
      kill $Pid
done

Important use:
- Bash until loop

Padding zeros in file name using bash

Example 1
---------
Files in my current directory are:

$ ls | paste -
a.10.done
a.2.done
m.100.done
n.1.done

Required: Rename the above files to

a.0010.done
a.0002.done
m.0100.done
n.0001.done

i.e. make the "number part" of the filename equal width by padding the number with leading zeroes

The solution using awk and bash parameter substitution

for file in $(ls)
 do
   eval $(echo "$file"|awk -F'.' '{print "prefix="$1";sl="$2}')
   mv $file $(printf "%s.%04d.%s\n" "$prefix" "$sl" "done")
   echo "Renamed $file"
done

Reference:

- Read about awk and eval here and here
- A few posts on bash parameter substitution
- Why double brackets for arithmetic operations in bash ? Read here

Example 2
---------
Files in my current directory are:

$ ls | paste -
apl.1
apl.10
apl.100
apl.2
apl.5

Required: Rename the above files to

apl.0001
apl.0010
apl.0100
apl.0002
apl.0005

The solution:

for file in apl.*
 do
   psl='0000'${file#apl.}
   psl=${psl:(-4)}
   mv $file apl.$psl
done

Another way would be:

for file in apl.*
 do
   mv $file $(printf apl.%04d ${file#apl.})
done

Remove files without match using bash grep

I have already discussed in one of my earlier post how we can find the files which do not contain a particular pattern (string).

As I mentioned in that particular post:

"Well, when I first heard of this requirement; I just thought I will use "grep -vl pattern" piped with xargs to a regular find command (i.e. "find . -type f | xargs grep -vl pattern")
(-l : Suppress normal output; instead print the name of each input file from which output would normally have been printed.)

Then I realized that "grep -v" only works in line level, i.e. a file which contain the "pattern" in some line(s) may also contain some line(s) which "do not" contain that "pattern"; so as a whole that file will also be in the list of files which "do not" contain the "pattern" even though it contains the pattern."

From GREP(1) man pages:

-L, --files-without-match
Suppress normal output; instead print the name of each input file from which no out-put would normally have been printed. The scanning will stop on the first match.

-q, --quiet, --silent
Quiet; do not write anything to standard output. Exit immediately with zero status if any match is found, even if an error was detected. Also see the -s or --no-messages option.

So to remove or delete the files which do not contain the pattern or string "hello"

$ find . -type f -exec grep -L "hello" {} \; | xargs rm

or

$ find . -type f \! -exec grep -q "hello" {} \; -print | xargs rm