Input file:
$ cat cnr.txt
HIDDENHAUSEN|99.60
FIEBERBRUNN|99.07
MELLENDORF|99.04
HERBSTEIN|99.02
ACHTERWEHR|98.82
GOLM|98.82
PARA|98.82
BOGEN|98.61
SAINTANDRE|98.55
OLSZTYN|98.61
HYDERABAD|99.02
Output required: Print those lines for which 2nd field has occurred more than once. i.e. required o/p:
HERBSTEIN|99.02
ACHTERWEHR|98.82
GOLM|98.82
PARA|98.82
BOGEN|98.61
OLSZTYN|98.61
HYDERABAD|99.02
Awk solution:
$ awk 'NR==FNR && a[$2]++ {b[$2];next} $2 in b' FS="|" cnr.txt cnr.txt
Related posts:
- Difference between awk NR and FNR variables
- Posts on awk NR==FNR
- Awk FNR variable usage example
- Remove duplicates based on field using awk
- Remove duplicates from file without sorting using awk
UNIX BASH scripting
Dedicated to all BASH newbies and Linux one liner lovers. Useful AWK,SED,BASH one liners.
Thursday, July 9, 2009
Print all duplicate lines using awk
Wednesday, July 8, 2009
Split a string to characters in Bash
Input string:
1234 5678
o/p required:
1
2
3
4
5
6
7
8
Using -w option of Linux fold (wrap each input line to fit in specified width)
$ echo "1234 5678" | fold -w1
Awk solution : Make field separator (FS) as NULL so that each character represents a field
$ echo "1234 5678" | awk 'BEGIN{FS=""}{for(i=1;i<=NF;i++)print $i}'
Using bash:
#!/bin/sh
str="1234 5678"
while [ -n "$str" ]
do
printf "%c\n" "$str"
str=${str#?}
done
Using sed (Using & as matched string)
$ echo "1234 5678" | sed 's/[0-9 ]/&\n/g'
Tuesday, July 7, 2009
Close all windows in vimdiff - vi tip
vimdiff
a powerful file comparison tool (edit two or three versions of a file with Vim and show differences)
Two important window commands to close windows while performing vimdiff on multiple files. (Also applicable when you open multiple files using :split or :vsplit)
ctrl-w c - Close this window
ctrl-w o - close all Other windows (mnemonic - Only)
Two of vimdiff important commands:
do - (diff-obtain) obtain differences from other window into this one
dp - (diff-put) push differences in current window to the other window
Saturday, July 4, 2009
Remove duplicate lines from vi editor
In order to remove all duplicate lines (rows) from a file which is opened in vi editor,
Use the following command
:sort u
which will sort all lines and remove duplicates and will keep only unique lines.
Other solutions for removing duplicate lines from vi editor can be found here
Related posts:
- Remove duplicate consecutive fields or lines from file
- Remove duplicate blank lines using awk in bash
- Remove duplicates based on fields using awk
- Remove duplicate lines without sorting file using awk
Saturday, June 27, 2009
Print last section using awk RS variable
This post is mainly intended for the awk newbies.
Input file:
$ cat grn.log
Sat Jun 27 20:56:36 IST 2009 : Init
Sat Jun 27 20:56:39 IST 2009 : Phase1,mval=45
Sat Jun 27 20:56:46 IST 2009 : End
---
Sat Jun 27 21:06:15 IST 2009 : Phase1.4
Sat Jun 27 21:06:39 IST 2009 : Phase4,kval=23
Sat Jun 27 21:06:59 IST 2009 : Phase5,kval=29
Sat Jun 27 21:07:36 IST 2009 : End
---
Sat Jun 27 21:15:36 IST 2009 : Init
Sat Jun 27 21:16:29 IST 2009 : Phase1,mval=42
Sat Jun 27 21:16:46 IST 2009 : Cont
Output Required: Print the last section of the above file.
In general AWK reads one line at a time, and breaks up the line into fields.
We can print the last line this way:
$ awk 'END{print}' grn.log
o/p:
Sat Jun 27 21:16:46 IST 2009 : Cont
Now setting awk RS variable to "---", we can tell awk to treat a whole section as a line and then we can print the last section in the same way as above.
$ awk 'BEGIN{RS="---"}END{print}' grn.log
o/p:
Sat Jun 27 21:15:36 IST 2009 : Init
Sat Jun 27 21:16:29 IST 2009 : Phase1,mval=42
Sat Jun 27 21:16:46 IST 2009 : Cont
© Jadu Saikia http://unstableme.blogspot.com
