Fake Progress Bar in shell

Just to illustrate the use of "until" loop in BASH, here is a small script to generate a fake progress bar.

#!/bin/sh
clear
echo "Installation in Progress" [`basename $0`]
count=0
until [ $count -eq 10 ]
do
echo -n "###"
sleep 1
count=`expr $count + 1`
done
echo -n "[100%]"
echo

$ ./fakeprogress.sh
Installation in Progress [fakeprogress.sh]
##############################[100%]

Removing leading 0's

Suppose Number = 0000000231
Output Required = 231
i.e. we have to remove the leading zero's from the number.

$ echo "0000000231" | awk '{printf "%d\n",$1}'
231

$ echo "0000000231" | awk '{print $1 + 0}'
231

$ echo "0000000231" | sed 's/^0*//g'
231

Extracting top level dir names

A very simple post! This is for those who "just" started awk, others can skip :-)

Suppose dir.names contain the following data
$ cat dir.names
/usr/local/test
/usr/bin/test
/opt/ivoli/dat
/opt/lib/test
/tmp/files/test
/tmp/local/test

Purpose is to extract the top level directory names i.e.
/opt
/tmp
/usr

$ awk -F "/" '{print "/"$2}' dir.names | sort | uniq

Reverse the digits of a number : BASH beginner

It used to be among our college assignments, a good one for BASH beginners.

#!/bin/sh
#Sum of all digits in a number

num=12334
mod=0
rev=0

echo "Number= $num"
while [ $num -gt 0 ]
do
mod=`expr $num % 10`
rev=`expr $rev \* 10 + $mod`
num=`expr $num / 10`
done

echo "rev= $rev"

$ ./reverse.sh
Number= 12334
rev= 43321

Also a one liner replacement of the above script

$ echo "12334" | rev
43321

Sum of all digits of a number: BASH beginner

Here is my trick for the same.

$ echo "12334"
12334
$ echo "12334" | sed 's/[0-9]/ + &/g' ##mind the spaces before and after +
+ 1 + 2 + 3 + 3 + 4
$ echo "12334" | sed 's/[0-9]/ + &/g' | sed 's/^ +//g' ## Space bet ^ and +
1 + 2 + 3 + 3 + 4
$ expr `echo "12334" | sed 's/[0-9]/ + &/g' | sed 's/^ +//g'`
13

And this is the traditional script for the same.

#!/bin/sh
#Sum of all digits in a number

num=12334
tot=0
mod=0
echo "Number= $num"
while [ $num -gt 0 ]
do
mod=`expr $num % 10`
tot=`expr $tot + $mod`
num=`expr $num / 10`
done
echo "Sum= $tot"

$ ./sumofdigits.sh
Number= 12334
Sum= 13

Checking after every 5 seconds whether a user is log into the system or not

Again, sort of academic assignment question, but for a BASH learner, I feel will be useful.

#!/bin/sh

USER="build"
total=0

echo "Checking whether \"$USER\" is logged in or not"
echo "Starting at : `date`"
while :
do
who | grep $USER > /dev/null
[ $? = 0 ] && echo "Logged in" && exit
sleep 5
total=`expr $total + 5`
echo "User \"$USER\" Not yet logged in($total secs passed)"
done

$ ./checklo.sh
Checking whether "build" is logged in or not
Starting at : Fri Dec 14 09:48:17 IST 2007
User "build" Not yet logged in(5 secs passed)
User "build" Not yet logged in(10 secs passed)
User "build" Not yet logged in(15 secs passed)
Logged in

The same can also be checked using until loop

$ until who | grep build; do echo "Not yet" ;sleep 5; done; echo "Logged In"

"trap" ing Signals in shell script

Block Generation

Kind of a CHESS board block here.

#!/bin/sh
for m in `seq 8` ### Outer for loop ###
do
for n in `seq 8` ### Inner for loop ###
do
T=`expr $m + $n`
S=`expr $T % 2`
if [ $S -eq 0 ]; then
echo -e -n "\033[47m " ### white ###
else
echo -e -n "\033[40m " ### black ###
fi
done
echo -e -n "\033[40m" #### background colour=black ###
echo "" #### printing new line ###
done

AWK: Adding salary - Shell Beginner

$ cat sal.txt
Jex Mon clerk 12001
Aji Tue sales 13003
Jex Wed clerk 13123
Salna Thu sales 34000
Aji Mon sales 13123

Adding the Salaries for Jex Salna and Aji

$ awk '{arr[$1]+=$4} END {for (i in arr) {print i,arr[i]}}' sal.txt
Salna 34000
Jex 25124
Aji 26126

And if you want to see how many occurrences of each employee name(first field)

$ awk '{count[$1]++}END{for(j in count) print j,count[j]}' sal.txt
Aji 2
Jex 2
Salna 1

IP Addressing: Two

Link To : IP Addressing: One

CIDR

Subnet Mask CIDR
---------------------------------------------
255.0.0.0 /8
255.128.0.0 /9
255.192.0.0 /10
255.224.0.0 /11
255.240.0.0 /12
255.248.0.0 /13
255.252.0.0 /14
255.254.0.0 /15
255.255.0.0 /16
255.255.128.0 /17
255.255.192.0 /18
255.255.224.0 /19
255.255.240.0 /20
255.255.248.0 /21
255.255.252.0 /22
255.255.254.0 /23
255.255.255.0 /24
255.255.255.128 /25
255.255.255.192 /26
255.255.255.224 /27
255.255.255.240 /28
255.255.255.248 /29
255.255.255.252 /30

Q. IP: 210.10.10.54
SubnetMask: 255.255.255.0/24
N/W Address ??

Ans: N/w address: 210.10.10.0

Q. 220.92.92.5/24
Ans:
11011100.01011100.01011100.00000101 (Binary of 220.92.92.5) AND
11111111.11111111.11111111.00000000 (Binary of 255.255.255.0)
-----------------------------------------------------------
11011100.01011100.01011100.00000000

i.e: 220.92.92.0 (N/W address)

Subnetting Class C
-----------------------
255.255.255.128 /25 1000 0000
255.255.255.192 /26 1100 0000
255.255.255.224 /27 1110 0000
255.255.255.240 /28 1111 0000
255.255.255.248 /29 1111 1000
255.255.255.252 /30 1111 1100

Ex1: (255.255.255.192/26)

Subnet the n/w address 192.168.10.0
Subnet Mask 255.255.255.192

Ans:
------
n/w address: 192.168.10.0
@How many subnets?
192 is 1100 0000, 2 bits on
i.e 2^2=4 subnets
@How many hosts per subnet?
6 bits off,
i.e (2^6-2)=62 hosts
@Valid subnets?
256-192=64
i.e subnets are 0,64,128,192

n/w' 'Broadcast 'Range
192.168.10.0 192.168.10.63 0-63
192.168.10.64 192.168.10.127 64-127
192.168.10.128 192.168.10.191 128-191
192.168.10.192 192.168.10.255 192-255

Finding an unused port

#!/bin/sh
#To find an unused port

PORT=3306
quit=0

while [ "$quit" -ne 1 ]
do
netstat -a | grep $PORT >> /dev/null
if [ $? -ne 0 ]
then
quit=1
else
PORT=`expr $PORT + 1`
fi
done
echo "Free Port is $PORT"

Printing from 1-100 using BASH for loop

This sounds very easy. But for me when I tried, I was in a confusion, as for me BASH for loop is of the form

for i in 1 2 3 4 5

in that sense I had to write

for i in 1 2 3 4 5 6 7 8 . . . . 100 (from 1 to 100, which is a very poor way!!), I never knew that we can write the same this way

for ((i=1;i<101;i++))
do
echo $i
done

even in one more way

for i in `seq 100`; do echo $i; done

I found this very helpful.

Command in discussion - seq

seq is a very useful command to generate sequence of numbers. Some of its uses are discussed below.

(Printing from 1 to 5)
$ seq 5
1
2
3
4
5

(Printing from 2 to 5)
$ seq 2 5
2
3
4
5

(Printing from 2 to 12, with 3 increment,FIRST INCREMENT LAST)
$ seq 2 3 12
2
5
8
11

(-s, --separator)
$ seq -s : 1 5
1:2:3:4:5

(Adding 1 to 10)
$ seq -s " + " 1 10
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10

$ expr `seq -s " + " 1 10`
55

(-w, --equal-width)
$ seq -s : -w 1 10
01:02:03:04:05:06:07:08:09:10

(Use with for loop)
$ for i in `seq 5`
> do
> echo "square of $i is `expr $i \* $i`"
> done
square of 1 is 1
square of 2 is 4
square of 3 is 9
square of 4 is 16
square of 5 is 25

which is a replacement of :
1) for i in 1 2 3 4 5
or
2) for ((i=1;i<=5;i++))

Writing Case - Shell Begineer

This is for them, who are very new to Shell Prog, just an example to see the use of case, hope this helps :-)

#!/bin/sh
#To find the day

day=`date| awk '{print $1}'`

modify()
{
case $1 in
Sun) echo "Sunday";;
Mon) echo "Monday";;
Tue) echo "Tuesday";;
Wed) echo "Wednessday";;
Thu) echo "Thursday";;
Fri) echo "Friday";;
Sat) echo "Saturday";;
*) echo "Not Estimated";;
esac
}

echo "Today is "`modify $day

Solaris: Virtual Interface Adding/Removing

(Source : http://www.tech-recipes.com/solaris_networking_tips35.html)

Virtual interfaces allow a single Ethernet interface to listen on additional IP addresses.

Given an ethernet interface hme0 (use ifconfig -a to identify the names of your interfaces), you can create a subinterface called hme0:1 with the following command:
# /sbin/ifconfig hme0:1 plumb

You can set the IP address of the interface to 172.22.23.34 and turn on the interface with the following command:
# /sbin/ifconfig hme0:1 172.22.23.34 up

# /sbin/ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
hme0: flags=1000843 mtu 1500 index 2
inet 172.22.23.39 netmask ffffff00 broadcast 172.22.23.255
ether 9:0:21:aa:be:5f
hme0:1: flags=1000843 mtu 1500 index 2
inet 172.22.23.34 netmask ffff0000 broadcast 172.22.255.255

Unless you do some additional nonstandard things in your network, all of the subinterfaces on a physical interface need to be in the same subnet.

To make the virtual interface persist following a reboot, you can add the ip address or hostame from /etc/hosts in the file /etc/hostname.hme0:1


Removing the virtual interface from Solaris:
(Source: http://www.tech-recipes.com/solaris_networking_tips139.html)

Solaris allows multiple virtual or logical interfaces to exist with different IP addresses on the same physical interface. Virtual interfaces are created or plumbed up using the plumb directive in ifconfig. They are unplumbed using the unplumb directive.

To remove the virtual interface eri1:7, run the following command as root:
# ifconfig eri1:7 unplumb

Printing only entries those are inside "double quotes"

Printing only entries those are inside "double quotes"

$ echo "I \"see\" dead \"people\""
I "see" dead "people"

$ echo "I \"see\" dead \"people\"" | sed 's/[^"]*"\([^"]*\)"[^"]*/\1 /g'
see people

IP Addressing: One

Binary to decimal conversion
-----------------------------------
1000 0000 = 128
1100 0000 = 192
1110 0000 = 224
1111 0000 = 240
1111 1000 = 248
1111 1100 = 252
1111 1110 = 254
1111 1111 = 255

Hex to decimal conversion
------------------------------
A 10
B 11
C 12
D 13
E 14
F 15

Coversion:
----------------
128 64 32 16 8 4 2 1
192(128+64) 1 1 0 0 0 0 0 0
240(128+64+32+16) 1 1 1 1 0 0 0 0
** Put 1 in the corresponding places, else 0

IP Addressing:
-----------------
1)Numeric identifier assigned to each machine on the n/w
2)Designates the specific location of a device in the n/w
3)S/w add(logical add) hard coded in NIC
4)used for routing in n/w layer
5)32 bit

Terminilogy
-------------
1)bit(1/0)
2)byte(8bit)
3)Octet(8bits grouping)

N/W Address:
-----------------
Designation used in routing to send pkts to a remote n/w
All 0 in host part
e.g.: 10.0.0.0/172.16.0.0/192.168.0.0
Broadcast address:
---------------------
-Used by applications and hosts to send info to all nodes on a n/w
-All 1's in host part
-Routers will do bradcast only when it has an update in its routing table.
-e.g:
255.255.255.255: all n/w + all nodes
172.16.255.255: all subnets and hosts on n/w 172.16.0.0
10.255.255.255: broadcast to all subnets and hosts on n/w 10.0.0.0

** IP addressing : hierarchial
MAC: flat

3 methods:
-------------
-32 bits

@ Dotted decimal: 172.16.30.56
@ Binary: 10101100.********.********.********
@Hex: AC:10:1E:38

Classification:
---------------
ClassA: N.H.H.H
ClassB: N.N.H.H
ClasssC:N.N.N.H
ClassD:Multicast
ClassE: Reasearch

**Unicast/braodcast/multicast
ClassA:
---------
First bit of first byte n/w add must always be off(0)

i.e. 0XXX XXXX
0000 0000::: 0
0111 1111::: 127
(0-127)

ClassB:
----------
First bit of first byte n/w add must always be 1 Second bit always off(1) i.e.
1000 0000:::128
1011 1111:::191
(128-191)

ClassC:
----------
First 2 bits 1
3rd bit 0
i.e.
1100 0000:::192
1101 1111:::223
(192-223)

ClassD & E
-----------
(224-255)
224-239: ClassD: Multicast
240-255: ClassE (Reserved)

Reserved IP:
---------------
1) N/W add of all 0s (this n/w)
2) N/W add of all 1s (all n/w)
3) N/W 127.0.0.1 (reserved for loopback testing, allows the node to send a test pkt to itself without generating n/w traffic)
4) Node add of all 0s (n/w address)
5) Node address of all 1s(all nodes, 128.2.255.255 means all nodes on n/w 128.2(B))
6)Entire IP add set to all 0s (used by cisco router to designate the defaulr route)
7)Entire IP add set to all 1s (Broadcast to all nodes on the current n/w)
all 1s broadcast (limited broadcast)
255.255.255.255

Class A Address:
--------------
N.H.H.H

e.g: 49.22.102.70 (49:: n/w address, 22.102.70::node address)

@NW add of all 0's (0000 0000) is reaserved.
@Address 127 is reserved for diagnostic

i.e. we can use 1-126 to designate class A n/w
i.e Actual number of usuable class A n/w add=128-2=126

24 bits for node address
i.e 2^24=16777216 unique combinations

case 4 and case 5:: Actual number: 16777212

(A huge number)

Valid host IDs :

All host bit off:: n/w address:: 10.0.0.0
All host bit on:: broadcast address: 10.255.255.255

i.e between 10.0.0.1<=>10.255.255.254

Class B address:
-------------------
N.N.H.H

e.g: 172.16.30.56 (172.16:: n/w address; 30.56:: node address)

2^16 unique combinations of n/w address
but class B n/w address to start with 1/0, so 14 positions to play
i.e 2^14=16384 unique class B n/ws

Node: 2^16-2=65534 node address

valid host ID:

All host bit 0:: n/w address: 172.16.0.0
All host bit 1:: broadcast:: 172.16.255.255

i.e 172.16.0.1<=>172.16.255.254

ClassC address:
------------------
N.N.N.H

e.g:
192.168.100.102 (n/w address is 192.168.100; Node add:: 102 )
as first 3 bit portions are always 110

i.e 24-3=21:: 2^21=2097152 n/w

2^8-2=254 node address

valid host id:
192.168.100.1 <=> 192.168.108.254

Private IP address:
------------
can be used in a primvate n/w
not routable through internet

CLASS A: 10.0.0.0 _ 10.255.255.255
Class B: 172.16.0.0._ 172.31.255.255
Class C: 192.168.0.0 _ 192.168.255.255

Subnetting:
--------------
Subnet address:
----------------
@ Every machine on the network must know which part of host address will be used as subnet add. This is done by assigning a subnet mask to each machnie.
@32 bit value that allows the receipnt of the IP packet to distinguish the n/w ID portion of IP add from the host ID portion of IP address.

default:
A 255.0.0.0/8
B 255.255.0.0/16
C 255.255.255.0/24

e.g:
255.255.0.0/21
1111 1111.1111 1111.1111 1000.0000 0000
255.255.248.0

CIDR:Classless Inter-Domain Routing
---------------------------------------------------
CIDR is a new addressing scheme for the Internet which allows for more efficient allocation of IP addresses than the old Class A, B, and C address scheme

Why Do We Need CIDR?
------------------------
With a new network being connected to the Internet every 30 minutes the Internet was faced with two critical problems:

@Running out of IP addresses
@Running out of capacity in the global routing tables

Link to IP Addressing: Two

Some useful SED regular exps

Just a few important ones,

A) /[0-9]\{3\}/p # print lines with 3 consecutive digits

$cat newfi
jadu123asd
12jksa
123jks
1234jksas

$sed -n '/[0-9]\{3\}/p' newfi
jadu123asd
123jks
1234jksas

B) /regards/!s/this/that/ # unless 'regards' is found, replace 'this' with 'that'
C) 9!s/this/that/ # Except on line number 9, replace 'this' with 'that'
D) 3,/this/d # delete all lines from line 3 up to 'this'

Writing log functions in BASH

Simple way of writing log functions in a shell script.

#!/bin/sh
....
....
LOGFILE=./log.`date +%s`.out

#LOG functions
f_LOG() {
echo "`date`:$@" >> $LOGFILE
}

f_INFO() {
echo "$@"
f_LOG "INFO: $@"
}

f_WARNING() {
echo "$@"
f_LOG "WARNING: $@"
}

f_INFO "Checking 0 kb files"
...
... calculation for 0 kb file checking
...
f_WARNING "There are $BLFILES 0 kb files found, this can cause problem sometime, still continuing .."


Now
$ cat log.1194832036.out
Mon Nov 12 07:17:16 IST 2007:INFO: Checking 0 kb files
Mon Nov 12 07:17:16 IST 2007:WARNING: There are 3 0 kb files found, this can cause problem sometime, still continuing ..

Check whether $ENV_VAR is set or not

If you want to see whether a particular env var is set or not set, write something like this in your script

$ if (set -u; : $HOME) 2> /dev/null
> then
> echo "HOME env is set"
> else
> echo "Not"
> fi

A simple alternative

$ [ ! -z $HOME ] && echo "set" || echo "not"
set

SED and variable access

Q: Why don't my variables like $var get expanded in my sed script?

Ans : Instead of the 'single quotes' we need to use "double quotes" , unix shells never expand $var in single quotes
so

$ S="asd"; echo "asd_2310" sed "s/2310/$S/g"
asd_asd

Accessing Env Variable in AWK

Suppose we have a file with some statistics for users in the system.

$ cat oot.txt
user5|23|killall|23-10-2007
user2|21|rm -rf|23-10-2007
user3|0|killall|23-10-2007
user8|12|ups_sync|23-10-2007
user6|23|killall|23-10-2007

The logged user is "user8"
$ echo $USER
user8

Now to print the line for the logged user, we have to use ENVIRON

$ awk -F "|" '$1==ENVIRON["USER"] {print $0}' oot.txt
user8|12|ups_sync|23-10-2007