Linux Commands Memo

This is a linux command line reference for common operations. The network commands are in a separated Network Commands Memo.

basic

apropos whatis Show commands pertinent to string.
man -t ascii | ps2pdf - > ascii.pdf make a pdf of a manual page
which command Show full path name of command
time command See how long a command takes
time cat Start stopwatch. Ctrl-d to stop.

files

ls -la > dirlist 2>&1 Redirect both stdout and stderr to a file.
rename ‘s/.jpeg$/.jpg/’ *.jpeg Mass rename with perl rename command
rename .jpeg .jpg *.jpeg Mass rename with util-linux rename command
find ./ -type f -print | xargs chmod 640 Change permissions to 640 for all files in subtree.
find ./ -type d -print | xargs chmod 751 Change permissions to 751 for all sub-directories.
shred -u private.txt delete a file from disk after securely erasing the content.

dir navigation

cd - Go to previous directory
cd Go to $HOME directory
(cd dir && command) Go to dir, execute command and return to current dir
pushd dir Put dir on stack so you can popd back to it
pushd +3 Rotate the dir stack, putting third entry at top

File searching

ls -lt List files by date, newest first
ls /usr/bin | pr -T9 -W$COLUMNS Print in 9 columns to width of terminal
find -maxdepth 1 -type f -print0 | xargs -0 ls -lS --block-size=1k List files by decreasing size
find -size +1M -ls List files bigger than 1 Megabyte.
find -name ‘*.[ch]’ | xargs grep -E ‘expr’ Search ‘expr’ in this dir and below.
find -type f -print0 | xargs -r0 grep -F ‘example’ Search all regular files for ‘example’ in this dir and below
find -maxdepth 1 -type f | xargs grep -F ‘example’ Search all regular files for ‘example’ in this dir
find -maxdepth 1 -type d | while read dir; do echo $dir; echo cmd2; done Process each item with multiple commands (in while loop)
find. -xtype l Find broken links
find -type f ! -perm -444 Find files not readable by all (useful for web site)
find -type d ! -perm -111 Find dirs not accessible by all (useful for web site)
locate -r ‘file.txt’ Search cached path index for names.
locate -r ‘file[^/]*.txt’ Search cached path index for names. ‘file’ must be in last component.

disk space

ls -lkS Show files by size in kb, biggest first.
ls -lt sort by modification time, newest first
du -sh * | sort -k1,1rh | head Show larger directories in current dir.
sudo du -hs /home/* | sort -k1,1h Sort paths by increasing use
du -ah --max-depth=0 * | sort -k1,1rh | head -n 15 Show 15 larger directories or files in current dir.
df -h Show free space on mounted filesystems
df -i Show free inodes on mounted filesystems
sudo sfdisk -l /dev/sda Show disks partitions sizes and types (MBR part)
sudo sgdisk -p /dev/sda Show disks partitions sizes and types (GUID part)
dd bs=1 seek=2TB if=/dev/null of=ext3.test Create a large sparse test file (taking no space).
>| file truncate data of file or create an empty file

text handling

tr -dc ‘[:print:]’ < /dev/urandom Filter non printable characters
echo “(33) 06.61 62-63+84” | tr -d [:blank:][:punct:] clean a phone number string
tr -s ‘[:blank:]’ ‘t’ </proc/diskstats | cut -f4 cut fields separated by blanks
tr -s ‘[:blank:]’ </proc/diskstats | cut -d’ ‘ -f4 cut fields separated by blanks
wc -l file count lines (w words, -b bytes)
cut -d: -f1 /etc/passwd | sort Lists all usernames in alphabetical order.
dd if=/dev/urandom count=1 | base64 -w 0 | cut -c 1-16 generate random 16 chararacters password
openssl rand -base64 16 | cut -c 1-16 generate random 16 chararacters password
tr -dc ‘[:alnum:]&~#|_@=+$%*<>,?;.:/!-‘ < /dev/urandom | head -c${1:-16}; echo generate random 16 chararacters password
date +%s | sha1sum`|:coreutils:`cut -f1 -d’ ‘ generate new 4O alphanumeric chars password
paste -d ‘,:’ file1 file2 file3 Merges given files line by line
mount | column -t table of mounted filesystems
join -t‘0’ -a1 -a2 file1 file2 Union of sorted files
join -t‘0’ file1 file2 Intersection of sorted files
join -t‘0’ -v2 file1 file2 Difference of sorted files
join -t‘0’ -v1 -v2 file1 file2 Symmetric Difference of sorted files
column -s, -t <tmp.csv pretty print csv
printf “%03o\n” “’%” octal code of ascii character %
printf “Ox%02x\n” “’%” hexacimal code of ascii character %
printf “%d\n” “’%” decimal code of ascii character %
iconv -f ISO8859-1 -t UTF-8 -o file.utf8 file.txt convert encoding
iconv -l List known coded character sets
sha1sum file checksum of a file (use also other sums: sha256sum, sha512sum, md5sum)
sha1sum -c checksumlist check the sums against the files

encryption

gpg -c file Encrypt file. More commands in the GnuPG Memo.
gpg file.gpg Decrypt file.
openssl -h Help including available ciphers
openssl list-cipher-commands long list of available ciphers
openssl enc -aes-256-cbc -salt -a encrypt stdin to stdout using 256-bit AES in CBC mode, and encode in base64
openssl enc -aes-256-cbc -salt -in file.txt -out file.enc encrypt to binary file.enc using 256-bit AES in CBC mode
openssl enc -d -aes-256-cbc decrypt binary data on stdin
openssl enc -d -aes-256-cbc -a -in file.enc decrypt base64 encoded file
openssl enc -aes-256-cbc -salt -a -pass file:/path/to/password.txt encrypt stdin to stdout, provide password in a file

archives and compression

tar -cjf dir.tar.bz2 dir/ Make bzip2 compressed archive of dir/
tar -jxf dir.tar.bz2 Extract archive (replace j, by z for gzip, or lzip)
tar -cxf dir.tgz --exclude ‘*.o’ --exclude ‘*~’ dir/  
tar -xf dir.tgz --to-stdout dir/file.txt Print file to stdout
tar -c dir/ | gzip | gpg -c | ssh user@remote ‘dd of=dir.tar.gz.gpg’ Make encrypted archive of dir/ on remote machine.
find dir/ -name ‘*.txt’ | tar -c --files-from=- | bzip2 > dir_txt.tar.bz2 Make archive of subset of dir/ and below.
find dir/ -name ‘*.txt’ | xargs cp -a --target-directory=dir_txt/ --parents Make copy of subset of dir/ and below.
( tar -c /dir/to/copy ) | ( cd /where/to/ && tar -x -p ) Copy (with permissions) copy/ dir to /where/to/ dir
( cd /dir/to/copy && tar -c . ) | ( cd /where/to/ && tar -x -p ) Copy (with permissions) contents of copy/ dir to /where/to/
( tar -c /dir/to/copy ) | ssh -C user@remote ‘cd /where/to/ && tar -x -p’ Copy (with permissions) copy/ dir to remote:/where/to/ dir
zip -r /path/to/archive.zip dir zip a directory
unzip archive.zip extract archive
unzip -l archive.zip list archive content
unzip archive.zip file.txt Extract one file from archive
dd if=/dev/vg0/vol0 of=/dev/vg1/vol1 bs=4096 Copy a partition to another one (bs must be a divider of volume blocksize)
dd bs=1M if=/dev/sda | gzip | ssh user@remote ‘dd of=sda.gz’ Backup harddisk to remote machine.
dd bs=4096 if=/dev/vgsource/root_snap| ssh -c ‘chacha20-poly1305@openssh.com’ rootr@remote dd bs=4096 of=/dev/vgremote/root_copy copy a partition to remote machine
dd bs=4096 if=/dev/vg0/root_snap| | ssh-c ‘chacha20-poly1305@openssh.com’ root  
killall -s USR1 dd Ask dd to print the state of the current transfer.

process management

ps axww list all processes
ps axuww list all processes and resource used
ps axmu list all processes and threads
ps axf -o pid,args List processes in a hierarchy.
ps ax -o pcpu,cpu,nice,state,cputime,args --sort -pcpu | sed ‘/^ 0.0 /d’ List processes by decreasing cpu rate (see also top).
ps ax -opid=,rss=,args= --sort=+rss | sed ‘/^s*0>/d’ | pr -TW$COLUMNS List processes by mem (KB) usage (see also top).
ps -o user --sort user| uniq -c| sort -n -k1 number of processes per user.
ps -C lighttpd -o pid= pid of lighttpd.
pgrep light pid of processes having light in their name.
pgrep -a daemon pid/command-line of all processes having daemon in their name
pidof lighttpd pid of lighttpd.
ps uw -C lighttpd user oriented list of process lighttpd.
ps -C firefox-bin -L -o pid,tid,pcpu,state List all threads for a particular process.
ps -p 666 -o etime= List elapsed wall time for process id 666
ps ew 666 show command and environment of process 666
kill -9 1234 Send SIGKILL to process 1234
killall -s USR1 dd Send signal USR1 to the dd program
pkill -s USR1 dd Send signal USR1 to the dd program
pmap 1234 Memory map of process 1234

monitoring, process admin

tail -f /var/log/messages Monitor messages in a log file.
less +F /var/log/messages Monitor messages in a log file.
lsof -p 666 List paths that process id 666 has open.
lsof /path/to/file List processes that have specified path open.
lsof -u foo Processes and files of user foo
lsof -u foo Processes no of user foo
lsof -t -c pcmanfm files open by pcmanfm
fuser -va 22/tcp List processes using port 22
fuser -va /home List processes accessing the /home
sudo tcpdump not port 22 Show network traffic except ssh.
sudo tcpdump -ni eth0 ‘dst 192.168.1.5 and tcp and port http’ all HTTP session to 192.168.1.5.
last reboot Show system reboot history.
free -m Show amount of (remaining) RAM (-m displays in MB)
watch -n.1 ‘cat /proc/interrupts’ Watch changeable data continuously.
watch -t -n1 uptime Clock with system load.
nice command Low priority command.
sudo renice 19 -p 666 Set process 666 to low scheduling priority (0<pr<20)
sudo renice +2 -p 666 Lower the scheduling priority.
chrt -i 0 command Low priority command (more effective than nice)
sudo ionice -p 666 io class and priority of process 666. Higher priority 0
sudo ionice -c3 -p 666 Sets process 666 as an idle io process.
htop -d 5 Better top (scrollable, tree view, lsof/strace integration, …)
iotop What’s doing I/O.
sudo iftop What’s using the network.
vmstat 3 monitor processes, memory, paging, block IO, traps, and cpu activity.(columns are explained in the manual.)
vmstat -m usage of kernel dynamic memory.

Users

id -a Show the active user id with login and groups.
last Show last logins on the system.
w users logged on, and their processes.
groupadd admin Add group “admin”
useradd -c “Linus Torvald” -g admin -m linus Add new user
usermod -a -G sudo linus add group “sudo” to linus groups.
adduser --uid 3333 linus Add new user, with interactive prompt, create home dir.
userdel linus Delete user linus

system information

uname -a Show kernel version and system architecture.
cat /etc/debian_version Get Debian version
lsb_release -a Full release info of any LSB distribution
cat /etc/issue Show name and version of distribution.
cat /proc/partition Show all partitions registered on the system.
grep MemTotal /proc/meminfo Show RAM total (see also free, vmstat)
cat /proc/cpuinfo Show CPU(s) info
lscpu Show CPU(s) info
lsdev hardware info from the /proc directory
sudo lspci -tv Show PCI info
sudo lshw Show hardware configuration of the machine
sudo hwinfo Show hardware configuration of the machine
lsusb -tv Show USB info
mount | column -t List mounted fs on the system (and align output)
grep -F capacity: /proc/acpi/battery/BAT0/info Show state of cells in laptop battery
dmidecode -q | less Display SMBIOS/DMI information
dumpe2fs -h /dev/part1 | grep -e ‘\([mM]ount\)\|\([Cc]heck\)’ info about fs check
sudo e2fsck -f -v -t -C 0 /dev/part1 Check health of partition
sudo sdparm -C stop /dev/sdb Stop scsi (also usb) disk
sudo hdparm -i /dev/sda Show info about disk sda
dmesg Detected hardware and boot messages

sed

See sed manual and sed1line.

sed -n '8,12p' Print lines 8 to 12
sed -n '/regexp/p' Print lines which match regular expression
sed '/regexp/d' Print lines which don’t match regular expression
sed -n '/begregexp/,/endregexp/p' Print section of file between two regexp
sed '/begregexp/,/endregexp/d' Print file except section between two regexp
sed '/^#/d; /^ *$/d' Remove comments and blank lines
sed -i 's/[ \t]\*$//' file.txt Delete trailing space at end of lines
sed -e :a -e '/^\n*$/N;/\n$/ba' Delete blank lines at end of file.
sed -i 42d ~/.ssh/known_hosts Delete a particular line
sed ':a; /\\$/N; s/\\\n//; ta' Concatenate lines with trailing \
sed = filename | sed 'N;s/\n/\t/' Put a left count number on each line of a file
sed = filename | sed 'N; s/^/     /; s/ *\(.\{6,\}\)\n/\1  /' Put a right aligned count on each line
sed 's/\x0D$//' Dos to unix eol
sed 's/$/\\r/' Unix to dos eol

File attributes, Extended Attributes and ACL

They are three different sets of attributes that can be supported from filesytems. For more details look at the File attributes section and the ACL section.

Note: for ext 2/3/4 fs you may need to (re)mount with “acl” or “user_xattr” options. Or set the filesystem default with tune2fs. On btrfs acl and xattr are enabled by default.

getfacl foo Show ACLs for file.
setfacl -m u:nobody:r foo.txt Allow a specific user to read file.
setfacl -x u:nobody foo.txt Delete a specific user’s rights to file.
setfacl --default -m group:users:rw- dir/ Set umask for a for a specific dir.
getcap file Show capabilities for a program.
setcap cap_net_raw+ep your_gtk_prog Allow gtk program raw access to network
getfattr -m- -d Show all extended attributes (includes selinux,acls,…)
setfattr -n “user.foo” -v “bar” . Set arbitrary user attributes

Desktop management

xset q display X user preferences.
xset -b Turn off system beep
xset -b Turn on system beep
xwininfo Info of the window selected by mouse click.
xwininfo -name emacs Emacs window info.
xprop Xserver properties of the window selected by mouse click.
xdpyinfo Xserver dimension and resolution.
wmctrl -lG List managed windows with their geometry.
wmctrl -l -x List managed windows with their WM_CLASS.
wmctrl -d List desktops, current desktop has a *
wmctrl -s 3 switch to desktop 3
wmctrl -a emacs switch to desktop containing emacs and raise it.
wmctrl -r emacs -t2 send emacs to third desktop
wmctrl -r emacs -e 0,-1,-1,756,495 resize emacs to 756x495 pixels
xdotool search --onlyvisible --class emacs windowsize --usehints %1 80 24 resize emacs to 80 columns x 24 lines.
xwit -columns 80 -rows 24 -names foo resize foo window.
xwit -columns 80 -rows 24 -select select and resize a window.
xwit -rows 34 -columns 80 -property WM_CLASS -names emacs resize all emacs windows.

Images manipulation

The syntax is given for ImageMagick . If you prefer GraphicsMagick just put gm before the operation. The the option related to an input file comme before the file name in GraphicsMagick and after in ImageMagick.

identify photo.jpg information about an image file
convert photo.png -resize 2048x1536 -quality 80 photo.jpg resize an image
convert apple.jpg -crop 128×128+50+50 apple_crop.jpg crop an image
convert lying.jpg -rotate 90 standing.jpg rotate an image
convert *.jpg ouput.pdf Create a single PDF from multiple images with ImageMagick
import snapshot.jpg Take a snapshot of a mouse selected desktop area.

Pdf

gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -dFirstPage=2 -dLastPage=2 -sOutputFile=page2.pdf input.pdf Extract a page from pdf document
pdftk input.pdf burst Burst a PDF document into pages and dump its data to doc_data.txt
pdfseparate input.pdf p-%d.pdf separates xx.pdf into separate pages: p-1.pdf, p-2.pdf, …
pdfseparate -f 2 -l 3 input.pdf p-%d.pdf separates from page 2 to page 3: p-2.pdf, p-3.pdf
pdfjam intput.pdf ‘2,3’ --outfile output.pdf separates pages 2 and 3
qpdf intput.pdf --pages intput.pdf 1-3 --output.pdf separates pages 2 and 3
gs -q -sPAPERSIZE=a4 -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=all.pdf file1.pdf file2.pdf … Join many pdf files into one.
pdftk in1.pdf in2.pdf cat output out1.pdf Join two pdf files
pdfunite in1.pdf in2.pdf out1.pdf Join two pdf files
pdfjam file1.pdf ‘-‘ file2.pdf ‘1,2’ file3.pdf ‘2-‘ --outfile output.pdf merge all pages of file1.pdf, page 1 and 2 of file2.pdf and all pages up from page 2 of file3.pdf
qpdf file1.pdf --pages file1.pdf --pages file2.pdf 1-2 --pages file3.pdf 2- -- output.pdf merge all pages of file1.pdf, page 1 and 2 of file2.pdf and all pages up from page 2 of file3.pdf
pdfimages input.pdf img extracts all images as impg-000.ppm, img-001.ppm,…
pdfcrop --margins ’5 10 20 30’ input.pdf output.pdf crop a pdf with left, top, right and bottom margins of 5, 10, 20, and 30 pt
pdfjam --trim ‘1cm 2cm 1cm 2cm’ --clip true file1.pdf --outfile output.pdf crop a pdf with left, top, right and bottom margins of 1cm 2cm 1cm 2cm
pdfjam --nup 2x2 input.pdf --outfile output.pdf recombines the pdf file to contain 4 pages per page.
pdftk secured.pdf input_pw mypass output public.pdf save a public copy of a password protected file
qpdf --password=mypass --decrypt secured.pdf public.pdf save a public copy of a password protected file

Refs