Selected Linux commands

Tips & first steps

Command-line tricks:

• Type Control-R and start typing = will match last command that started that way
• Tab will autocomplete everything, but from all available commands

Most useful to quickly locate commands, executable is "locate", such as:

 locate memcached # or
 whereis memcached
 apropos ftp # searches for all "ftp" programs

Traditional find files uses:

 find / -name 'file.ext' # "/" to start at root, option -name to search for file names

Very useful to find software which may not yet be installed:

 apt-cache search YourSearchTerm

Linux version numbers:

 sudo cat /etc/debian_version # full Debian version number 
 uname -a # Kernel / Xen version numbers

Get help

 apropos ftp # searches for all program descriptions with ftp
 man 7z # manual page for 7z
   # shift + ? → searching a string (by regular expression)
   # shift + n/n → find the next/previous match
   # q → quit

Most commands provide a help option and the following works:

 a-command --help
 a-command -h
 # sometimes there is an info on usage
 a-command --usage

Stop services / demons

QUICK:

• sudo /etc/backupscripts/services-start.sh
• sudo /etc/backupscripts/services-stop.sh

DIRECT:

sudo /etc/init.d/apache2 stop
#sudo  /etc/init.d/memcached stop
sudo /etc/init.d/mysql stop
#sudo /etc/init.d/tomcat5.5 stop
sudo /usr/share/fedora/tomcat/bin/shutdown.sh
sudo /etc/init.d/webmin stop

to restart (template to copy and use directly):

#sudo /etc/init.d/memcached start
sudo /etc/init.d/mysql start
sudo /etc/init.d/apache2 start
#sudo /etc/init.d/tomcat5.5 start
sudo /usr/share/fedora/tomcat/bin/startup.sh
sudo /etc/init.d/webmin start

Fedora may or may not be installed under the main tomcat. As of 2009-08, it can be started/stopped using (fedora folder is a softlink to current installed version; alternatively one can use "$FEDORA_HOME"):

 sudo /usr/share/fedora/tomcat/bin/shutdown.sh
 sudo /usr/share/fedora/tomcat/bin/startup.sh

Disk usage and rights

How much space on disks?

 df -l # diskfree, local disks only
 df -lh # diskfree, human friendly (size in MB, GB, etc.)
 df .  # diskfree current disk

Disk usage = "tree size": where is the space used:

 du -h --max-depth=1 # '''analyze only 1 level of directories deep'''
     # -h = human readable, MB, GB, factor 1024; -si would be factor 1000.
 du -S # do not add up content of folders, keep values Separate (useful for manual analysis)

For rights, it is often necessary to change the group of a file or folder. It is not necessary to change the owner as well (chown -R).

 ls -l    # will display owner and group names
 ls -g    # will only display group names (easier)
 chgrp -R # R: do it recursively

When copying folder trees, it is easy to loose essential information. Use

 cp -pr  # preserve owner, rights, etc., copy recursively; 
 # but devices, sockets, etc. still are not handled, if this is necessary use:
 tar -p

Searching

Locate does not work, and using Find is tough. Good info: http://content.hccfl.edu/pollock/unix/findcmd.htm Example for find:

# find from the root level
  find / -name index.html
# find from the current directory level
  find ./ -name index.html
# find from the upper directory level
  find ../ -name index.html
# regular expression search:
# find a file with extension .svg and ä-something in the current subdirectory ./e
  find ./e/ -regex '.*ä.*\.svg'
# find “only in this directory here” image file names with regular expression search (case insensitive)
  find . -maxdepth 1 -iregex ".*\(jpg\|jpeg\|png\|gif\|svg\|tif\|tiff\)"
# find an execute something on the fly
  find /var/www/v-species/o/media/0/  -user root -name '*' -exec sudo chown -R  www-data:www-data '{}' ';'
  # tip: twice a -exec ... statement can trigger two commands during the search

For searching inside files, use grep (-r searches recursive, but only within the file pattern):

# set highlighted colors for grep findings in the current session
# permanently it can be set too in your home directory ~/.profile
  export GREP_OPTIONS='--color=auto'
# find xml in svg files recursively
  grep -r "xml" *.svg
# find <script> in all files recursively
  grep -r "<script>" *
# with line numbers in the file and combined with find
  find ./ | grep -n -r "<script>" *
# open the file at a specific line number position
  nano +30 myfoundfile.php # nano editor
  vi +30 myfoundfile.php # vi-editor

grep -r "x" index.html would not work! But using the “pipe” (|) character and combine the two commands will do the trick.

Process handling

top (interactive taskmanager)

top or top u www-data lists all processes or only for user www-data

h get help

c show command’s path instead of process names

k kill a process by PID

u filter for a specific user

q quit

Sorting: b → sorted column in bold; > or < → select next or previous column for sorting; F → a specific column can be selected for sorting; R → alters sorting toggles between descending or ascending order

z is coloring the display and Z customises colors

W write and save current settings of top

ps (taskmanager: processes’ snapshot)

ps -ef list all processes, including services, use kill (number) or pkill (name)

kill processes by name

kills a process, for example killall memcached

Kill a hanginge apt-get:

ps -e | grep apt
kill [THE_PID_processID]

Other commands

» changing permission of files/drectories:

chmod [options]... mode [,mode]... file... 

permissions can be set either by octal number code or by characters:
chmod u+x myscript.sh (add executable mode to a shell script by the user/owner: u → user, + → add, x → executable)
chmod u-x myscript.sh (remove executable mode to a shell script by the user/owner - → remove)
chmod a+r file (allow read permission to everyone: a → all users, + → add, r → readable)
chmod -R u=xrw,go=r ./directory (set «drwxr--r--» for files and directories recursively: -R → recursively, u → user, g → group, o → other people)
Note that doing chmod -R u=rw,go=r ./directory recursively without x removes (listing) access to the directory! Hence, don't set -R 644 to all directories recursively, only one by one:

find . -type d -exec echo 'chmod to 644 for: ' '{}' ';' -exec chmod u=rw,go=r '{}' ';'

This sets only directories at once recursively: it finds here (the dot .) for directory type and executes 2 commands. '{}' in quotes is the found (relative) path here. In detail:

find here in current (→ .) directory directory type start execution the echo bash command the found string of find (=relative path) stop execute command option here start another execution the chmod command set user rights the found string of find (=relative path) stop execute command option here

find . \ -type d \ -exec \ echo 'chmod to 644 for: ' \ '{}' \ ';' \ -exec \ chmod \ u=rw,go=r \ '{}' \ ';'

» Directory listing:

ls -lap 

to better see all files, -l use long listing, -a list all files/directories, i.e. also hidden ones, -p to see folders marked with trailing “/”

A source helping to understand the Linux file system is: http://www.pathname.com/fhs/pub/fhs-2.3.html

tree 

show directory and files as a tree, -L use 1 level down, e.g. tree -L 1

.
├── backups -> /mnt/dump/var/backups
├── tmp
└── www

» Reading files:

tail file 

displays the end of files; tail -9 mylog.log gets the last 9 lines

cat file 

concatenate files and print on the standard output

cat /etc/passwd to see user list

cat -v filename to display non-printing characters so they are visible. If the file has been edited on a Windows machine it can sometimes add CR/LF (VM) characters on the end of each line (hidden by default on most editors), so #!/bin/sh becomes #!/bin/shVM. This causes error: bad interpreter ^M. To remove such characters, use e.g. cat infilename | tr -d "\r" > outfilename

more file 

read as much as the screen can display and wait untill Enter shows a new line

less file 

the opposit of more

» Editing/comparing files:

nano file 

an easier editor on debian 4 with syntax highlighting set globally in /etc/nanorc

vi file 

complicated but enhanced command line editor with syntax highlighting, auto indent and macro functionality.

Basically vi has two modes: a writing mode and a command line mode. Esc i = insert/writing mode, Esc again = command mode; in command mode: ":u" = undo, ":w" = write/save, ":q" = quit, ":q!" = quit without save, ":x" = save & quit, "?word" = search for “word” (N → next, n → previous match)

vimdiff file1 file2

show differences with syntax highlighting in 2 columns (default)

vimdiff -o file1 file2 horizontal instead of columns, ":q" = quit

» User management:

adduser, deluser, passwd 

user management. More on user management: http://www.cae.wisc.edu/site/public/?title=linaccounts

sudo adduser USERNAME GROUPNAME adds existing user to existing group

sudo passwd USERNAME allows to reset passwords for users

vnc = special vnc user, not sure whether useful.

» Download:

wget http://... 

downloads the specified file. Option -c also allows an interrupted download to continue and --document-output another output name, e.g. wget http://... -c --document-output=outputname.html

scp user@hostname.net:/path/on/the/server /local/path 

downloads the specified file from a server to a local machine.

Install commands

# To find package names use: 
aptitude search (keyword)
apt-cache search (keyword)
apt-cache search (keyword) | sort
apt-cache search 'php.*sql'
apt-cache search 'elvis|vim'
 
# Clean-up:
apt-get autoremove # will remove packages no longer needed.
apt-get clean      # will clear the repository completely
 
# Install
apt-get update     # retrieve new lists of packages
aptitude search '~i' # show all installed packages
apt-get install  (package name) # install new packages (pkg is libc6 not libc6.deb)
 
dpkg -l (keyword)
# to list all packages with codes for status, e.g.'ii' for installed,'rc' for
# removed, but configuration files still there; 
dpkg -s (package name) # to get information on the status of a package

ls colors

Putty displays ls out in colors. These are:

• Executable files: Green
• Normal file : Normal
• Directory: Blue
• Symbolic link : Cyan
• Pipe: Yellow
• Socket: Magenta
• Block device driver: Bold yellow foreground, with black background
• Character device driver: Bold yellow foreground, with black background
• Orphaned syminks : Blinking Bold white with red background
• Missing links ( - and the files they point to) : Blinking Bold white with red background
• Archives or compressed : Red (.tar, .gz, .zip, .rpm)
• Image files : Magenta (.jpg, gif, bmp, png, tif)

OR (other source):

Type                Foreground Background
Folder/Directory    blue      (default)
Symlink             magenta   (default)
Socket              green     (default)
Pipe                brown     (default)
Executable          red       (default)
Block               blue      cyan
Character           blue      brown
Exec. w/ SUID       black     red
Exec. w/ SGID       black     cyan
Dir, o+w, sticky    black     green
Dir, o+w, unsticky  black     brown

Renaming file extensions

Linux does not support wildcards in the target of a move command the way Windows does. The equivalent for Windows:

rename *.jpeg *.jpg

is

for files in *.jpeg; 
   do n=${files/.jpeg/.jpg}; # saves to variable $n
   mv $files $n; 
done

Related: To add a prefix use:

 for i in *.jpg; do mv -i "$i" "XXX_$i"; done

Archiving

With the general zip, p7zip-full etc. installed, the following commands work (-mx=9 = max. compression):

 # -9 is optional, higher compression
 zip archivename.zip file.sql /folder -9  
 unzip archivename.zip
 # for real good compression use:
 # (a = add, -mx7 and -mx9 = higher compression, x = extract)
 7z a -mx=9 archivename.7z file.sql /folder
 7z x archivename.7z

Note: Because 7z will not store owner or group information, the option -r = recurse into subfolders is not recommended. To archive folders use (where ! is the folder name, as in WinSCP custom commands):

 # tar + 7z a folder: cf = create file, a = add, -si = Read data from StdIn
 tar cf - "!" | 7za a -si -mx7 "!.tar.7z"
 # tar/7z to folder: x = eXtract with full paths, -so = Write data to StdOut, bd = Disable percentage indicator
 # xf = extract file
 7za x -so -bd "!" | tar xf -

Note: inside WinSCP, the tar/7z command to folder results in error (ok in ssh), but simply selecting SKIP results in correct result, this seems to be more a bug of the way WinSCP handles messages than of the process (?).

Problem searching

Who logged on? See: tail --lines=100 /mnt/dump/var/log/auth.log

Apache processes: See if the apache process count starts rising again over time: ps uax | grep apache2 | wc The first number will be the count+1 of apache processes (The extra 1 is the grep). 7 is ok, ram observed 150 on overload.

Writing shell scripts

Very helpful: http://www.calpoly.edu/~rasplund/script.html