How long has the system been up?


What architecture is the system?


Create a list of useful configuration information for this system

sosreport –list -list plugins

e.g. bootloader  apache  kernel  hardware  memory  samba etc

Display kernel boot messages

dmesg       (/var/log/dmesg)

Display last logged on users

last -x

Read the system log (syslog)


Display all kernel information

uname -a

Display release numbers

/etc/lsb-release.d      /etc/redhat-release

Check SNMP is working if HP Systems Management is installed

snmpwalk -v1 -c community_string localhost enterprises | grep -i dl


Logical Volume Management

Logical volume management on Linux is a different way of using available disks (block devices), allowing for more flexible allocation of space on filesystems stored on one or more disks by grouping disks into volume groups, then creating logical volumes within those volume groups without boundaries.  Filesystems are created on the logical volumes which may use a portion of a disks full capacity, or the capacity of more than one disk.  The only limit is the total space in the volume group which is determined by the sum of the total physical space of all the physical block storage devices in the volume group.

You can add more devices to an existing volume group and even take them away (be careful!).  Volume groups, Logical Volumes in them, and the Filesystems on them can also be extended, resized or removed as necessary.

Scan for block devices that can be used for Logical Volume Management


Display current physical volumes and their LVM Status


Use block device /dev/hdb for LVM

pvcreate /dev/hdb

This command also takes multiple devicenames in one go, separated by spaces.

Create a volume group consisting of the physical volumes

vgcreate VolGroup01 /dev/hdb

Extend an existing volume group onto the block device just added

vgextend VolGroup00 /dev/hdb

Read the man page on vgextend for all available options.

Create a logical volume within the volume group

lvcreate -L +25G /dev/VolGroup01/LogVol00

Extend existing logical volume into the new unused space in the volume group

lvextend -L +25G /dev/VolGroup00/LogVol00

Expand the filesystem into the new free space in the logical volume.

resize2fs /dev/VolGroup00/LogVol00/

Verify the new size using df

df -kh

Display volume group information

vgdisplay VolGroup01

Display physical volume information

pvdisplay /dev/hdb

Display logical volume information

lvdisplay /dev/VolGroup01/LogVol00

Un-mount the filesystem

umount /filesystem

Check and Fix errors on the filesystem

e2fsck -f /dev/VolGroup01/LogVol00   

or  /dev/mapper/LogVol00

Resize a logical volume

lvresize -L new-size /dev/VolGroup01/LogVol00

Resize a filesystem

resize2fs /dev/mapper/LogVol00

e.g. 1500M or 44G


Disks and Partitions

List scsi devices


Explore the directory containing SCSI information

ls -F /proc/scsi

Display block devices on the system

ls /sys/block

Display disk devices

fdisk -l

Rescan SCSI bus without reboot

echo ” – – -” > /sys/class/scsi_host/host#/scan

Determine SCSI host value

ls /sys/class/scsi_host


Process Management Commands

Display load average, top CPU processes (use SHIFT M to sort by memory use).


Display a tree of all processes


Display all processes

ps -aux

Customise ps output to your requirements

ps -eo pid tid class rtprio ni pri stat comm

Find zombied processes

ps -eo pid stat | grep Z

Find processes waiting for disk

ps -eo pid stat | grep D

Trace execution of process


Display free memory


Display virtual memory status


Display inter-process communication status

ipcs -l or -a

Display names of slab memory in size order



Upgrading VMWare ESXi hosts

So your ESXi environment has a few virtual machines running, and their OS’s are all kept up to date, but what about bringing the ESXi host itself up to date?  This is the quickest and easiest way I ‘ve found of getting the job done.

Download the .zip package from VMWare for your ESXi version.  This will need an internet connection.


If you don’t already have it installed, you can download and install vSphere client by typing the name or IP address of your ESXi host into your web browser.  This will not need an internet connection.

You’ll also need the VSphere CLI, which will need to be downloaded from VMWare.  This will need an internet connection.

Should you have any installation issues, you may want to download the .NET Redistributable Package from Microsoft and pre-install that before attempting to install the VMWare products.

Once you have vSphere Client and vSphere CLI installed and the .zip package ready,

Connect to the VCenter Server / ESXi host and shutdown or VMotion any running virtual machines.

Place the ESXi host into Maintenance Mode.

Open vSphere CLI.

cd C:\Program Files\vmware\vmware vsphere cli\bin\perl

perl –server esx_host_ip –username root –bundle –install

Enter the root password when prompted.

It’ll go quiet for a while, but you can see that something is happening in VSphere Client.  The job will be “In Progress” for around 2 or 3 minutes on modern hardware with 1Gb/s network connectivity.  Only do one host upgrade at a time to prevent IO errors occurring which will halt the upgrade and leave locked files in /var/update/cache/ which will require a restart of the host to clear costing you time.  This is especially true if you are connected at only 100Mb/s over the network.

When you see the words “Installation Complete” in the vSphere CLI terminal, the upgrade part is complete.  Leave the host in Maintenance Mode for now and reboot it from VSphere Client.

When the host is back up, log on again using VSphere Client, and take it out of Maintenance Mode.

Thats it.  power up the VM’s, VMotion them back from the other hosts in the cluster, or just let DRS take care of it depending on your environment.

Repeat for each host in the cluster.


Disk Recovery and Forensics

Who doesn’t love the word “Forensics”?  It’s a word that brings out the inner geek in all of us, yet the reality is usually pretty grim – like when your only hard drive containing all your important files and photos fails.

The first thing you should do if you suspect your hard drive is failing or has failed is not attempt to write to it and if necessary hard shut the machine down asap by pushing and holding the power button on your PC.  Any further writes could lunch the drive for good making recovery impossible.  In otherwords STOPPP!!

Anyway, here’s some notes from recent tinkerings with Ubuntu Rescue Remix (Google it, Download it).  It’s a bootable Live CD which boots a computer into a command line only Linux environment, and for the remaining 2% who are still reading, provides you with a good handful of tools that stand you the best chance of recovering data from a failing hard disk.

Assuming you’ve just booted it and your hard disk(s) are attached, the first thing to do is identify which disk corresponds to which device name in /dev.  This can be done using lshw or fdisk -l

lshw > /tmp/hardware

cat /tmp/hardware | less

The next step is to clone the dodgy disk to either another disk, or to an image file or both.  You choose.

ddrescue /dev/sda /dev/sdc

or (restartable clone to an image file)

ddrescue –direct –retrim –max-retries=3 /dev/sda imagefile logfile

If you’ve cloned to another healthy disk, then you should fsck /dev/sdc to fix any errors, then attempt to mount it with mount /dev/sdc1 /mnt/mydisk and see if you can read any data on it.  You may be as good as done at this point with no further need to go on to employing other more targeted tools for recovering data off an unmountable drive.  Failing that, try to stay calm (really – it helps), clone the disk to an imagefile the best you can, then read on.  If you can’t stay calm, then run testdisk and benefit from a more intuitive menu driven interface of various recovery options.


Or if you’re enjoying this new found challenge of getting the photos back before the missus finds out, read on about using foremost and other similar, powerful recovery commands.

sudo foremost -i imagefile -o /recovery/foremost -w       (list recoverable files only)

sudo foremost -i imagefile -o /recovery/foremost -t jpg           (recover jpg files only)

If you suspect that the partitioning information on the drive is gone, then you can replace it using gpart to guess what the previous partitioning scheme was based upon whats on the drive.  This is good if you’re an overzealous techy who blanked the drive to install the latest OS without thinking about who else had an account on the computer and what they may have had stored.  Not good.  Don’t do it again.

sudo gpart /dev/sda

Or instead of using foremost, you could try scalpel.  Like foremost, but configurable and well, a bit better.

vi /etc/scalpel/scalpel.conf     (to configure options)

sudo scalpel imagefile -o /recovery/scalpel/

Or maybe try magicrescue on the cloned disk if there’s multiple file types to be recovered (requires the presence of recipes for the filetypes to be recovered).


Enable DMA on the cloned disk first to speed things up.

hdparm -d 1 -c 1 -u 1 /dev/hdc

sudo magicrescue  -r gzip -r png  -d /recovery/magicrescue /dev/sdc

If it’s specificly photos you’re wanting to recover, then there are two tools to choose from; photorec and recoverjpeg.

sudo photorec imagefile         (imagefile is the disk imagefile, not an image as in picture)

sudo recoverjpeg /dev/sdc1       (recovers any obvious jpeg files on partition /dev/sdc1)

If the files you want to recover were deleted on the original drive, then assuming the drive has come from a windows computer and was formatted with NTFS, then you can use ntfsundelete to recover the deleted files.

ntfsundelete -s /dev/sdc1     (scans for inodes of deleted files which can be subsequently recovered)

ntfsundelete /dev/sdc1 -u -i 3689 -o work.doc -d /recovered/ntfsundelete

If you want to recover old files previously written to a disk containing a new FAT filesystem, then you’re into using autopsy and dls, fls, icat and sorter from sleuthkit to create a secondary image of unallocated blocks contained in the image and list the inodes of files apparently contained within them, recover those files and optionally sort them by filetype, respectively.

sudo autopsy -d /media/disk/autopsy      (use your local ip address)

dls imagefile > imagefile_deletedblocks        (create secondary, smaller imagefile)

fls imagefile_deletedblocks -r -f fat -i raw      (list inode numbers of any deleted files found)

icat -r -f fat -i raw imagefile_deletedblocks inode_number > myfile.doc    (recover a file)

sudo sorter -h -s -i raw -f fat -d out -C /usr/share/sleuthkit/windows.sort /imagefile

This just touches upon ways you can recover lost data, with a few useful examples, but remember each command in it’s own right has a multitude of options which can be perused using the man command and reading the accompanying manual.  You can also google man sorter for example, and read the man page in a web browser.  I hope you get some data back!