Adding SSH host key to known_hosts

$ sudo sh -c "ssh-keyscan -H -t rsa,dsa hostname >> /etc/ssh/ssh_known_hosts"

-H Hash all hostnames and addresses in the output.  Hashed names may be used normally by ssh and sshd, but they do not reveal identifying information should the file's contents be disclosed.

-t type Specifies the type of the key to fetch from the scanned hosts.  The possible values are “rsa1” for protocol version 1 and “dsa”, “ecdsa” or “rsa” for protocol version 2.  Multiple values may be specified by separating them with commas.  The default is “rsa”.

Optimising Linux for minimal writes - useful for an SSD

Tip 1: Use ext2 instead of ext3/ext4.

ext3 and ext4 are journaled filesystems and so maintain a log of all filesystem changes (which can be used for recovery if need be).  If you're using a laptop chances are sudden power failure isn't really a threat so you'll lose less by not having a journaled filesystem - if you're using a desktop your choice to use a non-journaled filesystem should be an informed one.

Tip 2: Don't update file access times.

Modify your /etc/fstab adding noatime to the options for mounted drives.  This will stop writing to the file system every time a file is accessed.  (This is also used to improve disk I/O performance for critical applications - git does this for example, though they may have switched to relatime).

Tip 3: Mount "non-essential" write-heavy directories as tmpfs.

Update /etc/fstab to mount /tmp, /var/tmp and /var/log as tmps:

#              

tmpfs           /tmp            tmpfs   defaults        0       0
tmpfs           /var/tmp        tmpfs   defaults        0       0
tmpfs           /var/log        tmpfs   defaults        0       0

Unfortunately when /var/log is now mounted it won't have any of the necessary directories so we'll need to create them each time on boot.  Update /etc/rc.local with the following snippet just before the "exit 0" line.

for dir in apparmor apt ConsoleKit cups dist-upgrade fsck gdm installer libvirt news samba speech-dispatcher unattended-upgrades; do
if [ ! -e /var/log/$dir ] ; then
mkdir /var/log/$dir
fi
done

You can get the list of directories you need by running cd /var/log; ls -d */

These tips can be found repeated on the web but I gathered most of my info from here.  That page also lists some further optimisations for the kernel to take full advantage of an SSD's speed.

Mounting an encrypted logical volume

I found this useful when trying to do some recovery work on my computer which uses dm-crypt for full disk encryption.  The following was done using the Ubuntu Maverick (10.10) livecd.

Mount the encrypted partition

root@ubuntu:/home/ubuntu# sudo cryptsetup luksOpen /dev/sda4 data

Enter passphrase for /dev/sda4:

root@ubuntu:/home/ubuntu# ls /dev/mapper

control  data

Mount the logical volume

root@ubuntu:/home/ubuntu# aptitude install lvm2

root@ubuntu:/home/ubuntu# vgscan

  Reading all physical volumes.  This may take a while...

  Found volume group "vg0" using metadata type lvm2

root@ubuntu:/home/ubuntu# vgchange -ay vg0

  2 logical volume(s) in volume group "vg0" now active

root@ubuntu:/home/ubuntu# mkdir /mnt/tmp

root@ubuntu:/home/ubuntu# sudo mount /dev/vg0/root /mnt/tmp

Linux: Add existing user to existing group

I seem to need to look this up at least once a month....


usermod -a -G ftp tony # add tony to the ftp group

Install dependencies with dpkg (kinda)

dpkg -i /tmp/package.deb # Try to install package, fail, but generate list of unresolved dependencies
apt-get -f --force-yes --yes install # Resolve generated dependencies
dpkg -i /tmp/package.deb # Install package (with dependencies now met)

Delete blank lines in vim

Thanks to stackoverflow for this one.

:g/^$/d will delete all blank lines in a file

Why? Because :g will execute a command on lines which match the regex and :d deletes :)

Put the results of a command in the current vim buffer

:r will read a file into the current buffer in vim
:! will execute a command
You can use them together to read the results of a command and put them in the current buffer
e.g. :r ! ls -1 /home/user/directory | sort -r

Update Ubuntu Release from the command line

sudo apt-get install update-manager-core # may/may not be necessary
sudo do-release-upgrade


If there's no new release found and you think there should be, check /etc/update-manager/release-upgrades and change Prompt=lts to Prompt=normal if necessary

Reset Moinmoin password

I always forget this

moin account resetpw --name=username password

Postgres Recovery

Postgres killed itself during a disk failure and while later trying to start it up I got messages along the lines of:

database system was interrupted; last known up at ...
database system was not properly shut down; automatic recovery in progress
redo starts at 309/3BA1EB48
record with zero length at 309/3C9F8ED8
redo done at 309/3C9F8EA8
last completed transaction was at log time ....
could not fdatasync log file 777, segment 60: Input/output error
startup process (PID 23142) was terminated by signal 6: Aborted
aborting startup due to startup process failure

On further investigation it sound like the transaction log was corrupted. This can be fixed with pg_resetxlog. This will clear the write ahead log and may result in some data loss or loss of integrity but when nothing else works it's a lifesaver. The documentation describes some follow up steps to ensure the integrity of data after postgres is starting properly.

You can do a dry run:
sudo -u postgres /usr/lib/postgresql/8.4/bin/pg_resetxlog -n /var/lib/postgresql/8.4/main
and if that indicates a new segment and there's no other option then you might as well reset with:
sudo -u postgres /usr/lib/postgresql/8.4/bin/pg_resetxlog /var/lib/postgresql/8.4/main
or
sudo -u postgres /usr/lib/postgresql/8.4/bin/pg_resetxlog -f /var/lib/postgresql/8.4/main

Pretty Print XML in Python

From the command line:

python -c "import xml.dom.minidom; xml = xml.dom.minidom.parse('myxmldoc.xml'); print xml.toprettyxml()"