Kris Davidson

I don’t know how that got on there

Thursday, the 22nd of September, 2011

Someone handed me a couple of floppy disks the other day and wanted me to get some important files from them, these disks had been sitting in the back of a cabinet since 1996. I had three USB floppy drives and a couple of hardware floppy drives handy so I thought I’d give it a go and tried both disks in each drive. All I got was erroring on almost every sector with messages like:

sd 5:0:0:0: [sdc] CDB: Read(10): 28 00 00 00 01 1e 00 00 01 00 00 00 end_request: critical target error, dev sdc, sector 286
sd 5:0:0:0: [sdc] Unhandled sense code
sd 5:0:0:0: [sdc]  Result: hostbyte=invalid driverbyte=DRIVER_SENSE
sd 5:0:0:0: [sdc]  Sense Key : Medium Error [current] Info fld=0x11f
sd 5:0:0:0: [sdc]  Add. Sense: Recorded entity not found

It’s most likely that these errors were caused by age but floppy disk drives also have differing alignment between the drive the data was written with and the drive used for reading the data.

There was only one thing I could do really and that was give ddrescue a try. There are two utilities called ddrescue, to be exact it was GNU ddrescue version 1.11. What I decided to do was try running ddrescue using three different read methods, on each disk in every drive (different alignments). My hope being that by writing to the same image file and using the logfile feature of ddrescue that I could amalgamate the good parts of each read into one complete image file.

Note: You’ll probably need to change the options below for each floppy device, file location, etc, if the drives are in different computers it’s probably easiest to save the files to some central location.

First we try to copy as much data as possible, without splitting sectors or retrying on failures:

sudo ddrescue --verbose --no-split --cluster-size=1 /dev/sdc floppy1.img floppy1.log

Now we retry any previous errors twice, using uncached reads:

sudo ddrescue --verbose --direct --max-retries=2 --cluster-size=1 /dev/sdc floppy1.img floppy1.log

Finally we try again but with the retrim option set, so that ddrescue will try to reread full sectors:

sudo ddrescue --verbose --direct --retrim --max-retries=2 --cluster-size=1 /dev/sdc floppy1.img floppy1.log

At this point you hopefully have a complete floppy disk image (I only had to use three of the available floppy drives) and then depending on the situation and damage you can either run the image through fsck, mount it, write it to a new disk or run it through a file carver.

In my situation I was able to use the mtools suite to copy the required files directly from the floppy disk image files.

Kris’ rules of IT

Wednesday, the 7th of September, 2011

I really need to post more but I’m just too busy (and lazy), it’s a pity I couldn’t script it like everything else I do. Anyway I was going through some old files of mine and I found this set of rules I put together for some colleagues a while back:

1. No one reads anything.
2. They forget what they read.
3. They deny what they have read.
4. Users and clients will lie.
5. Fast, cheap, reliable. Pick two.
6. Always cover your ass, but come clean when it’s your fault
7. Always backup even when told not to.
8. If you don’t have a replacement it will break soon.
9. Generally no one understands what you’re talking about or doing.
10. Rebooting while not ideal will usually fix a problem.

No one reads anything, They forget what they read, They deny what they have read

All three of these are variations of the same principle, basically don’t make assumptions and expect people to have read anything. While it would be nice if they did review documentation, guides and announcements I’m not holding my breath.

Users and clients will lie

Some people will lie when you’re trying to troubleshoot, usually because they think they know what is wrong or because they’re feeling stupid or guilty and trying to cover themselves.

Fast, cheap, reliable. Pick two

Pretty self explanatory, something can be fast and cheap, fast and reliable or cheap and reliable, but not all three.

Always cover your ass, but come clean when it’s your fault

Don’t go overboard with this, I simply mean that you should get written confirmation (e-mail will do) whenever you’re asked to do something you think is illegal or whenever you want to do something that will potentially cause downtime. If you do happen to cause downtime then come clean immediately and explain what happened.

Always backup even when told not to

This is kind of like the covering your ass rule, but always backup and test those backups.

If you don’t have a replacement it will break soon

I don’t know why, call it unprecedented coincidence but every time I’ve ran out of replacements for something, even if it’s something that never needs replacing or was just replaced last week then I’ll need one.

Generally no one understands what you’re talking about or doing

This can be annoying but try to explain things in a simple and clear manner, using the terminology of the person you’re talking to while avoiding being patronising. As IT departments need to justify their existence sometimes it never hurts to keep time sheets or maintain a trouble/incident/help ticket system.

Rebooting while not ideal will usually fix a problem

We’re all aware of the ‘Have you tried turning it off and on again?’ stuff, it usually works but a lot of the time it just fixes the symptoms instead of the underlying problem and it’s good to keep that in mind.

By your command

Thursday, the 21st of July, 2011

It’s been a while since my last update, but as with any good IT guy I’ve been both too busy and too lazy to write anything. Anyway, I thought I’d make a note of some commands more for my own use that anyone else’s, but if someone else does find one useful then that’s a bonus.

Split a file at a word or pattern into multiple files:

awk '/Pattern to split at/{n++}{print > "split" n ".txt"}' FILE

Notes: Replace FILE with the file name you wish to run the command against.

Take a screenshot via SSH:

DISPLAY=:0.0 import -window root /path/to/directory/screenshot.png

Notes: None.

Search for something that looks like an e-mail address:

egrep -io '([[:alnum:]_.-]+@[[:alnum:]_.-]+?\.[[:alpha:].]{2,6})'

Notes: If don’t have egrep, replace egrep -io with grep -Eio.

Run a query on multiple tables in a database matching a pattern

mysql -uUSER -pPASSWORD -D DATABASE_NAME -e "show tables" -s | egrep "SEARCH_PATTERN" | xargs -I "@@" mysql -uUSER -pPASSWORD -D DATABASE_NAME -e "DROP TABLE @@;"

Notes: Replace the markers where indicated (USER, PASSWORD etc) and obviously change the example query.

Command line screencast

mkfifo /tmp/fifo;(nc -q0 -k -l -p 5000 < /tmp/fifo > /dev/null &);script -f /tmp/fifo

Notes: Run nc ADDRESS 5000 to connect and watch.

Empty all log files

for file in $(find /var/log -type f); do > $file; done;

Notes: None.

Watch MySQL queries

watch -n 1 mysqladmin --user=USER --password=PASSWORD processlist

Notes: Replace the markers where indicated (USER, PASSWORD etc).

Find potential duplicate files

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

Notes: None.

Mounting a case-insensitive filesystem on Linux

Thursday, the 25th of November, 2010

Recently I had a web developer who was coding on a Linux server, he was used to working on Windows servers and was having problems dealing with the case sensitivity of the EXT3 filesystem. I looked into it and in addition to using the Apache mod_speling [sic] module, also decided to go a step further. As root or using sudo do the following:

1. Create a file of the size you wish the filesystem to be, 1GB in this example:

   dd if=/dev/zero of=fat32.disk bs=1K count=1M

2. Format the filesystem, fat32 in this example but you can use NTFS or another case-insensitive file system:

   mkfs.vfat fat32.disk

3. Mount the filesystem and test it, you may wish to use uid= and gid= to mount it as a specific user and you’ll probably want to adjust the umask to something less permissive:

   mount /media/fat32.disk /var/www/fat32 -t vfat -o loop,owner,group,umask=000

4. Add it to /etc/fstab so that it gets mounted after a reboot:

   /media/fat32.disk /var/www/fat32 vfat loop,owner,group,umask=000 0 0

I wouldn’t use this for production systems, in that situation I’d recommend a dedicated FAT32 or NTFS partition. If you do have to run this on a production system it should be fine for 99% of setups, it just feels like a quick hack to me.

Starting services in safe mode

Saturday, the 11th of September, 2010

Recently I was trying to figure out how to start additional services in Windows safe mode. I had a user whose laptop kept crashing at login, I had a quick look and several theories came to mind but uptime was important, so as a temporary workaround I set it up in safe mode with networking.

A few days later the user calls and wants to be able to print in safe mode. I look into it, do some searching, but the prevailing wisdom seemed to be that it wasn’t doable. This sounded like an MCP party line to me so I decide to explore the registry. Eventually I find the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control key which has sub-keys of Minimal and Network. Minimal being safe mode, Network being safe mode with networking. It seems to be a whitelist of services, drivers and drive groups that are allowed to start or load.

Therefore it is possible to start additional services and load additional drivers in safe mode – just add a key for the service or driver short name, then a string for type. The below entry (if in a .reg file) would allow the Print Spooler to start in safe mode with networking.

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SafeBoot\Network\Spooler]
@="Service"

If you want a list of all drivers, driver groups and services starting in normal mode and their corresponding short names check HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services.

I’d caution against whitelisting too much as it kind of defeats the purpose of safe mode, though in certain situations as a quick hack it can useful. It may also be something worth checking the next time you’re dealing with a particularly nasty malware infection. I haven’t seen anything which exploits it yet, but I imagine something does.