Icinga plugin state UNKNOWN after upgrade

I was updating the monitoring setup at work the other day and suddenly got hit by a few State: UNKNOWN - **ePN /usr/lib/nagios/plugins/check_rbl: plugin did not call exit() messages.

When the plugin was executed outside of Icinga it worked fine, online searches revealed nothing of use, I spent ages trying to adjust the config file and check command for the plugin but with no success.

Eventually I was able to track it down to the embedded Perl interpreter used by Icinga. The solution was fairly simple I just had to add nagios: -epn as a comment within the first 10 lines of the script. This told Nagios to use an external Perl interpreter when running this plugin.

This solution should apply to anything based on Nagios by the way: Icinga, Shinken, Centreon-Engine, Opsview, etc.


Fear not for I have returned

It’s been a while since I last updated and the truth is I just don’t have time to write proper blog posts. Keeping that in mind, but still wishing to keep this up-to-date. I’ve decided to post quick tips and snippets that I find useful. Here’s the first:

I’ve been doing some database work recently and I wanted to unify the character set and collation for all databases and tables. Changing the database collation was easy to do in bulk. The tables less so and I was not manually changing 13,014 tables, eventually I came up with this:

That should generate a bunch of statements that look like this:

For every table in every database except the information_schema, mysql and performance_schema databases. I just pipe these right back in to MySQL but you can output them to a text file and run that later if you prefer. This snippet should also to be useful for any mass/multiple/bulk table edits or changes.


I don’t know how that got on there

Someone handed me a couple of floppy disks the other day and wanted me to get some important files from them, these disks had been sitting in the back of a cabinet since 1996. I had three USB floppy drives and a couple of hardware floppy drives handy so I thought I’d give it a go and tried both disks in each drive. All I got was erroring on almost every sector with messages like:

It’s most likely that these errors were caused by age but floppy disk drives also have differing alignment between the drive the data was written with and the drive used for reading the data.

There was only one thing I could do really and that was give ddrescue a try. There are two utilities called ddrescue, to be exact it was GNU ddrescue version 1.11. What I decided to do was try running ddrescue using three different read methods, on each disk in every drive (different alignments). My hope being that by writing to the same image file and using the logfile feature of ddrescue that I could amalgamate the good parts of each read into one complete image file.

Note: You’ll probably need to change the options below for each floppy device, file location, etc, if the drives are in different computers it’s probably easiest to save the files to some central location.

First we try to copy as much data as possible, without splitting sectors or retrying on failures:

Now we retry any previous errors twice, using uncached reads:

Finally we try again but with the retrim option set, so that ddrescue will try to reread full sectors:

At this point you hopefully have a complete floppy disk image (I only had to use three of the available floppy drives) and then depending on the situation and damage you can either run the image through fsck, mount it, write it to a new disk or run it through a file carver.

In my situation I was able to use the mtools suite to copy the required files directly from the floppy disk image files.


By your command

It’s been a while since my last update, but as with any good IT guy I’ve been both too busy and too lazy to write anything. Anyway, I thought I’d make a note of some commands more for my own use that anyone else’s, but if someone else does find one useful then that’s a bonus.

Split a file at a word or pattern into multiple files:

Notes: Replace FILE with the file name you wish to run the command against.

Take a screenshot via SSH:

Notes: None.

Search for something that looks like an e-mail address:

Run a query on multiple tables in a database matching a pattern

Notes: Replace the markers where indicated (USER, PASSWORD etc) and obviously change the example query.

Command line screencast

Notes: Run nc ADDRESS 5000 to connect and watch.

Empty all log files

Notes: None.

Watch MySQL queries

Notes: Replace the markers where indicated (USER, PASSWORD etc).

Find potential duplicate files

Notes: None.


Mounting a case-insensitive filesystem on Linux

Recently I had a web developer who was coding on a Linux server, he was used to working on Windows servers and was having problems dealing with the case sensitivity of the EXT3 filesystem. I looked into it and in addition to using the Apache mod_speling module, also decided to go a step further. As root or using sudo do the following:

  1. Create a file of the size you wish the filesystem to be, 1GB in this example:
  2. Format the filesystem, fat32 in this example but you can use NTFS or another case-insensitive file system:
  3. Mount the filesystem and test it, you may wish to use uid= and gid= to mount it as a specific user and you’ll probably want to adjust the umask to something less permissive:
  4. Add this to /etc/fstab so that it gets mounted after a reboot:

I wouldn’t use this for production systems, in that situation I’d recommend a dedicated FAT32 or NTFS partition. If you do have to run this on a production system it should be fine for 99% of setups, it just feels like a quick hack to me.