Running ExtraDB on a Linux block device

Since the future of free MySQL appears a bit uncertain due to the fact that its owner happens to be well recognized for its self-developed proprietary database, I decided to have a look at the fork that is led by the original main developer of MySQL: MariaDB.

Interestingly, things have not changed much once you get your head around the fact that the database itself as well as the engines have been renamed in the documentation, but still use their familiar names in the config files.

So the state-of-the-art storage engine that used to to be InnoDB is now called ExtraDB, but all its config options still start out with innodb_. Also the engine names in table type specifications still use the old names. So mostly, you just replace MySQL with MariaDB and keep going.

There is one catch however when using a raw block device for ExtraDB tablespace: It no longer works to pass the absolute name of the block device's /dev file (eg. /dev/sdb1). Examining mysqld.err reveals that the daemon actually tried to access a file by the name .//dev/sdb1, which looks to me like somebody took Windows support quite a bit too far.

120116 15:21:21  InnoDB: Operating system error number 2 in a file operation.
InnoDB: The error means the system cannot find the path specified.
InnoDB: If you are installing InnoDB, remember that you must create
InnoDB: directories yourself, InnoDB does not create them.
InnoDB: File name .//dev/sdb1
InnoDB: File operation call: 'open'.
InnoDB: Cannot continue operation.

Fortunately there is a simple workaround for this bug, simply create a symlink or a replica of the respective device node inside the /var/lib/mysql directory and point the daemon to plain sdb1.

cp -a /dev/sdb1 /var/lib/mysql/


/etc/mysql/my.cnf:

innodb_data_file_path           = sdb1:10000269312raw

Bulk-shrink images with ImageMagick

I'm one of those (apparently few) people that are annoyed by emails showing me e.g. the location of a screw on a piece of metal with 3 pictures of 2 Megabytes each.

So as I'm trying to lead by example (uber-smug, ain't I?), when sending pictures I took with a digital camera myself I reduce those images to around 600 pixels along their larger dimension, which will do nicely in most cases and save the other party the trouble not only of downloading huge files but also of staring at a tiny irrelevant detail of the picture that happens to be in the top left corner after the image viewer has ceased up their entire screen.

The best interactive tool for the occasional view-crop-correct-resize-compress task is obviously gthumb, but when faced with a larger number of images, it's commandline-time:

mogrify -quality 82 -resize 600x600\> *.jpg

This will shrink all jpg files in the current directory to fit within a 600x600 pixel bounding box while preserving aspect ratio (so the images are not distorted as one might first think). The closing angle bracket at the end ensures that images that are smaller than the target size are not enlarged. Note that the mogrify command changes the images in place, so you may either want to make sure to have a backup of the original files, or go for find/convert instead.
For more information, here is the resize operator's official documentation.

Another little-known but very excellent feature of the ImageMagick suite is the -strip operator, which will remove all EXIF data from an image. This can be useful especially when the picture was taken on a modern smartphone which will normally embed all sorts of information in the image's metadata such as location and time of the shot, as well as the model name of the device the image was taken with. Maybe you wouldn't want to share that with everyone that should see the picture.

Lowering the (re)bar

Although the Erlang programming language has been on my radar for many years now, I never quite grasped the concept of how to put things together such that a bundle of functionality might be properly deployed to and run on a target system. The target system would of course be Linux :)

Fortunately, in the last couple of years an effort has emerged that promises to make things easier. It's a tool called rebar, and its canonical use seems to serve as the build system for a database application by the name riak. Some of the participant's names on the rebar mailing list ring familiar, so it seems possible that it may even be adopted into OTP at some point.

There are quite a number of "getting started" tutorials around for rebar, one right inside the rebar github wiki, and also this one being still quite recent early in 2012. rebar is (or at least has been) a moving target, and some tutorials you may find are badly outdated.

However even following the supposedly applicable instructions, I invariably ran into this error at the generate stage:
$ ./rebar generate
==> rel (generate)
{"init terminating in do_boot","Release exemplar uses non existing application exemplar"}

Crash dump was written to: erl_crash.dump
init terminating in do_boot (Release exemplar uses non existing application exemplar)

It took me quite a while to get past this problem, and the key here is that the Erlang runtime needs to accept your working directory as an application, which requires two things to be true:

1. The directory that contains the src and ebin directories, as well as rebar.config (not rebar.conf btw!) needs to have exactly the same name as the Erlang application you are creating.
2. The directory which comprises that application directory needs to be part of Erlang's lib_dirs (an extension searchpath for /usr/lib/erlang/lib) so it will be considered by the runtime when it is looking for applications.

What is left out by all the tutorials I found is that for #2 to become true, one needs to manually edit the reltool.config file that is created by the rebar create-node step and edit the lib_dirs assignment to include that parent directory (can be an absolute path, or given relative from the rel directory).

So if you are trying this out in /tmp/exemplar like

/tmp/exemplar
|-- deps
|   `-- exemplar
|       |-- ebin
|       `-- src
|           |-- exemplar_app.erl
|           |-- exemplar.app.src
|           `-- exemplar_sup.erl
|-- rebar.config
`-- rel
    |-- exemplar
[...]
    |-- files
    |   |-- erl
    |   |-- exemplar
    |   |-- exemplar.cmd
    |   |-- nodetool
    |   |-- start_erl.cmd
    |   |-- sys.config
    |   `-- vm.args
    `-- reltool.config

then your /tmp/exemplar/rel/reltool.config file should be changed to start like

{sys, [
       {lib_dirs, ["/tmp/exemplar/deps"]},
[...]

At first I was wondering whether this manual edit is really the right thing to do since this file is generated, but checking out the riak rel directory, the reltool.config file there is checked into git and also contains a manual lib_dirs assignment.

Another mistake to avoid is naming your node (rebar create-node nodeid=<name>) anything different from your main application, which will result in an obscure error message

Command 'generate' not understood or not applicable

Adding a user in mysql for a specific database

Whenever installing some new webapp on a LAMP stack, the typical procedure goes such that the app gets its own database in mysql/mariadb/drizzle/whatever and needs a user to access it.

The database is of course created like

create database <database>;

To add the user, one usually uses a command like this in the mysql client shell:

grant all privileges on <database>.* to '<new_user>'@'localhost' identified by '<some_password>';

In which I like to use a randomly generated password (e.g. using makepasswd) for each user.

Besides assigning rights, the latter command also creates <newuser> as a side effect, and even flushes privileges in one fell swoop.

One might further want to prune the granted rights to only work on the actual data, not allowing modification of the tables/schema, but most webapps come with an installer built in that creates/updates the app's tables so this is left as an exercise for the reader.

autossh / screen without the hassle

$ autossh -M 0 <host> -t 'screen -x'

Coping with gnome3

The most important thing to know is of course gnome-tweak-tool, which is pretty much self-explaining once you know it's there.

On less capable machines, fallback mode can be quite cozy too, provided you can do with the limited number applets available. For laptops where vertical screen estate is scarce, know that the panel (bar at the bottom) can still be set to autohide like in the olden days, however it must be clicked with Alt+RightMouseButton to bring up a preferences menu that actually includes this option. Same goes for the top menu, which can hold application icons in case you forgot.

To get rid of the annoyance that is edge tiling (the fact that a window goes fullscreen whenever it happens to touch the top of the screen while you're trying to move it around), one can use gconf-editor and remove the tickmark from /desktop/gnome/shell/windows/edge_tiling.

A great summary of Tips'n'Tricks on this topic is found here. It is based on Mint Linux which deviates somewhat from stock gnome3, but the guide is still very useful.

google-musicmanager -p

Thankfully I did not have to face this situation again with google music manger's reluctance remembering the password when using 2-step authentication as a (supposedly) nice guy wrote about the -p command line option here. Simply have it followed by an application password and you're good to go.
Let's hope it'll be just a short while till Google fixes music manager to support 2-step properly...