Excellent good-looking plots

When doing research one usually doesn't care about the aesthetics of plots or the visualized results, as long as they are clear enough to be interpreted. However, when making presentations or in publications, nice plots have an interesting impact and, even though they do not change how good results are, they still make things look more professional.

I have been usin Matlab for a while now. It is one of those softwares that you usually hate, especially if you come from a better-structured programming background. Leaving execution speed out of the equation, Matlab sucks in many ways but there a few pros that build up its popularity, such as extremely easy and straightforward debugging and the available toolboxes and functions on Matlab Central. However, it is very easy to find blogs like Abandon Matlab, where some posts really make the point about leaving Matlab forever and finding a better and more appropriate alternative.

Anyways, enough of Matlab hate. The point is that I was looking for nice plots and I remembered about that amazing piece of software called Mathematica. I will not discuss the differences between Mathematica and Matlab, but just say that they were made for different purposes. However, take a look at the following plots generated with Matlab and Mathematica respectively, from the same data (click to see the real image since blogger is automatically introducing some JPEG artifacts):

Matlab	Mathematica
load data.mat; stem(x,y); hold on; xlabel('x','Interpreter','LaTex'); ylabel('f(x)','Interpreter','LaTex');	data = Import["test.mat", "LabeledData"]; ListPlot[ Transpose[Flatten[{"x" /. data, "y" /. data}, 1]], Filling -> Axis, AxesLabel -> {x, f[x]} ]

The difference is easy to grasp by just looking at the plots: Mathematica does a great job, while Matlab looks just ok. Something I usually do to make Matlab plots is to apply a grid with grid on, but the plot still looks not as professional as with Mathematica. Obviously there is space for cheating here; maybe if you look at how much Mathematica code is needed you may say that I haven't been fair enough. However, most of the code in the Mathematica snippet is needed because the data is read from a Matlab MAT file.

In my opinion, the most amazing and simple detail that Mathematica uses and Matlab does not is antialiasing. It is a very subtle detail but it makes plots look softer and, somehow, more human and easier for our eyes to watch. There have been some attempts such as this script. However, that script is an smart user attempt to generate a plot with antialiasing but the antialiasing is simulated by resizing the plot, which still doesn't look as good as Mathematica's output (images not shown here but you can try it on your own).

Beyond that, there are some problems when exporting plots from Matlab. First, exporting to PDF generates a PDF file that contains a whole page such as A4 and the plot in the middle with huge white spaces around. This is very annoying when working with pdflatex and the PDF generated by Matlab must be cropped. Moreover, even if exporting to PNG, the final image file does not look exactly as what is seen on the computer screen. This can lead to an endless 'fight' with Matlab's exporting options, sometimes without success. All of this doesn't seem to happen to Mathematica, at least considering what I have tried so far. PDF files look great and have the exact size of the plot and can be inserted straight away into latex documents.

I don't want to go into much detail in this post since it could take a long time to discuss plot customization options in Matlab and Mathematica. My experience shows that Matlab works well to show the data properly and it is still interpretable, but it lacks the quality of softwares such as Mathematica or Matplotlib (this last one is another very nice plotting tool, free of charge). For more examples on the plotting power of Mathematica see here, and for Matplotlib here.

Nice Linux PDF manipulation utilities

I found myself writing many reports with LaTex lately. Using pdflatex has it advantages, but things can get quite annoying when one wants to insert a figure which was generated in Matlab or any other program.

Specially, MATLab does not do a nice job when exporting PDFs and leaves a whole blank area (the page itself actually) which is not desirable if we want to put a figure in a latex document. Fortunately, there is a linux program called pdfcrop that does the job correctly (not 100% trustable, but 90% of the times I get good results).

Another useful program is pdfimages, which extracts images from a pdf file. The pdfimages output is generally in huge pgm files, so it's better to convert it to something like png which results in smaller files, still usable with pdflatex.

Finally, pdfjoin and pdf90 allows one to join several files into a single one and rotate pages respectively. The Ubuntu package for these two is called pdfjam.

Ultra simple incremental backups with rsync

Recently I bought an external hard drive for backups. While searching for the best way to do the backups I found rsync, which looks well suited for these tasks.
Then I found this webpage that provides some scripts to achieve circular snapshots.

However, most scripts on the web are extremely large and complex compared to what I need. So finally I made my own script that makes automated incremental backups, with no cycling but keeping a history file to identify each backup.

Here is the code:
<br />#!/bin/bash<br />##############################################<br />## This script should be called from the dir<br />## where the backup will be made to<br />###############################################<br /><br /># directory to backup<br />sourceDir=/media/something<br />rsyncFlags='--modify-window=1 -a --verbose --delete '<br /><br />#find if we have a backup already<br />find backup.0 -maxdepth 0 > /dev/null;<br /><br />if (( $? != 0 )); then {<br /> echo 'First Backup';<br /> nextBackup='0';<br /> firstBackup=1;<br />} else<br />{<br /> # find last backup<br /> lastBackup=`find backup.* -maxdepth 0 | sed 's/backup.//' | sort -n -r | head -1`;<br /><br /> echo 'Resuming backup' $lastBackup '...';<br /><br /> # +1 for next one<br /> nextBackup=$(( $lastBackup + 1 ));<br /> echo $nextBackup;<br /> firstBackup=0;<br />}<br />fi<br /><br />lastBackup=backup.$lastBackup;<br />nextBackup=backup.$nextBackup;<br /><br />if (( $firstBackup == 1 )); then {<br /> rsync $rsyncFlags $sourceDir/ $nextBackup/;<br /> R=$?;<br />} else<br />{<br /> rsync $rsyncFlags --link-dest=../$lastBackup $sourceDir/ $nextBackup/;<br /> R=$?;<br />} fi<br /><br />if (( $R != 0 )); then {<br /> echo '=======> ERROR DURING BACKUP!!';<br /> exit $R;<br />} fi<br /><br /># make link to last backup<br />rm -f HEAD;<br />ln -s -f $nextBackup HEAD<br />if (( $? != 0 )); then {<br /> echo '=======> ERROR Creating link!!';<br /> exit $?;<br />} fi<br /><br /># save history<br />echo -e $nextBackup '\t\t' `date -R` >> backup-history;<br />if (( $? != 0 )); then {<br /> echo '=======> ERROR Saving History!!';<br /> exit $?;<br />} fi<br /><br />echo ' ';<br />echo '------------- BACKUP DONE ----------------';<br />echo ' ';<br />

This script will create backup directories called backup.xxxx, where xxxx is the backup number. This number is zero for the first backup done. Rsync hard-links the unchanged files to the previous backup which is a key method to save space.

With this script I am backuping a NTFS partition and that's why the option --modify-window=1 is needed in rsyncFlags. It is important to write sourceDir without a trailing forward slash!

In order to provide easy access to the last backup, a symbolic link called HEAD will point to the latest backup. Additionally, a file called backup-history holds the exact time and date where each backup was performed.

NOTE: I am not responsible for this script and it's correctness. There might be problems such as if there are backup.something folders or files in the backup directory where the script is called, and other bugs that may arise. This is a very simple and minimalistic script!

Merging Qt and Eigen

Again in ViBOT, image segmentation assignment, Matlab is really slow, wait minutes for results...

So I decided to try to use Qt for the GUI and OS abstraction layer together with Eigen which is another amazing template-based library for matrix manipulation. The important code to write was to link both libraries, taking advantage of Qt's amazing QImage class which is able to open several file formats and perform low-level pixel access. In a few words, I had to put all the image information contained in QImage into a Eigen's matrix.

Luckily, this task is very simple. Here there is some code:

#ifndef MIMG_H
#define MIMG_H

USING_PART_OF_NAMESPACE_EIGEN

#include <QImage>

#include <Eigen/Core>
#include <Eigen/Array>

//general type, maybe float or double needed
typedef MatrixXf    MImgType;

class MImg
{
public:
 //creates an all-black image
 MImg(unsigned int h, unsigned int w);

 //creates image from QImage
 MImg( const QImage &img );

 MImgType    R,G,B;  //each component
         //made public for faster access

 unsigned int    getHeight();
 unsigned int    getWidth();

 QImage *    toQImage();  //convert to QImage

 /**
   Maximizes dynamic range of three channels
   independently!
  **/
 void    maximizeIndependentDynamicRange();

private:
 unsigned int mH,mW; //height, width

};

#endif // MIMG_H

#include "mimg.h"

MImg::MImg(unsigned int h, unsigned int w)
{
  R = MImgType::Zero(h,w);
  G = MImgType::Zero(h,w);
  B = MImgType::Zero(h,w);

  mH = h;
  mW = w;
}

MImg::MImg( const QImage &img )
{
  int w = img.width();
  int h = img.height();

  R = MImgType::Zero(h,w);
  G = MImgType::Zero(h,w);
  B = MImgType::Zero(h,w);

   //now copy values..
    for (int y=0; y < h; y++)
        for (int x=0; x < w; x++)
        {
            QRgb color = img.pixel(x,y);
            R(y,x) = qRed(color)/255.0;
            G(y,x) = qGreen(color)/255.0;
            B(y,x) = qBlue(color)/255.0;
        }

  return img;
}

void    MImg::maximizeIndependentDynamicRange()
{
  double  min, max;

  min = R.minCoeff(); max = R.maxCoeff();
  R = (R.cwise() - min) / (max - min);

  min = G.minCoeff(); max = G.maxCoeff();
  G = (G.cwise() - min) / (max - min);

  min = B.minCoeff(); max = B.maxCoeff();
  B = (B.cwise() - min) / (max - min);
}

unsigned int    MImg::getHeight() {
  return mH;
}

unsigned int    MImg::getWidth() {
  return mW;
}

It is important to mention that this code only handles RGB and won't care about grayscale images or any other type of colour models. The advantage of having the image in this matrix form is that Eigen provides an easy syntax for matrix manipulation, along with many modules performing least squares, Cholesky, diagonalization, etc.

Matlab and n-dimensional array sorting

Wow.. it's been such a long time since the last post!! I've been quite busy with ViBOT, specially during the last two weeks. Anyway, I thought it would be nice to write a post about some nice Matlab functions I found quite useful:

reshape:this is a nice function that lets you reshape any array or matrix into any other array or matrix with different dimensions, as long as the number of elements is kept the same. It is very useful when one wants to loop through every element of a two or three dimensional array. If A=[1 2; 3 4] then reshape(A,[1 4]) will return [1 2 3 4]. To get back to the original array we can then use reshape([1 2 3 4],[2 2]).

ind2sub: this is a useful function when manipulating arrays that were linearised with reshape. From Matlab help: "The ind2sub command determines the equivalent subscript values corresponding to a single index into an array". A simple example is the following:

• A=[1 2; 6 5]; B = reshape(A,[1 4]);
• [sV, sI] = sort(B, 'descend'); % sort linearised array
• disp(sV(1)); %show max value: 6
• disp(sI(1)); %show max index: 2
• [x,y] = ind2sub( size(A), sI(1) );
• disp(x); disp(y); % (x,y) == (2,1), we got the coordinates in the original matrix

There is also a sub2ind function which does the reverse transformation. ind2sub was very useful to ease the sorting task when applying the Hough transform for circle detection, where there is an accumulator matrix which is 3-dimensional.

FPGAs are taking over!

From the moment I knew and learned about FPGAs (Xilinx) I looked forward to use them to replace large logic circuits. This way the system would be not only scallable and the logic programmable but costs should be reduced too.

Unfortunately, most of the times the FPGA alternative was much more expensive than the equivalent logic circuit implemented with separate logic ICs.

Finally the day came and I got the chance to build a board with an ARM7 core + Spartan3A FPGA. Total price was reduced and the system became fully programmable. The ARM7 chip (LPC23xx) configures the FPGA on startup which happens to be really fast (52kib for XC3S50A). The microcontroller and FPGA are connected through a parallel bus with many control lines.

The XC3S50A is optimal in the sense that it only requires 3.3V and 1.2V supplies so it can be directly connected to the microcontroller pins.

Here there are some pictures:

Qt-Embedded: Capturing screen with QPixmap

I've been working on a product manual lately. I needed to include several LCD screenshots into it so I tried to come up with an easy way to capture snapshots from our Qt/Embedded app.

Qt/Embedded provides a nice method to save window/framebuffer contents directly to an image file. However, I wanted to send the 'take-snapshot' command from a tty console (telnet/serial/etc) since there weren't any other buttons on the system to trigger that.

A QTimer is set up. Periodically it checks the file /tmp/doCapture. If it exists a snapshot is taken and an image file is saved. Its filename is taken from the contents of /tmp/doCapture. After saving the image /tmp/doCapture is deleted.

Here is the code, which should be placed in the main window, whose width and height cover the whole screen:

mainWindow::mainWindow()
{
 // captureTimer should be declared in mainWindow's class definition
 captureTimer = new QTimer(this);
 connect( captureTimer, SIGNAL(timeout()), this, SLOT(captureTimerEvent()) );

 captureTimer->start(1000); //check interval
}

void mainWindow::captureTimerEvent()
{
 QString tmpFile = QString("/tmp/doCapture");

 if ( !QFile::exists(tmpFile) )
     return;

 QFile f(tmpFile);
 if ( !f.open( QIODevice::ReadWrite ) )
     return;

 char buf[200];
 if ( f.readLine( buf, sizeof(buf) - 4 ) == -1 )
     return;

 buf[strlen(buf)-1] = '\0'; //remove \n created by 'echo'-- not safe!

 strcat( buf, ".png" );

 //capture
 QPixmap p = QPixmap::grabWindow( this->winId() );

 if ( p.save( buf ) )
     printf("------- GRAB OK\n");
 else
     printf("------- ERR GRAB!\n");

 /* delete file */
 f.remove();
}

This way, all I have to do to take a snapshot is to write:

echo pngfilename > /tmp/doCapture