Configuration files in Python

There is a very nice wrapper for libconfig for python, right here https://github.com/keimoon/python-libconfig
(This is connected to the previous post on libconfig here)

Why class balancing happens 'automatically' in AdaBoost

This is a very interesting characteristic of AdaBoost, that explains why, in general, it is not necessary to balance the initial weights during training for very skewed datasets.

I will use the notation from the AdaBoost Wikipedia article.

First, assume that we have $N_p$ positive samples and $N_n$ negative samples, with $N_n >> N_p$, representing a skewed dataset. Let's name the ratio between the number of negative and positive samples as $K = \frac{N_n}{N_p}$.

Now, assume that we are at the first iteration, and we have uniform weights $D_{t=1}(i) = \frac{1}{m}$ for all training samples. At the first iteration $t=1$ the goal is to find a weak learner

$h_{t} = \underset{h_{t} \in \mathcal{H}}{\operatorname{argmax}} \; \left\vert 0.5 - \epsilon_{t}\right\vert$, with $\epsilon_{t} = \sum_{i=1}^{m} D_{t}(i)I(y_i \ne h_{t}(x_{i}))$

since all weights are equal, $\epsilon_t$ can be expressed as $\epsilon_{t} = \sum_{i=1}^{m} I(y_i \ne h_{t}(x_{i}))$, so it is just the unweighted misclassification error.

Now it comes the interesting part: assume that the weak learners in the weak learner pool are very weak, so that the lowest misclassification error $\epsilon_t$ is achieved with a weak learner that classifies all samples as negatives. (This seems unreasonable, but I have seen it happen frequently on skewed datasets).

With this in mind, $\epsilon_1 = \frac{N_p}{N_p + N_n}$. With simple math we can compute the value of $\alpha_1=\frac{1}{2} \log \frac{1 - \epsilon_1}{\epsilon_1}$ and update the distribution $D$ for iteration $t=2$, obtaining

$$\frac{D_{t=2}(+)}{D_{t=2}(-)} = \frac{N_n}{N_p} = K$$

where $D_{t=2}(+)$ and $D_{t=2}(-)$ are the weight of the positive and negative samples respectively.

This means that the effect of the first weak learner was to balance the dataset, so that the sum of the positive weights equal the sum of the negative ones. Another equivalent interpretation is that the first weak learner just adds a constant to the predictive function, namely $-\alpha_1$ in this particular case.

However, there are some differences w.r.t. balancing the weights initially:

• The first weak learner is 'wasted' and could be avoided.
• If shrinkage is introduced, it is likely that many weak learners will be 'wasted'. Though 'wasted' is not very meaningful, because they would still be useful and help minimize training loss.
• The fact that the first weak learner returns -1 for all training samples does not imply that it will also for any unseen new sample. The implications of this are hard to tell though.
• I am assuming that a weak learner can return the same value for all training samples, which is the case for a decision stump, but it may not happen with other types of weak learners (e.g. svm)
• .. I am pretty sure there are other differences

Cheers!

C++11: compile-time lookup-table/array with constexpr

I've been trying a few of the new features of C++11, wondering how difficult it would be to create a compile-time self-populated array.

Here there is a piece of code that creates an array that contains the values of  sin(0.2 x) + cos(0.5 x), completely at compile-time, and then prints it in ascii form to stdout. The most tricky part is the one containing a variadic template, which computes a set of indices from 0 to N-1.

You can see the output it produces here

#include <iostream> #include <vector> #include <string> #include <cmath> #include <iomanip> /** Range generation, * from http://stackoverflow.com/questions/13313980/populate-an-array-using-constexpr-at-compile-time **/ template<unsigned... Is> struct seq{}; template<unsigned N, unsigned... Is> struct gen_seq : gen_seq<N-1, N-1, Is...>{}; template<unsigned... Is> struct gen_seq<0, Is...> : seq<Is...>{}; /** A table consisting of indexes and values, * which will all be computed at compile-time **/ template<unsigned N> struct Table { unsigned indexes[N]; double values[N]; static constexpr unsigned length = N; }; template< typename LambdaType, unsigned... Is> constexpr Table< sizeof...(Is) > TableGenerator(seq<Is...>, LambdaType evalFunc) { return {{ Is... }, { evalFunc(Is)... }}; } template<unsigned N, typename LambdaType> constexpr Table<n> TableGenerator( LambdaType evalFunc ) { return TableGenerator(gen_seq<n>(), evalFunc); } /** Function that computes a value for each index **/ constexpr double myFunc( unsigned idx ) { return sin(0.2 * idx) + cos(0.5*idx); } int main() { constexpr unsigned length = 100; // create compile-time table constexpr Table<length> table = TableGenerator<length>( myFunc ); // print values in vertical form, pretty-looking ;) const double lineMult = 12; const double lineOffset = 30; for(auto v : table.values) { const unsigned numSpaces = (unsigned) ( lineOffset + v * lineMult + 0.5 ); std::cout << std::setfill(' ') << std::setw( numSpaces ) << "o" << std::endl; } std::cout << std::endl; return 0; }

Wacom Pen & Touch CTH670 and Linux

I've just got a Wacom Pen & Touch tablet CTH670. I could not make it work straight away with Archlinux, but then found out that I had to follow the steps in this forum post , which was originally meant to be for the CTL-470/k.


In case the attachment in the linked forum post doesn't work, I am duplicating it here.


UPDATE: there seems to be a small issue with that version posted above. The version that works like a charm with my tablet is http://sourceforge.net/projects/linuxwacom/files/xf86-input-wacom/input-wacom/

Trying to read/write configuration files?

Loading, parsing and writing configuration files is generally not a trivial task. I found this amazing library recently, which does the job with a few C++ lines.

For example, if you have a configuration file such as:

#---------------------------- # Example Configuration File #--------------------------- # application: { /* This section defines some settings for our * main application window, such as size and * position. */ window: { title = "My Application"; size = { /* width */ w = 640; /* height */ h = 480; }; pos = { x = 350; y = 250; }; }; a = 5; b = 6; } Fields can be easily read with something as simple as:

Speeding up Matlab over X11 Forwarding

Running Matlab over ssh is handy. If one wants to use the GUI, X11 forwarding is possible, but it is painfully slow if both machines are not inside a local network.

I found information about how to speed it up after googling for a while. In the end I think it is worth it to summarize the steps here:

• In the current folder where you are going to run matlab from, create a file name java.opts with the content 

                   "-Dsun.java2d.pmoffscreen=false" (without the double commas).

• Use compression with ssh, by connecting to the server with

                      ssh -c arcfour,blowfish-cbc -Y -C username@server

The speed up obtained is particularly noticeable when changing focus between different windows in the Matlab GUI.

GNOME3 Fallback mode 'fix'

I switched to archlinux recently. I was used to Gnome2, and Gnome3 is quite of a disappointment in the sense that it involves re-learning how to do things, when software is supposed to adapt to us. Anyway, I don't plan to start a discussion on that, googling already reveals many 'gnome3 sucks' posts which provide lot of detail on the 'why'.

In summary, I tried KDE4, but didn't feel comfortable either. Finally I ended up in GNOME3 fallback mode, whose default theme is disappointing as well. I found this theme which makes it look friendlier http://gnome-look.org/content/show.php?content=145210 , but had some problems in nautilus and background font color. So I modified it and uploaded it here.

Now my desktop reminds me of Gnome2 and sometimes I believe it is even friendlier and better looking ;)