My 404s Don't code and drive

Setting Install Directory When Using CMake

Sometimes we want to install a build to a different directory. In CMake this can be done by setting the CMAKE_INSTALL_PREFIX variable. When we are building OpenCV3 for examle we would do,

cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ../opencv-3

Batch Crop Images Using ImageMagick

ImageMagick comes with a tool to perform batch image transforms, it is called mogrify. To crop a specific rectangular area in all images in a directory the formula looks like this,

mogrify -crop [rectWidth]x[rectHeight]+[rectTopX]+[rectTopY] -path pathToStoreCroppedFiles pathToImgDirectory/*.extension

A realization of this might look like this

mogrify -crop 300x300+290+170 -path ./myCroppedFilesDir *.png

This crops a 300x300 rectangle whos top is positioned at 290x170 from all the png images in the current directory and saves them to the directory myCroppedFilesDir(make sure the dir exists before you run).

Fisher Vector & Caffe C++ Implementation

I implemented a C++ pipeline for learning Fisher feature vectors using VLFeat since Matlab should be avoided whenever possible. I managed to find Python bindings later.

The code does the following from a set of labeled images it extracts dense SIFT features. It finds the PCA representation of the SIFT reducing the dimension from 128 to 80 (or whatever dimension you want). It then computes the GMM clustering using the EM algorithm. From the learned the GMM it computes the Fisher vector representation. We can also add additional features from a text file, such as a sub layer from a CNN, like Caffe for example. After computing the feature vectors the code trains a linear SVM, one for each class label. The code saves all representations so that inference can be run on new images.

The code use CMake for compilation and requires the Boost, OpenCV and Eigen libraries in addtion to VLFeat. To train on some subset of images we can run from the terminal

./fisher train 

and to test new images

 ./fisher test

Just make sure you set the data paths correctly. You can find the code here.

Concatenating files horizontally

Lets say you have a set of text files that contains column entry data, for example a set of csv files, or text files. Each row in each of the csv files contains data entries relating to the same instances. We would like to concatenate all the csv files into one csv file. We can do this in the terminal using the paste command, like this,

paste file1.csv file2.csv | column  > X.txt

If we want to remove the , ; \t or anuy other character that usually separates csv files we can tell column to create a table separated by white space. Suppose the data in our csv files are separated by ; we can do

 paste file1.csv file2.csv | column -s ';' -t > X.txt

Somtimes this generates an error since the paste command can only handle 512 character long lines(if I remember it correctly). A better approach is to use this script which I found on StackOverflow

awk '{if ((getline a < "-") > 0) $0 = $0 ";" a; print}' f1.csv < f2.csv > fconcat.csv

where ; is the separator for the csv file.

Iterating over Lists of Lists OR Creating Nested For loops in Python

Lets say you have a list of lists containing values that you want to iterate over. The problem is that the number of lists keeps on changing so you can’t really make it into a hard coded set of nested for loops. What you really need is to dynamically create a set of nested for loops. In Python you can use the itertools for these kind of jobs. Extremly handy if you want to do grid search over a set of values for example. In code it might look like this,

# Python3
import numpy as np 
from itertools import product

# Create a list of np array values  
rangeList = []
rangeList.append(np.linspace(0.,10.,10))
rangeList.append(np.linspace(40.,1000.,10))
rangeList.append(np.linspace(10.,40.,10))

# Iterate over all of the lists values 
for vals in product(*rangeList):
    for val in vals:
        print(val,end=",")
    print(" ")

this will print the following,

>>  0.0,40.0,10.0,
    0.0,40.0,13.3333333333,
    0.0,40.0,16.6666666667,
    0.0,40.0,20.0,
    0.0,40.0,23.3333333333,
    0.0,40.0,26.6666666667,
    ....