This has been an on-and-off project for almost two years (version 1.0 described here). The purpose is simple: find images similar to a given image. Since it is not even well understood what images are similar, the progress in this area of “image-to-image” search (aka “visual image search”) is very slow-. So, instead, we focus on the goal of finding modified versions of the original. This release is a way to report a limited success we have achieved.The executable PxSearch.exe is accompanied by a small collection of images (download here, 7.2 MB). The system consists of the following modules:

• the collection of images that can be extended; 
• the database containing “signatures” of images, images’ origins, and other data; 
• the image analysis unit (produces the signatures); 
• the matching unit (matches the signatures); 
• user interface (uploads an image, searches for similar images in the collection, displays the matches as a list);

For every image to be added, first the image is converted to grayscale and then shrunk so that the larger dimension is 150. Then several of its secondary versions are created, analyzed, and added to the collection and their data is added to the database, total of 8:

1. original 
2. rotation, 5 degrees 
3. rotation, 45 degrees 
4. Gaussian blur 
5. salt and pepper noise 
6. stretch, 5% 
7. shrink, 5% 
8. crop from all sides, 5%

The entry in the database for each image contains the information about its origin:

• date and time, 
• the filename of the original image, 
• the way the image was produced from the original (shrinking, rotation, etc), 
• the signature of the image.

A signature is a sequence of 126 integers which is the output of image analysis: it is essentially the distribution of sizes of objects found in the image (the data comes form the same source as for Pixcavator).

Suppose the signature of the two images are {An} and {Bn}. Move along these sequences and compute the absolute value of the differences of n-th entries. The result is a distance formula as the “weighted 1-norm metric”:

D = Σ Cn |An - Bn|.

A search is deemed successful if most of the versions of the query image are at the top of the list. This is the case for images that are “good” in the sense that they have clear pattern (based on shapes not color). However, this standard is hard to quantify as it is dependent on the collection. Since the collection I used for testing was small (4500 images), I had to find a way to evaluate the quality of searches that is independent of the size of the collection, as much as possible. So, the quality score for a given image was

(average distance to its 7 versions) / (average distance to all images) * 100.

There are many interesting question to study based on this data and I will report further.

Inhibition of Experimental Corneal Neovascularization by Bevacizumab (Avastin) by Yonca A. Akova, MD Veysi Öner, MD Cem Küçükerdönmez, MD. The paper measures the affect of a certain drug on corneas of rats.

Análise da qualidade das sementes de Calophyllum brasiliensis Cambess utilizando eletrofotografia (PUIC) by Ana Paula Nandi Ferreira (PUIC, Curso Agronomia, Tubarão), Jasper J. Zanco (Orientador, Curso de Agronomia, Autor do Projeto) (in Portuguese). The paper analyzes the quality of seeds.

Quantitative Fluorescence Microscopy Based Screening by Kees Straatman (University of Leicester). The paper reviews some aspects of HTS (High throughput screening) and HCS (High content screening).

Gazopa is a new visual search engine that is “a venture project inside Hitachi”.

I tried its Facebook application. I uploaded a few standard images and a few test images of my own and ran Gazopa. Some of the matches were awful while others were sort of meaningful. See for yourselves. The first match is displayed under the target image.

Gazopa also found a cropped copy of the “cameraman”, but not a rotated copy. The inability to handle rotations is a common problem with almost all visual search engines. Pixcavator Image Search can handle rotations with ease (read about it here or wait for the last version - to be released soon).

As far as the underlying technology, the site says that “GazoPa enables users to search for a similar image from characteristics such as a color or a shape extracted from an image itself” and nothing more. So, even what they consider similar is unknown. 

As I have mentioned before, I am at the initial stages of writing a book on elementary computer vision (see part 1, part 2,  and the wiki). After Digital Image Processing Using MATLAB by Gonzalez, Woods, and Eddins, another one to consider is Computer Vision by Shapiro and Stockman.