Wikipedia’s article lists image analysis software in the form of a table. The columns are:

• Product  
• Developer 
• Cost (USD) 
• Open source 
• Software license  
• OS 
• Continuous 
• Industries, Uses and Applications

I thought that the type of license, open/closed source, OS, etc aren’t very interesting, while some more important, in my view, data should be added. Based on the inspection of the vendors’ sites, I tried to answer the following questions.

What is the price? In the absence of that information, I put $$$$ indicating that the price might be in the thousands.

Is there a free version available for download? Companies with $$$$ usually don’t have that.

Does the site provide examples of how the software has been used for image analysis? Very often, surprisingly little is provided.

Does the site reveal the methods/algorithms behind the software? Commercial vendors say nothing. Open source certainly qualifies for Yes in this category but most of the time the source is all you get. The actual math, algorithms, errors, etc are ignored.

To see the new table follow this link: Image analysis software.

These images came from researchers in medical image analysis. They represent “low-intensity multi-spectral image of the tumour in the early stage of development”. Their algorithm has been patented and the results are published in Medical Image Analysis.

The images below came from the same source and show the results of analysis of the first image by means of their algorithm (right) vs. Pixcavator’s (left). The comparison is unfavorable.

I ran Pixcavator myself with the first image, high intensity. I had to move the maximal contrast slider and in about 5 seconds I had a satisfactory result, below.

The low intensity image is here.

A new paper that uses Pixcavator:

Down-regulation of CXCR4 and CD62L in Chronic Lymphocytic Leukemia Cells Is Triggered by B-Cell Receptor Ligation and Associated with Progressive Disease [1] by Amalia Vlad, Pierre-Antoine Deglesne, Re´mi Letestu, Ste´phane Saint-Georges, Nathalie Chevallier, Fanny Baran-Marszak, Nadine Varin-Blank, Florence Ajchenbaum-Cymbalista, and Dominique Ledoux (Cancer Research 69, 6387, August 15, 2009).

From the paper:

“Progressive cases of B-cell chronic lymphocytic leukemia (CLL) are frequently associated with lymphadenopathy, highlighting a critical role for signals emanating from the tumor environment in the accumulation of malignant B cells.”

“BCR-stimulated and unstimulated fluorescent cells were mixed in RPMI 1640/10% FCS and added together onto the endothelial cell layer. After incubation for 2 h at 37jC, the nonadherent CLL cells were washed off. Remaining adherent cells were fixed, and 10 fields from duplicate chamber slides (average of 500 cells/field) were photographed under fluorescent microscope. Red and green fluorescence were separately quantified using the Pixcavator IA 3.3 software (Intelligence Perception Co.).”

Take a look at other papers that use Pixcavator.

I was about to review the newly released Google Similar Image Search when I ran across this one. The verdict: not so good.

The guy does not seem to realize though that Microsoft released its own similarity search a few months before. I am not judging because I missed it myself when it came out. It would be interesting to test and see which one is better (or not as bad). One point in favor of Microsoft is that Google didn’t index all images.

UPDATE: Another good revew at Rich Marr’s Tech Blog.

Photoshop CS4 from Adobe Systems is a powerful image and photo editor but not a tool for scientific image analysis.

The software has a huge multitude of tools for image processing (and, of course, photo manipulation). There is no point in listing them here. The “extended” version also has a few fun features like auto-blending or content-aware scaling.

However, its image analysis capabilities are very limited. After searching for a while, just these two below all I found:

Use selection tools to define and calculate distance, perimeter, area, and many other measurements. Record data points in a Measurement Log and then export the data, including histogram data, to a spreadsheet for further quantitative analysis.

Easily and accurately tally objects or features in scientific images with the Count tool, which eliminates the need to perform manual calculations or rely on visual assessments of changes from image to image. Save even more time by performing multiple counts in a single image. Use separate colors for each count and save your counts in the file.

Adobe Photoshop CS4 Extended is priced at $999.

As I have mentioned before, I am at the initial stages of writing a book on elementary computer vision (see part 1, part 2, part 3). The alst one was Computer Vision by Shapiro and Stockman. Now a few thoughts about Computational Homology by Kaczynski, Mischaikow, and Mrozek (Springer, 2004).

The link to this demo was sent to me by Ricardo Niederberger Cabral (thanks!). The demo program is called Vision4 and was created by Numenta. This is its main point:

This program demonstrates some capabilities of Numenta’s Hierarchical Temporal Memory (HTM) technology applied to visual object recognition. .. The HTM network contained in this demo has been trained to recognize four types of objects: cell phones, sailboats, cows, and rubber ducks.

Every image is given four ratings. Each represents how much the image resembles one of the four types.

As you can see, the goal is modest and there are no unsubstantiated claims of how this is ready to be applied in real life (and don’t get me started on academic publications!). This is refreshing. The program is also fun to play with. You can load your own images, you can add noise, blur etc to the images and see the effect on the recognition. The recognition results are often good and when they aren’t, it’s still interesting.

For serious purposes, it is unclear where this is going though.

It’s fine with me that there are only four categories – just one would be enough to test the concept. It does not bother me when a face is rated high in the cow category and another face high in the duck category. My main complaint is the instability of recognition under image transformations. For example, after turning “sailboat” a few degrees it became “cell phone”. A few degrees more and it becomes mixed – half “cow” (first image below). Adding noise, occlusion, etc has similar effect (second image).

Certainly, one does not expect rotations to affect image recognition. Meanwhile, a mixed recognition is a failed recognition and should be presented as such.

I am certainly biased here. I don’t believe in “build[ing] machines that work on principles used by the brain”. I don’t believe in trying to imitate brain and I’ve written a few times about that. Traditionally, a scientist tries to understand nature by observing it, analyzing it, etc. Instead, it is suggested to try to understand nature by first understanding how the brain understands it? Seems like a roundabout to me, bordering on a vicious circle. I also have serious reservations about the use of machine learning in computer vision.

Annoying bug: every time I start it, the program would turn on my webcam and it would keep it on even after I shut it down.

TechCrunch is happy to do PR for another visual search company: Milabra.

Milabra claims that it can categorize images, “from puppies to porn”:

…when searching through a library of images for dogs, Milabra doesn’t need to constantly compare each image with its database of known ‘dog’ images – instead, it can look for traits that it has learned to associate with “doggyness”…

The two examples in the demo are “beach” and “dog”. You upload an image with people on the beach, click “Search” and you get a page of beach photos… Wait, you don’t get to upload anything – this is just a video! So, there is no way to test their claims. Unfortunately, this is not unusual in this area and in computer vision in general.

If your software can recognize a puppy in an image (95% of the time as you claim), it should be easy for you to demonstrate this ability. Create a little web application (or desktop, I don’t care) that allows me to upload my own image which is then identified as “puppy” (or “tree”, or “street”, I don’t care). There is no such program. Why not? The answer is obvious.

In response to some skepticism, this is what one of the founders wrote:

…if you think that this cannot be done, then you are completely clueless: object classifiers have been made for more than 10 years now at leading CS labs around the world.

That reminds me of the episode of Seinfeld when Kramer decides to build levels in his apartment:

KRAMER: It’s a simple job. Why, you don’t think I can?

JERRY: Oh, no. It’s not that I don’t think you can. I know that you can’t, and I’m positive that you won’t.

This is Millabra’s team:

• MBA
• MS in Biological Engineering and PhD in neuroscience
• MS in Computer Science and Ph.D. in Biophysics
• Professional Project Manager
• Expert in computer networking, user interface design

JERRY: I don’t see it happening.

And what about TechCrunch? Same story again and again since I started to keep track a couple of years ago: they publish an enthusiastic report about a company doing image analysis/search/recognition, and then silence. The company slips into obscurity and there is no follow-up, nothing. These people never learn…

The people who do seem to learn, slowly, are the investors: Riya (like.com) $20 million or more, Polar Rose $5 million, Milabra $1.4 million. Or maybe this is just the effect of the economic downturn?

Gazopa is a new visual search engine that is “a venture project inside Hitachi”.

I tried its Facebook application. I uploaded a few standard images and a few test images of my own and ran Gazopa. Some of the matches were awful while others were sort of meaningful. See for yourselves. The first match is displayed under the target image.

Gazopa also found a cropped copy of the “cameraman”, but not a rotated copy. The inability to handle rotations is a common problem with almost all visual search engines. Pixcavator Image Search can handle rotations with ease (read about it here or wait for the last version – to be released soon).

As far as the underlying technology, the site says that “GazoPa enables users to search for a similar image from characteristics such as a color or a shape extracted from an image itself” and nothing more. So, even what they consider similar is unknown. 

As I have mentioned before, I am at the initial stages of writing a book on elementary computer vision (see part 1, part 2,  and the wiki). After Digital Image Processing Using MATLAB by Gonzalez, Woods, and Eddins, another one to consider is Computer Vision by Shapiro and Stockman.

The screenshot tells the whole story. The image of a table in the upper left corner is the query image. The rest are supposed to be “similar”. What is the image filled with numbers doing here you ask? Hmm… Oh yes, it’s a table of numbers!

Previous posts on the topic are here.

In a recent post I made a couple of comments about MATLAB’s choice as the software for Digital Image Processing Using MATLAB by Gonzalez, Woods, and Eddins. I listed “MATLAB is ubiquitous” as a pro and “MATLAB is expensive” and “MATLAB is good for education and possibly research, problematic for industry” as two cons. Coincidentally, there was a discussion on Hacker News centered on MATLAB. Here are some of the comments:

… designed for engineers (I mean engineers, not computer programmers) to explore matrix models interactively, then save their work as scripts – you were never meant to use m-files for general purpose programming… What you’re paying for with MATLAB is access to the Mathworks Toolboxes. If you need them then it’s absolutely worth every penny.

Matlab is, to the programmer with experience in almost any other language, a tremendous horror… That being said, if you have the mathematical chops to rearrange your problem into something solvable via matrix transformations, you can probably write it quickly and elegantly in Matlab without worrying too greatly about execution speed. Better, the built in toolboxes have already solved huge (engineering) problem spaces… Prototype the math in Matlab, implement in a language that doesn’t suck.

.. the matrix/vector/tensor core is very elegant and powerful.

You’re probably unlikely to write a real application in Matlab… when you pay for Matlab, you pay for the assurance that the implementation of the tools provided is correct and therefore your research is based on a proven foundation.

My biggest complaint about Matlab (besides the licensing) is that it’s just a horrendously bad programming language (if you can call it a language at all)… you have to buy a toolbox for everything…

Matlab is really really, annoyingly powerful. You’ve got almost anything to try your ideas, implement an academic paper. But, it is slow (execution time). Prototype with Matlab, implement with C++.

Even though image processing/analysis wasn’t discussed, it is good to see that my assessment is confirmed.

I can add a few things though.

Matlab does some “rearranging“ of image manipulation (games) into matrix operations, they can’t be analyzed as matrices as they aren’t subject to matrix operations. They are in fact tables.

Matrices, yes. Linear algebra? Linear maps, kernels, images? Not as much. Quotient spaces? No. How am I supposed to compute homology of images?

Also, there are no pointers. So how am I supposed to implement graph representations of images?

So, MATLAB is mathematically powerful but only if you understand mathematics very narrowly.

Eventually I’ll add this to the wiki under MATLAB.

As I mentioned in the last post, I am at the initial stages of writing a book on elementary computer vision. It makes sense at this point to provide a rationale for such a book.

Current textbooks either have extensive prerequisites or take too long to get the student to use what’s been learned in real-life computer vision projects.

Let’s consider an example. Suppose we know freshman or sophomore students in a technical discipline. They have to take their first course in image processing. What are they capable of doing at the end of a typical course? They know about image representation and how to handle image files. They know how to increase contrast and remove noise. They are familiar with image restoration, image enhancement, and image compression. All good, but this choice of topics draws students toward photo editing and away from the scientific and industrial applications.

I am talking about the image processing vs. image analysis dilemma. The former produces images and the latter produces data. More on this here.

As image processing is a time consuming topic, the students may only get a little taste of image analysis (image segmentation and related topics about image content).

The result is that in order to make their skills applicable to scientific image analysis, they will need to take a more advanced course on the subject. Such a course would require (some combination of) calculus 1-3, linear algebra, probability. Even then, 3D images, especially their topology, are rarely addressed.

So, there may be a need for something even more elementary than Digital Image Processing Using MATLAB by Gonzalez, Woods, and Eddins discussed last time.

As I have mentioned before, I am thinking about writing a book on elementary computer vision and image analysis. Of course, I’ll follow what’s already in the wiki. It will take a while and in the process I research some of the better books related to the subject. I think Digital Image Processing Using MATLAB by Gonzalez, Woods, and Eddins is one of the best and closest to what I have in mind. Here is a short analysis.

The bottom line is that even though this may be the most elementary book on the subject, it’s still intermediate with serious prerequisites.