These are three research papers that use Pixcavator I discovered (I wish someone simply let me know somehow…):

• The effect of the hydrocarbon–water interface structure on the behavior of an emulsion stabilized with dodecanephosphonic acid by Valeria Verdinelli, Paula V. Messina, Eduardo N. Schulzb, Daniel Salinas, Bruno Vuano, Pablo C. Schulz, published in Colloids and Surfaces A: Physicochemical and Engineering Aspects.
• Visualizing the mobility and distribution of chlorophyll proteins in higher plant thylakoid membranes: effects of photoinhibition and protein phosphorylation by Tomasz K. Goral, Matthew P. Johnson, Anthony P.R. Brain, Helmut Kirchhoff, Alexander V. Ruban and Conrad W. Mullineaux, published in The Plant Journal.
• The behaviour of double oxide film defects in liquid Al alloys under atmospheric and reduced pressures by R. Raiszadeh, and W.D. Griffiths, published in Journal of Alloys and Compounds.

It took me the whole summer but the lecture notes for Vector calculus are now online. They have been fully transcribed, edited, illustration added. There is certainly a lot of work left to make them more presentable, readable, etc. That will continue along with linking, internally and to the rest of the site. As a part of this work I’ll try to add something that I feel is missing – the discrete angle (image analysis, data analysis, etc).

The total number of articles has reached 446, with 1208 illustrations. There is some duplicate content but not a lot. The point is that one can take this material, rearrange it in a number of ways, and create various courses depending on the goals, or the audience, etc.

Still to come are the two courses I am teaching this fall: Calculus 1 and Differential Geometry. They will be transcribed some time next winter. More courses will continue to appear as a part of what I call my “fantasy math” project.

Some users expressed the need for longer evaluation time, so I extended the trial period for Pixcavator from 10 to 30 days. Also, for the users who are having trouble with installation and registration, for a number of reasons (firewall etc), please try the Student Edition. It requires no installation and can be freely copied.

The site, Computer Vision Primer, has been growing and has reached 379 pages with 1,018 illustrations. In particular, the transcription of the Vector Calculus course that I taught 2009/10 has recently started. Two lecture sets are finished (about 20% of the total) and the third is on the way.

The NSF summer REU program has started. These are the two projects that I will supervise:

• The topology of data by Joseph Snyder; and
• 3D image analysis by James Molchanoff.

In the last post I discussed some issues you encounter when you want to evaluate vegetation coverage based on image analysis.

Now, the area covered should be just a step towards what we are really interested in – the height of the vegetation (or volume, even better).

Let’s consider how one can compute the height of vegetation from a digital image. The idea is very simple:

 the average height = the area / the width.

Consider now what we see in the image.

Views from a side (vegetation in green) and from above:

Assumptions:

1. The board is a square and its dimensions are known.
2. The board is vertical (otherwise it’s impossible to know where the bottom is).
3. The bottom of the board is horizontal on the horizontal (along the board) ground.
4. The field of view of the camera includes the edge of the vegetation and the top of the board.

Then, the average height computed as below is independent from:

• the deviation of the angle of the camera from the horizontal,
• the distance from the camera to the board,
• the height of the position of the camera above the ground.

The measurements (the image in black, the bottom of the board in red):

These come from image analysis:

 A = the area of the board visible above the vegetation (sq pixel),
 W = the width of the board (pixel).

This is known:

 S = the length of the side of the board (in).

Then average height of the vegetation above the ground (in) is:

  H = S * (1 - A / W2).

Computations here.

This study was conducted in 2009 for a company that is “working in the online social media sector and are looking for an accurate image analysis solution that allows us to compare a reference photo to a large dataset of photos to determine if the reference photo is duplicated in the larger dataset.”

The full title of the report is “Image-to-image search with Pixcavator (PxSearch): a case study”. It was written by Dr. Ash Pahwa and myself and is presented here with minor modifications.

The first version of PxSearch was created in 2007. Using that version, initially the search results with the collection had 4-5 good hits (i.e., the transformed version of the original) at the top and then some bad hits. Some of the good matches weren’t even visible. After the upgrades, the results became 10 out of 10 or close. This improvement made this, more extensive, study possible. The results are OK, even though the collections are still very small. The company eventually went with another vendor, it’s still an interesting document to browse through.

Since 2009, there has been no work going on but, hopefully, this project will be one of the summer projects for the REU site.

Incidentally, I don’t like the term “reverse image search” popularized by TinEye. If the image search that we are used to at Google etc is “direct image search” (text-to-image) then the “reverse image search” is supposed to search for text based on images. Not only this isn’t what we are talking about, but also the problem hasn’t been even remotely solved (see this pathetic list: Visual image search engines). This is the reason I prefer “image-to-image search” to describe this application.

RGB stands for Red, Green, and Blue. These are the “channels” in a color image. Each pixel has 3 numbers between 0 and 255 assigned to it.

• (255,0,0) red,
• (0,255,0) blue,
• (0,0,255) green,
• (255,255,0) yellow,
• (255,0,255) magenta,
• (0,255,255) cyan,
• (g,g,g) gray, for any g,
• (0,0,0) black,
• (255,255,255) white

Every color image has three color channels – red, green and blue – and the image features you are after may be more pronounced with respect to one of them.

The channel-by-channel analysis allows one to consider each channel of the color image as a separate gray scale image and analyze them as needed. In Pixcavator just click a button in the Analysis tab for the channel you want.

In the example below, the circles are of pure red, green, and blue. As a result, the red circle which is (255,0,0) becomes 255 in the red channel. But 255 is equivalent to white in this gray scale image. So the red circle disappears in the red channel. Similarly, the green circle will disappear if you choose the green channel, etc.

This option is important for some applications such as microscopy. Different features are sometimes better revealed in different channels. Below is the original image with two clear, to the human eye, features: red walls and green “cells”.

Read about analysis of this image here: http://inperc.com/wiki/index.php?title=RGB_channels.

Most of the recent content has come from two main sources. First, I have been adding, as before, examples of image analysis from the users of Pixcavator. The second is the course I’ve been teaching since last fall: Introductory algebraic topology. I plan to add more content from the courses that I teach: Vector calculus (this summer), Introductory differential geometry (next fall), and maybe also something of lower level like Calc1 (next winter).

What is the goal? I would like the site to cover a big chunk of the math curriculum, interlinked within and with the computer vision / image analysis topics (see The Mathematics of Computer Vision). Even though the format is identical to Wikipedia the presentation is very different. This is a textbook: more details, more examples, exercises, etc. It can still be used for reference.

The content comes directly from my lectures. I use Tablet PC with Windows Journal. I started doing this last fall and I really love the results: bright, colorful slides, but with the spontaneity and flexibility of a chalkboard. Later I transcribe the lectures into text, put it on the site, and simply copy the illustrations. (Plus, I don’t have to deal with chalk on my shoes, pants, and lungs!) I think this approach has huge advantages over the common practice of simply posting video lectures online: searchability, cross-linking, speed of download, the person can read and work at his own pace, etc.

Q: “Immunohistochemistry was performed on my lung biopsies and now I have to analyse my staining. I would like to get a value for the colour intensity of my staining. … In the output of Pixcavator a mean value for grayness is given, but I don’t think all my brown staining is measured. Even if I adjust the threshold for size, not all objects are captured with a red or green contour.  ”

1. The difference in gray isn’t more pronounced because what is displayed is the average not the weighted average. So, one large dark object is outweighed by smaller, lighter ones. If you hover over objects (or look at the table) one at a time, you see a more noticeable difference. You can also save the data as a spreadsheet and compute the weighted averages. Certainly, if the difference is still too small, contrast enhancement would help.

2. It’s better to choose the green channel (or blue) than red. The features are much more distinct. If you click on “Display channel”, you’ll see the difference.

3. You are interested in dark objects only, red contours. So, the light ones only skew the averages. Click “Unmark light”.

4. The “Size” slider does not seem to reveal the features quite well. I used the simple thresholding instead, i.e., the “Intensity, dark” slider.

A screenshots is below. (This example may be somewhat similar: Measuring staining in the liver.)

More analysis here… 

Other examples of image analysis

This fall I have been teaching Topology I (Topology II next spring). I decided to emphasize algebraic topology and in fact started with it rather than point set topology which alone can take two semesters.

Outline
This is an introductory, two semester course on algebraic topology and its applications. It is intended for advanced undergraduate and beginning graduate students.

Part 1. Introduction to algebraic topology
Starts with topological issues in digital image analysis, informal introduction of homology

Part 2. Homology theory
Cubical complexes, their homology, and maps

Part 3. Overview of point-set topology
Minimized to the extreme (still could have cut even more)

Part 4. Homology groups
A more formal, group theory based, exposition

Part 5. Homology and uncertainty
Applications in computer vision, image analysis and data analysis

Part 6. Beyond homology
The fundamental group and cohomology

Also, I ran across this white paper from Hewlett-Packard: Algebraic topology for computer vision. Good review and an honest attempt to convince the practitioners to that this is something that they might need to know (good luck with that!).

Q: we need to “count cell/dna etc from still microscopic image… Here we stained DNA …S-phase are in red and G-phase is in Blue. We want to count how many red are there and how many blue are there.”

I used Pixcavator and did a bit of experimenting with the first image. The red cells are easy to capture – in the red channel, 71 total.

Now the red cells are so bright that in the blue channel they are also present. So, here you see both red and blue and have to subtract:

 299 - 71 = 228 of the blue ones.

 Using subtraction here isn’t a perfect solution clearly. To separate red and blue one might try to filter the spots based on the size or intensity.

Image processing can help too – I simply removed the red using Photoshop Elements.

The total was 204. 

One day you’ll be able to use the true color analysis.

See examples of image analysis.

Q. “I was wondering if it is possible to make it as an online service where people could log on to the server and upload their image and obtain the report or csv file.”

A. I have been thinking about a web app for Pixcavator for a while. The idea is certainly very attractive but there are some drawbacks. First, the computations that Pixcavator does are quite CPU and memory intensive. The testing that has been done shows that online computation (short of some kind of supercomputer) wouldn’t be any faster than if it is done on a modern PC. Second, uploading a 1000×1000 bmp file can take as long as the processing itself. Once it becomes common for people to keep their images (and I mean business/research images, not family albums) “in the cloud”, a web application will make more sense. Getting there may take some time but there is no doubt in my mind that this is the future.

I added a list of recent customers of ours here.

I also created a page for the course I am teaching: Introductory algebraic topology. The outline is there already and some of the articles have been written. The second one is Homology as a equivalence relation. Consider the question, What is a tunnel? It’s not as simple as it seems. It takes some work to find a good answer, the main part of which is: A tunnel is an equivalence class of closed surves.

Over the following months (it’s a two semester course) I’ll keep adding material as the course progresses.

I am also teaching Advanced calculus and some of this stuff will also find its way into the wiki.

A new research paper that uses Pixcavator:

Electron mirroring: control of electron transport and understanding of physical processes from SEM images [1] by Marziale Milani, Davide Bigoni, Claudio Savoia (in Proceedings of ITP2009 Interdisciplinary Transport Phenomena VI: Fluid, Thermal, Biological, Materials and Space Sciences October 4-9, 2009, Volterra, Italy.)

From the paper:

“Mirror effects occur when a primary electron beam scans an insulating sample and the charges on its surface accumulate to a high density. When the energy of the electrical field becomes higher than the primary beam one it prevents the charged particles from reaching the sample surface, reflecting them somewhere else in the vacuum chamber whose walls act as a mirror. The inner part of the specimen chamber can be therefore imaged.”

“Video signals were then analyzed as a function of detector bias voltage, i.e. variations in the mirror images due to variations of these parameters have been compared by acquiring (by “Pixcavator” software) the grey levels in selected portion of images and the geometrical shapes of some easily recognizable element in the mirror images.”

Mirror images of polyester samples:

For more papers follow this link.

Taking two images of the same scene from two slightly different locations and then matching the items in them pairwise gives you the distances to these items.

The image matching part is crucial and less trivial. For example on the right the corner of the cube may be a good pixel to choose. The rest of image is mainly featureless. The geometry is trivial, below.
Suppose we established a match between a pixel P in image I and pixel Q in image J. Let’s find the distance to what the pixels depict.

Two images with a red pixel in each image representing the same thing:

We need only to consider only the horizontal line through P,Q to find the distance to the object with the red dot. View from above; the eyes are the foci of the cameras. Black lines are the images:

“Triangulation”: the object lies on the line from the focus of the camera and its mark on the image. Here the big red dot is the actual location of the object:

The geometry:

D is what we are looking for.

The pink, and the blue, triangles are similar. So, after a bit of algebra we have:

where d=x+y is simply the distance the pixel moves as we switch from one image to the other, called the disparity.

More details in the article.

The site is being restructured right now. The goal is to integrate the Computer Vision wiki (mostly content) with the rest of the site (mostly product information). Virtually all of this content has already moved to the wiki. The only exception will be this blog. The menu at the top of this page will be removed and the pages it links to will be disabled, eventually.

This is the new welcome page of the site.

Please update your bookmarks, links, etc.