The site, Computer Vision Primer, has been growing and has reached 379 pages with 1,018 illustrations. In particular, the transcription of the Vector Calculus course that I taught 2009/10 has recently started. Two lecture sets are finished (about 20% of the total) and the third is on the way.

The NSF summer REU program has started. These are the two projects that I will supervise:

• The topology of data by Joseph Snyder; and
• 3D image analysis by James Molchanoff.

Q: “Interested in assessing whether your software could be used to determine the quality of a solder joint… Joint is composed of an aluminum tube and a copper pipe… The area of the copper pipe that overlaps or contacts the aluminum tube is the area that needs assessment. Would like to determine what percent of this area has a solder coating. The solder is a zinc78%/alum22% alloy… the copper pipe section could be flattened to a greater degree if this to would help with accuracy.”

As it has been suggested, it is a good idea to outline what is to be measured manually so that we can see if automatic analysis produces reasonable results:

In the screenshot the two numbers are at the top.

The area inside the red contour (i.e., inside your red line) is 140,582 pixels.
The area inside the green contour (i.e., gray region) is 106,787 pixels.

So the proportion of gray inside the red line is
106,787 / 140,582 = 76%.

By moving the sliders one can make the contours to fit better and improve the accuracy.

More here…

More precisely, these are the areas where the change of the gray – for light to dark or dark to light – is the fastest. Then one needs a threshold so that all pixels where this change is higher that this number are considered “edges”:

Mathematically, we deal with

 the rate of change of the gray level

             = the gradient of the gray scale function.

(In fact, one only needs the norm of the gradient.) Computation of the derivative however in the digital (discrete) context is a challenge as it is severely affected by noise. Consider the image of coins and its version with noise added.

If now edge detection is run, the results are unsatisfactory – too many irrelevant contours. 

Of course it may be possible to filter out the smaller contours. In this particular case it’s impossible because they are parts of large ones. In fact they form large fractal-like structures. This is the reason why edge detection may have to be preceded by smoothing of the image.

Read more…

Under Review summary (in Output tab) Pixcavator shows the data about the objects found in the image. Pixcavator displays the total area of dark and the total area of light objects – as percentages of the total size of the image (second row).

Under certain circumstances though, the contours of the same kind may be “nested” and, as a results, these percentages may be wrong or even above 100%.

Example below (measuring grass coverage): the dark shows the 151% coverage.

The number is certainly meaningless (there will be a warning about that in the next release).

Why is it above 100%? Because the area is covered several times by these objects. If you click “Color objects”, you’ll see one large object with red contour and many others inside of it.

What happens is easier to see in this simpler image:

The results of image analysis may considered “good” here, but only in the sense that we have captured some 3D information. In general, we restrict our attention to image with mostly 2d information (see Images appropriate for analysis).

What exactly happens here? The way Pixcavator’s sliders operate is this: the contour is allowed to grow until its size (or contrast) is over the bound set by the corresponding slider. Practically, this means that each potential contour C is compared to a contour C’ corresponding to the previous gray level. Then, if C passes but C’ does not then C is plotted.

For more, see Nested boundaries.

Now, the area covered should be just a step towards what we are really interested in – the height of the vegetation (or volume, even better).

Let’s consider how one can compute the height of vegetation from a digital image. The idea is very simple:

 the average height = the area / the width.

Consider now what we see in the image.

Views from a side (vegetation in green) and from above:

Assumptions:

1. The board is a square and its dimensions are known.
2. The board is vertical (otherwise it’s impossible to know where the bottom is).
3. The bottom of the board is horizontal on the horizontal (along the board) ground.
4. The field of view of the camera includes the edge of the vegetation and the top of the board.

Then, the average height computed as below is independent from:

• the deviation of the angle of the camera from the horizontal,
• the distance from the camera to the board,
• the height of the position of the camera above the ground.

The measurements (the image in black, the bottom of the board in red):

These come from image analysis:

 A = the area of the board visible above the vegetation (sq pixel),
 W = the width of the board (pixel).

This is known:

 S = the length of the side of the board (in).

Then average height of the vegetation above the ground (in) is:

  H = S * (1 - A / W2).

Computations here.

This would be hard to accomplish with images similar to these. To capture the vegetation effectively, one has to separate it from the background. Then, ideally, the latter would have to either uniformly lighter or uniformly darker than the former (see Gray scale images). The light/dark squares make the task very challenging.

Instead, one can digitally isolate the squares within each image, so that the area covered by vegetation can be estimated from a set of sub-images (i.e., individual squares) with uniform background colors:

In the screenshot, the colored areas are the complement of the vegetation. Their total area is 64.95%, so the vegetation takes the rest, 35.05%.

Blue gives a good separation of the background. 

More…

• Loading image of sizes >2000×2000 causes the software to stall (fixed, but still impractical for processing).
• Changing the color channels after processing causes messed up data in the Output tab.
• Summary in the Output tab isn’t updated when manually select/deselect objects.
• Some image processing tools in the Tools tab don’t work properly.

The full title of the report is “Image-to-image search with Pixcavator (PxSearch): a case study”. It was written by Dr. Ash Pahwa and myself and is presented here with minor modifications.

The first version of PxSearch was created in 2007. Using that version, initially the search results with the collection had 4-5 good hits (i.e., the transformed version of the original) at the top and then some bad hits. Some of the good matches weren’t even visible. After the upgrades, the results became 10 out of 10 or close. This improvement made this, more extensive, study possible. The results are OK, even though the collections are still very small. The company eventually went with another vendor, it’s still an interesting document to browse through.

Since 2009, there has been no work going on but, hopefully, this project will be one of the summer projects for the REU site.

Incidentally, I don’t like the term “reverse image search” popularized by TinEye. If the image search that we are used to at Google etc is “direct image search” (text-to-image) then the “reverse image search” is supposed to search for text based on images. Not only this isn’t what we are talking about, but also the problem hasn’t been even remotely solved (see this pathetic list: Visual image search engines). This is the reason I prefer “image-to-image search” to describe this application.

I will be supervising 2 students in 1-2 of these areas:

• image analysis, and/or
• image-to-image search, and/or
• topological data analysis.

Temporary page for the projects: Computational science training: 2010 projects.

Two images of an answer sheet below (the originals were 2.5 MB each with a resolution of 2504×3229 pixels).

Let’s see if we able to capture the bubbles.

Not hard at all. The first image shows how empty bubbles are captured, in the second – only the marked ones, but not crossed. (Here is a company that is doing something very similar [1]. Cost $1K.)

Examples with similar analysis:

• Image analysis for a hand-held diagnostic device,
• Microarray analysis.

Even though the origins of these images are very different, the images themselves are similar to these and the approach to analysis might be identical. There are many examples like this…

The first on the list also provides a prototype program that displays instead of the usual Pixcavator’s output table – a table of 0s and 1s for unmarked marked bubbles respectively.

Other image analysis examples