Vision-based analysis

Design and realisation

Within the project, the below schematic shows the basic design of the application with its inputs and output. If the well-known home pregnancy test is envisioned, the "JPG / PNG" block is the camera, looking at the tester's stripes, providing a stream of images to the application. The "GAL" block is a simple text file which can be read according the "Genepix Array List" layout. In this file the data is stored that tells the application which substances are printed on the test strip and where the lines physically are. The last input block for the application are the application's settings. The settings (pixel to micrometer ratio, input / output folders, etcetera) are stored in a text file as well. When the application starts, the settings file is read, the GAL file is read and then the application waits for the first camera image to be supplied. The application analysis the images (more details on this process below) and outputs the analysis results to a comma-separated-values (CSV) format file. The output is depicted by the bottom "CSV" block.

block diagram vision based diagnostics

From computer vision point of view, the application uses classic image processing to analyse the images. For classic image processing there is a 5-steps sequence to classify the image:

  1. Image acquisition
  2. Image enhancement
  3. Image segmentation
  4. Feature extraction
  5. Classification

The target of executing the above sequence is to be able to calculate where the vertical test lines are printed on the strip. To make this possible, there is 1 line on the strip that will always appear: the control line. Using the 5-steps sequence above, the control line can be found.

The data in the GAL file states where the control line is on the strip, as well as the test lines. The below images are taken by the camera and they show (from left to right) the strip and the lateral flow of the dispensed fluid on the strip. The control line that appears is the vertical line on the left in the image. The vertical line in the middle is test line number 1 and on the right is test line number 2. In reality, there are approximately 500 images and not only the 8 images shown below. Showing the complete set of images would result in a large page to be loaded and for the general idea behind vision-based diagnosis it would not contribute.

Example camera images of the fluid flowing over the strip and the appearance of the control line (left of image) and the test lines (middle and right of image)
0 0 0 0 0 0 0 0
t = 0 s t = 0 + N s t = 0 + 2N s t = 0 + 3N s t = 0 + 4N s t = 0 + 5N s t = 0 + 6N s t = 0 + 7N s

Using the 5-steps classic image processing sequence, the control line is found in the image sequence first, see the image on the left below. When the control line is found, the data from the GAL file and the settings file are used to calculate where the test lines could appear and give the regions in the image that need to be analysed by the application. To obtain a time reference moment to start the analysis of the images, the passing of the liquid front must be determined. With the location of the control line known, an arbitrary region (see the little red square in the image on the right below) outside the control line is introduced. This region is analysed for all the available images up to this moment. The image in the middle below shows the sum of all the pixels in the liquid front analysis region (the little red square). On the vertical axis the sum of all pixels is plotted for each available image up to the moment the control line has been found. The liquid front is passing in the analysis region where the delta sum of pixels is largest.

000
Image where the control line (red rectangle) is found Sum of pixels graph of the analysis region for liquid front detection Image where the passing of the liquid front is calculated (red square)

At this point the control line has been found and the location of the test lines in the image(s) calculated. Further, the liquid front is detected which provides a point in time to start the analysis. The below 2 images illustrate where the software has calculated test line 1 (middle vertical line) and test line 2 (vertical line on the right) have to be. A rectangular box is drawn along the edge of the rectangle and it is obvious that the test lines fall within the box.

00
Test line 1 in the middle is calculated to be located within the rectangle Test line 2 on the right is calculated to be within the rectangle

The analysis of the images pixels within the test lines rectangles now starts from the reference point (i.e. in this example the image where the liquid front passes). The analysis in for this project is the minimum, maximum, average and the sum of pixels within the analysis regions, with- and without compensation for the change in background pixels (i.e. the pixels surrounding the lines printed on the strip). All analysis data is exported to a CSV file and can be easily imported into spreadsheet software. The graph image below shows the plots for the not-background-compensated averages of test lines 1 and 2.

0
Plots of the average pixel value for test lines 1 and 2, not compensated for background pixel changes

In terms of hardware used and costs in this specific project:

  • A microscope - costs can vary from €500 to €10000
  • A Raspberry Pi - costs around €50
  • A camera - costs can vary from €25 to €250

This brings the total material costs to approximately €11000 (maximum).

In terms of hours spent on this specific project: this project was a continued development project and a "proof of concept" was already available. This shortened the development time significantly. A total amount of slightly over 250 hours have been needed to develop the application on the Raspberry Pi. For similar projects that are started from scratch, it is estimated that roughly 500 - 800 hours are required from beginning to end.

Conclusion

To obtain vision-based analysis equipment, specifically for your own needs, against relatively low costs is possible. One of the pro's is that if it is not certain that vision-based analysis is a suitable tool for your own purposes, it is not needed to acquire expensive equipment to test it. One of the more difficult parts in vision-based analysis is finding the right vision algorithm(s) using the 5-steps sequence of classic image processing. There are several approaches to find the correct algorithm, but time needs to be invested in all cases. This might be a disadvantage. On the other hand: when successful, a fully customised software application is yours to use.


Mc'bedded
Idastraat 5
5431 GJ Cuijk
Dit e-mailadres wordt beveiligd tegen spambots. JavaScript dient ingeschakeld te zijn om het te bekijken.

Telephone +31 6 20 40 61 02

Chamber of Commerce Arnhem 72696311

Bank account NL46 KNAB 0258 2265 60