ECTtracker - Debugging elements

Debugging elements

Debugging elements of ECTtracker are additional windows, which can be shown by the user in order to adjust the program and hidden in order to save desktop space during proper program operation. Each of these windows shows some statistical information, which can be used in order to perform a more precise and correct program setting procedure of ECTtracker, which results in an improved level of image recognition. As a result, the user will have less stress and work with the program more continuously. The amount of false positives during recognition will also decrease.

The current version of ECTtracker supports three types of debugging windows, i.e. Stripes Diagram, Channel Intensity and SCO9, as well as the convenient event log. Let's look closer at these elements.

Stripes Diagram

An additional window of the program that shows the level of coincidence of the images in the window and samples from the Matrix of Samples in real-time (see fig. 24).

The 'Stripes Diagram' window (Fig. 24. The "Stripes Diagram" window)

The number of stripes always corresponds with the number of rows in the Matrix of Samples. When the tracking process is started in ECTtracker, lines are painted from the left to the right. The color changes from the dark blue when there's no coincidence to green in cases of a significant coincidence. The Stripes Diagram window is intended to provide a fast evaluation of the tracking accuracy and allows the user to select the proper value of wlim parameter, as well as to select an appropriate structure and correct other settings in accordance with the needs of each specific user and computer's performance.

The main condition of the proper recognition is as follows – one of the stripes in the program has to be higher than another at any given time period. On the image above, one can clearly see when the user's eye was open (green color of stripe 0) and when it was closed (green color of stripe 1). This image shows the proper recognition level setting.

Channels Intensity

Current versions of ECTtracker include a significantly improved color processing logic and methods of comparing the analyzed image with previously saved samples. In previous versions, the samples and the image were compared only in RGB color scheme, where each color is defined as a massive of three numbers (intensity levels for red, green and blue colors).

But the human eye uses different principles. For example, the bright yellow color and the dark yellow color will have pretty different RGB values. Thus, the program now involves the HSL color model, which is a massive of three number defining shade, saturation and lighting levels. Values of the colors that are not considered extremely different to the human eye will have little difference in the HLS color model.

Let's look at RGB and HSL values for two previously mentioned colors, i.e. bright yellow and dark yellow (see fig. 25).

Difference between the RGB and the HSL values (Fig. 25. Difference between the RGB and the HSL values
Left color: RGB – 255 242 0, HSL – 38 240 120.
Right color: RGB – 221 210 0, HSL – 38 240 104)

In the RGB massive, red and green values are changing, while in the HLS massive, only the lighting value is changing. HLS massive values can be compared more easily, thus providing more precise results during the tracking process in ECTtracker.

Furthermore, if one takes two consistent frames from a video stream and zooms in, it will be obvious that the color of some pixels may change drastically between such frames (see fig. 26 and 27). As a result, RGB values of such pixels are significantly changing too.

Difference between colors of zoomed images (Fig. 26a. Difference between colors of zoomed images) Difference between colors of zoomed images (Fig. 26b. Difference between colors of zoomed images)

In order to compare an image with the samples, ECTtracker performs an analysis of the rectangular zone of each structure point, defining average values for the following parameters:

Thus, each frame of the video stream is compared with samples using 9 various values for each point of the structure (i.e. a rectangular area with the center in the structure point).

Various environmental conditions (the intensity of the lighting, the position of the light source, the distance between the face of the user and the camera, the efficiency level of the selected recognition structure, the camera resolution, various image shifts caused by the user's head movement, etc.) can result in significant value differences for each of the channels, even for one state of the eye.

In order to check the intensity level of channels, the user of ECTtracker can open the Channels Intensity window (see fig. 27).

Channels Intensity window (Fig. 27. Channels Intensity window)

This window contains real-time intensity diagrams for each of the channels:

The general algorithm of the proper setting procedure is as follows:

Important information! In order to perform a proper setting procedure, the user has to select the most intensive channels for different eye states, and the less intensive for one state. The value of such channels for the image recognition process is incredibly high. Important channels must have priority levels in the range from 5 to 8, while less important channels should be in the range from 1 to 3. The maximum range of the intensity level for each channel is from 0 (the channel value is not involved in the calculation process) to 9 (the maximum priority level).

ECTtracker allows the user to set the level of priority independently for each of the channels. In order to do that, the user has to change parameter 15's value in the settings window of the program. Priority levels for all channels are indicated one by one without spaces. For example, if the user selects HSL channels as the most important ones, the value of the parameter may look like 111111888, 222222667 or something like that.

When ECTtracker is used for non-medical purposes, (e.g. to sort various image libraries or analyze current states of various analog sensors, etc.), RGB channels must have priority. It's more convenient to use HLS color profile recognition when it's required to analyze various liquid clouding or gas chamber conditions. It's also recommended to use the HLS color profile for medical purposes (at home or at a medical center) when ECTtracker is used for patients with limited motor activity. In general, the level of importance of each channel depends on the current environmental conditions, thus a level can be different for each particular situation.

Even though the Channels Intensity debugging window is pretty easy to use, the priority selection process can become a challenge for the user without required experience. That's why ECTtracker supports the automatic priority calculation feature. After filling the Matrix of Samples in manual or automatic modes, the user has to select the 'Settings – Set Channel Formula' menu items. The program will compare intensity changes for each channel in similar states. After that, the program will compare intensity changes for different states. Channels with the minimum intensity changes for one state and the maximum changes for different states will get the highest priority. Channels with the maximum level of intensity changes for one state will have the lowest priority. The final value will be automatically added to parameter 15 of the settings window of the program.

Due to the automatic formula calculation feature, even the most inexperienced user can select the required setting to provide an optimal recognition level.

PROSCO9 (Structure diagram)

The quality of recognition level can also be improved using the window that contains the structure diagram. This window contains structures for each of the channels, as well as the intensity changes for each of the structure points (see fig. 28).

The SCO9 window for various recognition structures: The first row – structures for RGB scheme, the second – structures for delta RGB, the third – structure for HSL scheme The SCO9 window for various recognition structures: The first row – structures for RGB scheme, the second – structures for delta RGB, the third – structure for HSL scheme
(Fig. 28. The SCO9 window for various recognition structures:
The first row – structures for RGB scheme, the second – structures for delta RGB, the third – structure for HSL scheme)

The brightness level of the recognition structure points indicates the variability of each color channel for such points. Even though all structures are universal, some points can be out of the area of the image with the most significant changes. This may lead to reduced recognition quality. For example, when the user selects the recognition structure for one eye, some points may be located in the nose bridge area. Color values for both states for such points (for open and closed eyes) will not have a significant difference.

Thus, the SCO9 window allows the user to identify recognition structure points, which can be located in areas of the image with the most insignificant changes. In this situation it's necessary to perform an additional setting procedure, i.e. relocate the target window of ECTtracker more precisely, and increase the scale of the image in ECTcamera if it's needed.

In the majority of cases, the SCO9 window is intended only for advanced users who can edit the recognition structure files, i.e. change the amount and position of points and the priority level of each point. However, this debugging window can also be useful even for average users, helping them to set the program up.

PROEvents Log

This additional window of the program shows the list of all important events that took place in ECTtracker: tracking start and stop, profile loading or saving, settings changing. Appearance of the events log is shown on figure 29.

The events log of the program (Fig. 29. The events log of the program)

The header of the event log indicates the amount of rows in the current log and the total size of the log in bytes. This information can be shown or hidden through the separate 'Settings' – 'Statistics' menu items. The event log menu also provides some other functions: