ECTtracker - Debugging elements
Debugging elements of ECTtracker are additional windows, which can be shown by the user in order to adjust the program and hidden in order to save desktop space during proper program operation. Each of these windows shows some statistical information, which can be used in order to perform a more precise and correct program setting procedure of ECTtracker, which results in an improved level of image recognition. As a result, the user will have less stress and work with the program more continuously. The amount of false positives during recognition will also decrease.
The current version of ECTtracker supports three types of debugging windows, i.e. Stripes Diagram, Channel Intensity and SCO9, as well as the convenient event log. Let's look closer at these elements.
An additional window of the program that shows the level of coincidence of the images in the window and samples from the Matrix of Samples in real-time (see fig. 24).
The number of stripes always corresponds with the number of rows in the Matrix of Samples. When the tracking process is started in ECTtracker, lines are painted from the left to the right. The color changes from the dark blue when there's no coincidence to green in cases of a significant coincidence. The Stripes Diagram window is intended to provide a fast evaluation of the tracking accuracy and allows the user to select the proper value of wlim parameter, as well as to select an appropriate structure and correct other settings in accordance with the needs of each specific user and computer's performance.
The main condition of the proper recognition is as follows – one of the stripes in the program has to be higher than another at any given time period. On the image above, one can clearly see when the user's eye was open (green color of stripe 0) and when it was closed (green color of stripe 1). This image shows the proper recognition level setting.
Current versions of ECTtracker include a significantly improved color processing logic and methods of comparing the analyzed image with previously saved samples. In previous versions, the samples and the image were compared only in RGB color scheme, where each color is defined as a massive of three numbers (intensity levels for red, green and blue colors).
But the human eye uses different principles. For example, the bright yellow color and the dark yellow color will have pretty different RGB values. Thus, the program now involves the HSL color model, which is a massive of three number defining shade, saturation and lighting levels. Values of the colors that are not considered extremely different to the human eye will have little difference in the HLS color model.
Let's look at RGB and HSL values for two previously mentioned colors, i.e. bright yellow and dark yellow (see fig. 25).
In the RGB massive, red and green values are changing, while in the HLS massive, only the lighting value is changing. HLS massive values can be compared more easily, thus providing more precise results during the tracking process in ECTtracker.
Furthermore, if one takes two consistent frames from a video stream and zooms in, it will be obvious that the color of some pixels may change drastically between such frames (see fig. 26 and 27). As a result, RGB values of such pixels are significantly changing too.
In order to compare an image with the samples, ECTtracker performs an analysis of the rectangular zone of each structure point, defining average values for the following parameters:
- RGB (different values for each channel)
- Delta RGB (difference between the maximum and minimum values for each channel)
- HSL (separate values for each channel).
Thus, each frame of the video stream is compared with samples using 9 various values for each point of the structure (i.e. a rectangular area with the center in the structure point).
Various environmental conditions (the intensity of the lighting, the position of the light source, the distance between the face of the user and the camera, the efficiency level of the selected recognition structure, the camera resolution, various image shifts caused by the user's head movement, etc.) can result in significant value differences for each of the channels, even for one state of the eye.
In order to check the intensity level of channels, the user of ECTtracker can open the Channels Intensity window (see fig. 27).
This window contains real-time intensity diagrams for each of the channels:
- R – red color
- G – green color
- B – blue color
- dR – red color (the difference between the maximum and minimum values)
- dG – green color (the difference between the maximum and minimum values)
- dB – blue color (the difference between the maximum and minimum values)
- H – hue
- S – saturation
- L – lighting
The general algorithm of the proper setting procedure is as follows:
- The user has to add several samples for the open eye into the Matrix of Samples in order to check changes in the intensity level of the channels. Channels with the highest intensity levels for one state must have the lowest priority level during calculations.
- The user has to erase all samples from the Matrix and add new samples for the closed eye. The user needs to check the intensity level of the channels. As in the previous step, the channels with the highest intensity levels must have the lowest priority level.
- After doing that, the user has to erase the Matrix of Samples and add one sample with the open eye, and one sample with the closed eye. The channels with the maximum intensity levels for each of the states must have the highest priority level.
Important information! In order to perform a proper setting procedure, the user has to select the most intensive channels for different eye states, and the less intensive for one state. The value of such channels for the image recognition process is incredibly high. Important channels must have priority levels in the range from 5 to 8, while less important channels should be in the range from 1 to 3. The maximum range of the intensity level for each channel is from 0 (the channel value is not involved in the calculation process) to 9 (the maximum priority level).
ECTtracker allows the user to set the level of priority independently for each of the channels. In order to do that, the user has to change parameter 15's value in the settings window of the program. Priority levels for all channels are indicated one by one without spaces. For example, if the user selects HSL channels as the most important ones, the value of the parameter may look like 111111888, 222222667 or something like that.
When ECTtracker is used for non-medical purposes, (e.g. to sort various image libraries or analyze current states of various analog sensors, etc.), RGB channels must have priority. It's more convenient to use HLS color profile recognition when it's required to analyze various liquid clouding or gas chamber conditions. It's also recommended to use the HLS color profile for medical purposes (at home or at a medical center) when ECTtracker is used for patients with limited motor activity. In general, the level of importance of each channel depends on the current environmental conditions, thus a level can be different for each particular situation.
Even though the Channels Intensity debugging window is pretty easy to use, the priority selection process can become a challenge for the user without required experience. That's why ECTtracker supports the automatic priority calculation feature. After filling the Matrix of Samples in manual or automatic modes, the user has to select the 'Settings – Set Channel Formula' menu items. The program will compare intensity changes for each channel in similar states. After that, the program will compare intensity changes for different states. Channels with the minimum intensity changes for one state and the maximum changes for different states will get the highest priority. Channels with the maximum level of intensity changes for one state will have the lowest priority. The final value will be automatically added to parameter 15 of the settings window of the program.
Due to the automatic formula calculation feature, even the most inexperienced user can select the required setting to provide an optimal recognition level.
PROSCO9 (Structure diagram)
The quality of recognition level can also be improved using the window that contains the structure diagram. This window contains structures for each of the channels, as well as the intensity changes for each of the structure points (see fig. 28).
The brightness level of the recognition structure points indicates the variability of each color channel for such points. Even though all structures are universal, some points can be out of the area of the image with the most significant changes. This may lead to reduced recognition quality. For example, when the user selects the recognition structure for one eye, some points may be located in the nose bridge area. Color values for both states for such points (for open and closed eyes) will not have a significant difference.
Thus, the SCO9 window allows the user to identify recognition structure points, which can be located in areas of the image with the most insignificant changes. In this situation it's necessary to perform an additional setting procedure, i.e. relocate the target window of ECTtracker more precisely, and increase the scale of the image in ECTcamera if it's needed.
In the majority of cases, the SCO9 window is intended only for advanced users who can edit the recognition structure files, i.e. change the amount and position of points and the priority level of each point. However, this debugging window can also be useful even for average users, helping them to set the program up.
This additional window of the program shows the list of all important events that took place in ECTtracker: tracking start and stop, profile loading or saving, settings changing. Appearance of the events log is shown on figure 29.
The header of the event log indicates the amount of rows in the current log and the total size of the log in bytes. This information can be shown or hidden through the separate 'Settings' – 'Statistics' menu items. The event log menu also provides some other functions:
- 'Clear' – delete all records about program work from the log
- 'Save' – save log as text file
- 'Close' – close the log window.