ECTtracker - Hints and tricks

Additional recommendations for effective work with ECTtracker

This section of the manual will help the user to reach the optimal recognition quality level and provide stable operation of ECTtracker in various conditions. It contains the most important information and tips about how to locate equipment (cameras and source of light) or program windows, and how to perform the initial setting procedure of ECTtracker and fill the Matrix of Samples. This section also contains some recommendations about the usage of contrast markers in situations where the user can't reach the proper level of recognition with traditional methods.

Positions of display, web-camera and program windows

The most important factor in order to provide correct ECTtracker operation is the positioning of the camera in such a way that the state of the eyes of the user can be clearly identified and tracked. In order to do this, the patient has to look at the camera (if possible) in such a way that the pupil can be captured almost completely. The exact center of the pupil has to be in the center of all the images from the Matrix of Samples with the user's eye open. Meeting this condition gives more than enough in order to provide guaranteed recognition quality of eye state in the majority of situations.

The proper position of all the windows of the EyeComTec program complex is also very important. The user works with ECTkeyboard's matrix of symbols, focusing attention on this window almost all the time. So it may be more expedient to put ECTkeyboard windows as close as possible to the position of the web-camera. This is vital in cases where the display has a high resolution, because the difference between the viewing position and the window position can be significant. Figure 37 shows the recommended layout for cases when the camera is located above the display. Figure 38 shows the recommended layout for cases when the camera is located under the display. Figure 39 shows the recommended layout for cases when the user is working on a laptop.

(Fig. 37. If the camera is located above the display, the widow of ECTkeyboard
has to be located in the upper part of the desktop)

(Fig. 38. If the camera is located below the display, the widow of ECTkeyboard
has to be located in the lower part of the desktop above the camera) When ECTkeyboard is launched on a laptop the window has to be located in the upper part of the desktop

(Fig. 39. When ECTkeyboard is launched on a laptop
the window has to be located in the upper part of the desktop)

Figure 40 shows windows of ECTtracker in two different situations: when the user is looking at the matrix of symbols located in the upper part of the desktop (the left screenshot); and in the lower part of the desktop (the right screenshot). In both cases the web-camera is fixed on the upper side of the display. It's obvious that in the second case the user is looking down and the upper eyelid is more closed. Tracking quality is also significantly decreased by lowered lashes. All these factors result in an unstable image recognition by the program, thus ECTtracker will recognize the user's eye as closed more often.

(Fig. 40. Direction of user's gaze is changing depending on the location of the matrix of symbols on desktop)

The smaller the range of motion of the pupil during work with the matrix of symbols, the more accurately ECTtracker will analyze the image. Pupil movement can be reduced in 2 ways (see fig. 41):

decreasing the size of the matrix of symbols of ECTkeyboard (left image). This way can be useful for displays with low resolution and size;
increasing the distance between the user and the display (right image). This way is recommended for displays with high resolution.

(Fig. 41. Decreasing the range of motion of the pupil during work with the matrix of symbols)

Proper calibration and initial setting of ECTtracker

The quality of image recognition of ECTtracker depends not only on external conditions (correct position of display, web-camera, light intensity and position of the light source), but also on correct filling of the Matrix of Samples by the user. There are several important rules for the proper calibration of ECTtracker, which are strongly recommended to use during the initial setting of the program:

1. The user has to click exactly on the center of the pupil during the new sample adding procedure. The point of click is considered as the exact center of the sample and used for tracking stabilization.
2. Gaze of the patient has to be directed to the camera whenever possible. The distance between the patient and the display, as well as the size of the matrix of symbols, has to be selected in such a way that the pupil makes minimum movements during the text typing process.
3. During the process of the automatic or manual calibration, the user needs to ensure that the eyes are not open too wide (wider than during normal work with the matrix of symbols). During continuous work, eye muscles become fatigued; eyelids start to close, resulting in the possibility of a lower quality of sample recognition.
4. It's recommended to create no more than 2 samples for the open eye of the user and no more than one sample for the closed eye (see fig. 42).

(Fig. 42. Proper filling of ECTtracker Matrix of Samples)

5. During work with the matrix of symbols, the user has to select a value for the wlim parameter that provides the correct recognition of only one row of samples at any selected moment of time. The schematic information about the quality of recognition can be seen in the Debug Lines window (see fig. 43).

(Fig. 43. Samples recognition quality)

6. The size of the sample should not significantly exceed the size of the recognition structure in use (see fig. 44). In fact, the zone of recognition is limited by the position of structure key points, and any increase of the sample size will not increase the size of the detection zone or improve its quality.

(Fig. 44. Size of the sample and real recognition zone, i.e. structure)

7. The main rule for the structure selection: the maximum possible points of the structure should be located above the area of the image with the most visually distinguishable changes. Key points of the structure, which are located above the static area of the image, will adversely affect the quality of recognition, as well as lead to an increase in the computational load on the system. E.g. if the patient has retained the mobility of only one of his eyes, the user has to select the structure for one eye and locate it above the changing area of the video source. It's worth noting that the exact center of the sample has to correspond with the exact center of the pupil.

Improving recognition quality by changing grab-form size

The main window of ECTtracker shows the image captured by the grab-form ('target window' located above the video source). This image contains the analyzed sample. The schematic drawing of the grab-form, the sample and the object to be tracked (eye) are shown on figure 45.

(Fig. 45. ECTtracker positioning scheme)

If the user of the EyeComTec complex retained partial mobility, head movements may occur during work with the program. ECTtracker follows such movements and moves the zone of tracking and the analyzing area. As the result, the analyzed zone can go beyond the borders of the grab-form in some cases. Figure 46 shows samples of correct recognition (upper row) and analyzed sample loss (lower row).

(Fig. 46. Correct and wrong positioning)

In cases where the analyzed zone partially leaves the grab-form, higher values of the wlim parameter may result in a total absence of sample recognition (simply because the program can't find any corresponding sample in the limits of grab-form). Lower values of the wlim parameter may result in recognition of similar zones inside the grab-form: thus, the recognition structure may sharply move above a wrong object, e.g. eyebrows. In this case, the program will identify user's eye as permanently closed.

In order to avoid described situations, it's recommended to increase the size of the 'target window' to keep all possible movements of the user's eye within the frame of that window. This approach will not allow ECTtracker to lose a tracked object. The grab-form width can be changed by using parameter 24 of the settings window of the program, and the height by using parameter 25 of the settings window of the program (see fig. 47).

(Fig. 47. Setting the size of the grab-form)

Recommendations about source of light positioning

Proper lighting is really important for normal operation of ECTtracker, because it allows the user to obtain more stable sample recognition. In the majority of cases, the source of light is located above the user during work, giving the most of the light, while the rest comes from the display (see fig. 48).

(Fig. 48. Working in normal lighting conditions of the room)

When a source of light is located above the user, all the colors are significantly smoother, the eye zone is shaded and contrast is decreased. The recognition quality also decreases, because colors of pixels are less different. At the same time, by working with the contrast image, software can recognize samples even if the user makes head movements. Figure 49 shows the ECTtracker window in a situation with an upper source of light (on the left screenshot) and another situation when the majority of the light comes from the display (on the right screenshot).

(Fig. 49. Location of source of light may significantly affect the contrast)

Thus, in order to provide higher contrast for the image and improve the quality of tracking, the user has to decrease or turn off light from all sources that are located above the user. The user can also try to increase the brightness of the display, creating an additional direct light (see fig. 50).

(Fig. 50. Using the display as a direct source of light)

In cases where the level of display brightness is not enough to create the proper level of lighting, the patient can use a small LED-light as an additional source of light. The best way to do so is to fix such an LED-light in the upper or lower part of the display, directing it in such a way as to avoid dazzling, but where it still creates additional lighting (see fig. 51).

(Fig. 51. Using an additional source of light)

Using contrast marker

Sometimes obtaining a proper level of recognition and tracking can become more difficult due to various physical damages of the patient's face. Such damages may include different injuries, burns, burnt eyelashes, postoperative states of the eye and many others.

In such situations, ECTtracker calibration can be performed by using a marker of a contrast color (bright red, green) or any other paint (e.g. brilliant green). A small dot is applied to the center of the eyelid of the patient, followed by clicking on that dot during calibration of the program and creation of the sample with the closed eye (see fig. 52).

(Fig. 52. Using a contrast marker)

Thus, ECTtracker gets an additional element with high contrast, which allows it to identify the sample with the closed eye of the user more clearly.

Proper scaling of the image

To improve the recognition quality during work with the EyeComTec program complex, the user may need to use additional scaling of the image (increasing or decreasing). It's recommended to perform such scaling only with ECTcamera functionality. The user can change the scale by using the Scale submenu items or corresponding hot keys (see fig. 53).

(Fig. 53. Changing image scale in ECTcamera)