ECTtracker - First start. Getting Started

ECTtracker Setup Guide

In order to provide the proper image analysis level of the program, it's necessary to calibrate it and fill the Matrix of Samples. It's also necessary to change several parameters depending on the physical characteristics of the patient and performance of the computer. This is especially important during the first start of the application.

The initial setting procedure of ECTtracker and further work with the program include the following stages:

Start of the program and selection of the recognition structure

To start working with the program, the user needs to launch the eye-tracking application by opening the ECTtracker.exe file. In case the user needs to use a previously saved settings profile, then the user has to select the "Settings" – "Load Settings" menu item after starting the program, then choose the desired file in the dialog box of the program (see fig. 2). All the changes are applied after selection of the new configuration file.

Loading configuration file for ECTtracker (Fig. 2. Loading configuration file for ECTtracker)

After doing so (only if it’s necessary, because the type of the recognition structure is already predefined in the selected configuration file) the user has to select the recognition structure – for one eye only or for both eyes. Selection can be done through the "Samples" - "Choose Structure" menu items. Recognition structures differ in positions of key points, which are used during comparison of saved samples and images in the program window. Different structures are shown on figure 3. It is worth noting that different structures allow identifying the user’s selection not only by opening or closing eyes, but also using other actions. Such actions include eyebrow or lip movements or finger removing from the source image with bringing it back in a while; in other words – they include any clearly identifiable states of one object.

Different recognition structures (Fig. 3. Different recognition structures)

The main principle for the structure selection process is to put a maximum amount of key points above any fragment of the image with a maximum amount of visually distinguishable changes. Key points of the structure, which are located above a static area of the frame, increase the total load on the system and negatively affect the quality of tracking.

Selecting the image source and setting up the grab-form

Before tracking, the user has to predefine the image source in ECTtracker. If the program is used with a different application that acquires video from a web-camera or any other device, the user has to start that application first. If the program works with already recorded video, it's necessary to open the media player window first. This user's manual describes the joint work of ECTtracker and ECTcamera software. ECTcamera is an application intended for use with the web-camera of a laptop or any removable camera which is connected to a computer.

In order to provide the correct display of the video fragment from an ECTcamera window, it's required to place the target window (grab-form) above such window. During the start of ECTtracker, the user can see a semi-transparent rectangular area with a red border, which is intended for video capturing and transmitting it into the program. To set up video capturing, it's necessary to move the target window above the video window (see fig. 4). It’s worth noting that key points of structure in the video window of ECTtracker have to cover the maximum possible changeable area of the image.

Capturing video fragment with the target window (Fig. 4. Capturing video fragment with the target window)

If one needs to change the size of the grab form immediately, it can be done by right-clicking on it. The user has to select one of the actions in the menu that appears (see fig. 5).

The grab form size changing menu (Fig. 5. The grab form size changing menu)

The user can also left-click on the grab form in order to make it active, and use hot keys to change its size on the fly:

Each of the mentioned combinations will cause an immediate update of the grab form and the main window of the program.

After selection of the structure and correct positioning of the grab-form above the captured image, it's necessary to create or load the matrix of working samples for program operation and correct image recognition. If the user created such a Matrix of Samples before, it can be loaded:

If there was no Matrix of Samples created, the user has to create it in manual mode or by using the automatic calibration feature.

Automatic calibration

Automatic calibration involves a script, which shows auxiliary images with various eye states to the user (both eyes open, only right eye open, only left eye open, both eyes closed). The user has to follow such images and close one or both eyes in accordance with the image on the screen. Different examples of such images are shown on figure 6.

Auxiliary images for automatic calibration (Fig. 6. Auxiliary images for automatic calibration)

Depending on the selected script, images are going to be shown on different parts of the screen. If one divides the screen into 9 different parts, each part will correspond with one auxiliary image (see fig. 7). The program is going to create a different sample in the Matrix for each position and eye state. This method is necessary to provide proper working of the program regardless of the part of screen the user is looking at.

Various positions of auxiliary icons (Fig. 7. Various positions of auxiliary icons)

Parameter 48 of the ECTtracker settings window allows the user to select a file with the auto-calibration script. Functionality of the script can be determined by the name of the file, because it contains a special symbol encoding. A number before the S letter shows the quantity of eye states, which are shown on the auto-calibration icon. A number before the P letter shows the quantity of different positions of the icon on the screen. A number before the M letter shows the quantity of rows in the Matrix of Samples.

1st example: if the Eye_2S5P2M.dat script is selected, the user will see icons with open and closed eyes. The icon is going to be shown in 5 different screen positions (in the center, in the upper left corner, in the upper right corner, in the lower left corner and in the lower right corner). The calibration process is going to fill 2 rows of the Matrix of Samples.

2nd example: if the Eye_4S5P4M.dat script is selected, the user will see icons with four possible states of eyes (both eyes open, right eye open, left eye open, both eyes closed). The icon is going to be shown in 5 different screen positions. The calibration process is going to fill 4 rows of the Matrix of Samples.

Thus, it's possible to select the type of auto-calibration that fits a specific user. E.g. if the patient can move only one eye, it can be useful to load a script that makes two types of samples - open eye and closed eye (in such case, the user has to locate the grab-form only above the area of the working eye and use the recognition structure for one eye). If the patient can blink with both eyes, then it can be useful to load a script for 4 states of eyes and the recognition structure for both eyes. The Eye_2S1P2M.dat script is used by default in ECTtracker.

To start the calibration process, it's necessary to select "Samples" – "Automatic calibration" menu items or press the F11 button. The user will see images with a countdown timer changing one by one. There is a sound at the end of each time period, after which the program saves the sample and goes to the next image. The calibration procedure allows the user to fill the Matrix of Samples in the fully automatic mode.

Some results of the auto-calibration feature may not look good, due to:

In such cases, the user may erase the Matrix of Samples by using the "Samples"– "Reset Matrix of Samples" menu item, and can then re-calibrate, providing proper conditions for the process.

Manual calibration

The user can select samples for the Matrix of Samples in manual mode. By using this feature, the user can replace samples with bad quality received during auto-calibration, without re-calibration or selecting all samples in the manual mode.

The Matrix of Samples for ECTtracker can be filled in 2 ways:

In order to provide more comfort during the filling of the Matrix of Samples, the user can switch the places of the samples. In order to do that, the user has to stop the program by pressing the "Stop" item of the menu. After that, it’s required to use the left or right mouse buttons and click on a sample that needs to be relocated. That sample will be highlighted with a blue border (see fig. 10). After that, the user has to click on the second sample and they will be switched.

Switching samples in the Matrix of Samples (Fig. 10. Switching samples in the Matrix of Samples)

Important information! Proper calibration and correct Matrix of Samples filling are required to improve image tracking quality. The overall accuracy of the current state of the user's eye tracking depends on the initial setting quality. After performing a proper calibration, the patient will be able to work with the matrix of symbols longer, making less mistakes and reducing stress.

Guidelines for the filling of the Matrix of Samples:

More detailed recommendations, full descriptions, diagrams and illustrations can be found in the "Tips and tricks" chapter.

Completing configuration

To complete the initial setting process of ECTtracker, the user also has to indicate the following parameters in the settings window of the program (it can be opened with the "Settings" – "Show Settings" menu items or by pressing the F3 hot key):

In order to improve the overall image recognition quality, the user has to define the priority levels of the color channels. Advanced users can select these priority levels manually by changing the value of parameter 15 of the settings windows of the program. The program also supports the automatic priority calculation mode. In order to use it, it’s required to select the "Settings" – "Set Channel Formula" items of the main menu.

When the user completes the selection of all desired parameters and the recognition structure, specification of the analyzing area of the image using the target window and creation of the Matrix of Samples, ECTtracker can be started. To start operation of the program, the user needs to select the "Start" item of the main menu or press the F2 button.

Comparison of the analyzed image and samples from the Matrix is performed for each frame, while coordinates of centers of samples exceeding the predetermined matching level (wlim parameter) are shown in the list of recognized samples in the main program window. Information about the sample with the maximum matching level is shown in the information field of ECTtracker.

The program determines the most appropriate sample for the current image. The corresponding sample is highlighted with the red color in the Matrix of Samples. Depending on the row of this sample, the program selects a predefined key code and action, which are then sent to any receiving software. In cases where none of the samples correspond with the current image, the program completes one of the actions predefined in parameter 9 of the settings window. If parameter 9 has the C value (search samples from the center part), ECTtracker moves the search point to the center of the analyzed image and resumes the search process in a while. If this parameter has the F value (search samples starting from last coordinates, where match was found), the program is going to search the sample within the limits of the last matching zone. The amount of frames to skip when no corresponding samples are found can be specified in parameter 6 of the settings window of the program.