Easy-to-install system for daily walking ability assessment using a distance sensor array

2.1.Overview of distance sensor array device

Precise measurement of foot movement is indispensable to measure walking ability. A distance sensor is selected as a basic element of our device. The device comprises multiple distance sensors, constituting a distance sensor array. The proposed system measures walking along the sensor array and estimates the walking indexes from data acquired by the device.

Fig. 1.

Conceptual sketch of system installation in a house.

Figure 1 shows a conceptual sketch of the distance sensor array installed in the hallway of a house. To obtain standardized walking data, this device should be installed in a hallway along which a resident usually walks straight. The distance sensor array measures distances between sensors and resident’s feet while the resident is walking. Time series data of walking are obtained by sampling the output of each sensor with high frequency. Finally, walking ability can be estimated by analyzing the time series data. The device length is expected to be about 1.0 m for easy installation and for a resident’s comfortable life. In one day, multiple walking data are expected to be obtained by our system.

2.2.Possibility of the proposed system

This system presents important benefits from the perspective of privacy, ease of installation, and cost. First, the device will not be rejected by users because the device cannot reveal details of their personal life. Second, installation of this device can be done simply by putting the device on a floor. Finally, the distance sensor is an inexpensive sensor in itself. It is necessary to optimize the number of distance sensors to realize an inexpensive device. The benefits explained above enable us to infer that the proposed system using a distance sensor array has high implementability and practicality for automatic walking measurement in a home.

3.1.Distance sensor specifications

We adopted a distance sensor module (Sharp Corp.; GP2Y0A21YK0F, Fig. 2, left) as the main component of this device. This sensor module is composed of an infrared light emitting diode (LED) and a position sensitive device (PSD). The LED emits infrared light one thousand times per second. Then a target object reflects it. As Fig. 2 shows, the PSD detects the position of the light spot on a sensor surface. Then the distance between the sensor and a target object is measured by triangulation. This sensor can measure from 100 to 800 mm. Figure 3 portrays the electro-optical characteristics of the PSD sensor.

Fig. 2.

PSD sensor and its principle.

Fig. 3.

Electro-optical characteristics of the PSD sensor (GP2Y0A21-YK0F). This graph is made from the data-sheet [6].

3.2.Device specifications

Fig. 4.

Overview of distance sensor array device.

Fig. 5.

Close-up view of the measurement circuit and connections among multiple circuits.

Figure 4 presents an overview of the developed distance sensor array device. Distance sensors are arranged at intervals of 50 mm. Because the ankle width is about 60–90 mm, this device can absolutely detect a human ankle. Every four sensors are attached to one measurement circuit module. Six modules are connected and embedded in a case. Figure 5 shows a close-up view of the measurement circuit and connection between multiple circuits. Those six modules use the same electronic circuit board. Their identifiers are configured simply by changing jumper settings. Using the same circuit board to replace six different boards can reduce costs. The total number of sensors is 24. The full length of the device is 1200 mm, which is short enough to install in a hallway, but long enough to capture two or more steps for measuring step length because one step is generally 800 mm long, at most, for an adult male [11]. In our design, the distance sensors are placed vertically at 50 mm height above the floor. These specifications are based on the results of a preliminary experiment. Appendix A presents the details of the preliminary experiment.

Fig. 6.

Block diagram of distance sensor array.

Fig. 7.

Data processing flow in the system.

Figure 6 is a block diagram of the distance sensor array. The distance sensors output is sent to multiplexers in a control circuit. A microcontroller operates analog-to-digital conversion and sends values to PC through wireless module XBee. The output voltages of 24 distance sensors can be acquired at 60 Hz. A PC program estimates the step length and walking speed by analyzing time series distance data.

4.1.Regular walking data extraction algorithm

In an actual home environment, not all output of the device is a response to regular walking. Output includes noise and data obtained when a resident wanders around the device. It is necessary to extract only regular walking data as a result of straight walking. This section describes a process for extracting regular walking data from the device output.

Fig. 8.

Binary image generated from raw data.

The top of Fig. 8 depicts the conversion of raw data to a graph. The horizontal axis shows the time. The vertical axis shows the sensor output (distance). The middle of Fig. 8 shows converted sensor output as a grayscale image. The bottom of Fig. 8 shows a binary image. The binary image is created from the grayscale image by binarizing with a threshold (800 mm) because the maximum measurement distance of the sensor is 800 mm and the output distance is 800 mm or less when the sensor reacts to a foot or an obstacle. One pixel of this binary image has information related to the reaction of each time and each sensor. We explore a regular walking data extraction algorithm using this binary image.

Fig. 9.

Regular walking data extraction procedure from time-series data.

The extraction of regular walking data is conducted in two steps, as Fig. 9 shows. In the first step, a period during which at least one sensor reacts is extracted. For example, Fig. 9 shows four candidates of regular walking data. Only (3) shows regular walking data, (1) and (2) are noise, and (4) is obtained when a person stops on the way. The following two conditions can be criteria to extract regular walking data from the candidates in the first step:

4.2.Step length estimation algorithm

Fig. 10.

Step length estimation algorithm.

This algorithm aims to determine the lengths between various steps during human walking. Our proposed algorithm estimates the step length by measuring two foot-landing positions. When a person walks along the distance sensor array device and the person’s foot lands near a specific sensor, that sensor detects it for a long time. Here, we define “reaction weight” at each sensor as the summation of outputs at a certain time. In the top image of Fig. 10, the hatched area shows the reaction weight at each sensor. The bottom part of the figure presents a bar graph showing all sensors’ reaction weights. Because reaction weights near landing positions become larger, peaks are extracted as the landing positions. The bottom part of Fig. 10 shows data obtained when two foot landings are observed. Two peaks are evident in this chart. The following peak-extractor-processes detect abstract positions of peaks.

Next, the local center of gravity of reacting weights is calculated from five neighboring sensors. The local center of gravity xc is calculated as

4.3.Walking speed estimation algorithm

Next, this section describes a walking speed estimation algorithm. First, a walking movement is not a uniform motion involving the whole human body. Consequently, it is difficult to estimate walking speed merely by monitoring the speed of a specific human body part. For this reason, we devised the following technique to extract one walking movement sequence. The predefined binary image can be a good clue for this technique. Figure 11 portrays a binary image obtained when a person walks along the device and trajectories of right and left foot movement acquired by a motion capture system.

Fig. 11.

Binary image created from distance sensor array data and motion capture data. Horizontal axis shows time. Vertical axis shows the foot moving distance.

This motion capture data show that the right foot is moving forward while the left foot is contacting the ground – and vice versa. Similarly, the binary image shows the trajectory of the movement of both feet. For this study, we use the binary image and estimate walking speed by extracting one cycle of walking. The cycle starts at the heel strike of a foot and ends at the other foot heel strike. Detailed procedures of walking speed estimation are shown in Fig. 12.

Fig. 12.

Walking speed estimation algorithm.

Step 1: Dilate and erode the binary image for eliminating small holes and filling gaps [22].

Step 2: Detect the lines denoting landing in the binary image using a Hough Transform [12]. These lines are parallel to the time axis because the sensor ID values are constant. Consequently, in a parameter space for the voting procedure, the angle parameter is restricted to 0 deg (parallel to the time axis) so that unnecessary lines might not be detected. Two foot steps are detected as multiple lines as shown in the third image of Fig. 12.

Step 3: Classify these detected lines in Hough parameter space. At this point, because the number of landings in the sensing area is unknown, the number of clusters is also unknown. Therefore, aggregative hierarchical clustering is adopted as the clustering method because it is not necessary to define the cluster numbers in advance. The minimum pixel number in the sensor ID axis between two lines is the stopping criteria of clustering. If the distance between the closest two lines from different clusters is greater than a threshold value (4 pixels; 200 mm in our implementation), then the clustering process stops. In fact, two steps are classified as two clusters in the fourth image of Fig. 12. Pixels of the end points of the detected lines show the time when the sensors begin to react to the landings. The median of the end points is defined as the estimated landing time because one landing includes multiple lines. Two consecutive landing times are estimated in this way. Step duration, the total time spent on a step, is calculated by subtracting these times. Walking speed is calculated as the estimated value of the step length divided by this step duration.

4.4.Benefits of estimating algorithms

These algorithms entail no troublesome calibration. Nevertheless, it is difficult for these algorithms to be applied to extremely slow walking. For walking when the step length is shorter than the foot length, peak detection is difficult in the step length estimation process because landing positions overlap. In addition, these algorithms require no complex calculations. Moreover, the system processes walking data in real time.

5.1.Experiment of regular walking data extraction

In the regular walking data extraction procedure, our proposed system eliminates action data other than that of regular walking. The algorithm is applied to regular walking data and other action data. Its performance is evaluated. Four healthy men (aged 21–32 years, mean age 25.0 years) participated in the experiment. They walked 20 times straight and smoothly along the sensor array device. In addition, non-walking pseudo-action data were collected by the device. The participants were given some instructions to play non-walking actions. Collected data were classifiable into five action groups based on the home monitoring result presented in Appendix B. Each participant did each non-walking action twice. Figure 13 depicts examples of collected data. The data are shown as binary images. In summary, we obtained 80 regular walking data and 40 non-walking action data.

Fig. 13.

Measurement and example data of various actions.

“Precision”, “Recall (Sensitivity)”, and “Specificity” described in Eqs (2), (3), (4) are used as evaluation indexes.

Results are shown in Table 2.

Fig. 14.

Results of step length and walking speed estimation for extracted data as regular walking.

All regular walking data are extracted, which means that Recall (Sensitivity) is 100%. In contrast, Precision is 97.6% and Specificity is 95.0% because of the incorrect extraction of two other action data. These two data are “Stop on the way” data that were collected from two different participants. 21 data including incorrect data were targets of step length and walking speed estimation for these two participants. Estimation results are shown in Fig. 14. The estimated values from regular walking data are concentrated in a certain area. However, the estimated value from “Stop on the way” is plotted at a distant position from the area. Non-walking data can be excluded easily based on the average and distribution of regular walking data.

Fig. 15.

Close-up view of floor sensor and acquired data sample.

5.2.Basic performance evaluation

To examine the basic performance of walking ability estimation, we conducted an experiment with healthy young adults. This experiment was conducted to verify the feasibility of our system and to fix experimental procedures before conducting an experiment with elderly people. Participants were 8 men (aged 21–32 years, mean age 23.5 years). In the experiment, a high-resolution floor pressure sensor system [16] was used for acquiring reference data. Figure 15 (left) portrays a close-up view of the floor pressure sensor system. Table 3 shows specifications of the floor pressure sensor system. To collect walking data using both the distance sensor array device and the floor pressure sensor system, we constructed a walkway as shown in Fig. 16.

From floor pressure sensor system data, two steps’ footprints were extracted using a labeling process. Figure 15 (right) portrays an example of extracted footprints. Tail ends of two footprints are designated respectively as x1 and x2. The first time stamps, when the footprints are detected, are designated as t1 and t2. Consequently, step length l and walking speed v are calculated in Eq. (5).

Fig. 16.

Walkway used for the experiment.

Participants walked 20 times. Although the total number of walking samples was 160, 158 data were used for evaluation. Two data were excluded because two steps were not detected. When a step length was too long compared with the distance sensor array device, only one step was detected. Sometimes more than three steps were detected. However, the first two steps were used to estimate the step length and walking speed.

Evaluation indexes were mean absolute error (MAE), mean of the absolute relative error (MARE), and standard deviation of the absolute relative error (σARE). MAE is an average of the absolute error value and calculated in Eq. (6). MARE is calculated as shown in Eq. (7). Absolute relative error (ARE) is calculated in Eq. (8).

MAE is a metric used to evaluate how precisely the algorithms estimate walking ability in the view of its original unit. In contrast, MARE shows the accuracy ratio: relative performance. Compared to root mean squared error (RMSE) and relative root mean squared error (RRMSE), they are stable against outliers.

Fig. 17.

Comparison of estimated and reference values in the basic experiment.

Figure 17 presents scatter diagrams of the step length and walking speed. Reference values are plotted on the horizontal axis. Estimated values are plotted on the vertical axis. Almost all obtained data are distributed over step length 500–700 mm and walking speed 0.8–1.2 m/s. As shown in Table 4, the step length and walking speed can be estimated with high accuracy. MARE is less than 5% for both these indexes. The superior estimation accuracy was demonstrated in step length (about 15 mm), although distance sensors are arranged at intervals of 50 mm.

5.3.Experiment with elderly people

Based on the result of the basic performance evaluation, we conducted an experiment with elderly people, who are the main target of this research. Ten participants (5 men and 5 women, aged 72–86 years, mean age 78.5 years) participated in the experiment. All participants were attendees of “the Japanese Red Cross Day Care Center”. However, they were able to walk without a cane. Before the experiment, we obtained permission to conduct the experiment from the Center Director. All participants were informed of the purpose and methods of this study. Written informed consent was obtained from each participant’s family. The experimental setup was the same as that of the basic performance evaluation experiment. Figure 18 presents a snapshot of the experiment. Ten walking sequences were observed for each participant. We analyzed 100 walking sequences.

Fig. 18.

Snapshot of the experiment with elderly people.

Experimental results are presented in Fig. 19 and Table 5. Obtained data are distributed over step lengths of 300–600 mm and walking speed 0.5–1.0 m/s. Step length and walking speed were estimated with high accuracy, as shown also for the prior experiment. However, MARE and σARE, which show relative error, are worse than the results of the prior experiment. This is because denominators of those indexes are smaller, which means that the step length is shorter and walking speed is slower for elderly people than for healthy young participants. In fact, MAE, which corresponds to the resolution of estimation, shows a superior score that is equivalent to that of the prior experiment.

Fig. 19.

Comparison of estimated and reference values for the experiment with elderly people.