Quantitative Analysis of Eye Movements during Singing of Early Childhood Children

Early childhood children tend to make musical expressions watching other children or the teacher’s piano accompaniment. However, it has not been inspected yet how eye movement is affected by music. To provide the optimized procedure to capture eye movement’s characteristics reflecting music, the statistical technique was used to evaluate effective parameters. In this study, eye trackers (Tobii Glasses 2) were used to acquire data of eye movements during musical expression of early childhood children and to conduct quantitative analysis. 3-year-old, 4-year-old, and 5yearold children in two early childhood facilities (n=58) participated in eye-tracking while singing multiple songs of major and minor. This paper focuses on saccade (rapid eye movement) and gaze behaviors of early childhood children and mainly conducts, a three-way analysis of variance (ANOVA) on the acquired data (age * facility*tonality). As a result, it was found that the number of occurrences of saccade and the total moving distances of saccade showed a statistical significance between means regarding differences in the tonality of major/minor key of songs, and childcare forms.


Introduction
Regarding relationship of eye movement and body control, evidence from previous studies suggests that saccade and gaze behavior infer body position and postural control. However those studies are targeting only adults and not including music induced responsive body movements.
The author has been interested in evolution of musical expression of children influenced by music education. For four years, the author has quantified changes of body movements included in musical expressions utilizing 3D motion capture (XSENSE MVN system) during the practical process of musical expressions in order to evaluate development of the musical expressions in early childhood. Generally, currently available research reports that use 3D motion capture in the field of education show consideration of the results of specific motion analysis of traditional Japanese dance and sawing, mainly for adults, and development of learning support methods (Ando & Sumikawa, 2012;Sato, Kaiga, & Watabe, 2010). Researches on music and movement show responses to adult sound (Burger, 2013) viewing experiments and video analysis on the relationship between performer movements and expressions (Dahl & Friberg, 2007;Thompson & Luck, 2012). Those researches did not aim to capture the musical expressions during the process of continuous musical practice in early childhood.
In order to give clear vision of relationship between music and induced body movements, the author have conducted machine learning by using the motion capture data acquired in 2016 as the baseline training data, and performed varieties of classification model on the acquired data from 2017 to 2019 to predict the developmental degree of musical expression in early childhood. Machine learnings are well introduced in educational fields such as behavior recognition and individual recognition methods in daily life (Kodama, The method of this study is to quantitatively capture the saccade when a child singing a song, and the following steps are showed.

Eye Tracking Experiments During Singing A Song by Early Childhood Children
Children in early childhood are singing songs while watching the accompaniment of teachers in nursery schools and the children around them. The author confirmed it by video analysis when observing the behavior of children, but could not quantitatively grasp how the actual eye movements were. Therefore, in this study, the author inspected using an eye tracker (Tobii Pro glasses 2) to focus and quantify the eye movements of early childhood children during singing a song.
Tobii Pro glasses 2 is a glasses-type eye tracker composed of one scene camera and four eye cameras with remotely wired connection to data recorders. The glasses weight 45 gramms and capture wide angle, which enable natural viewing behavior by children. Each participant child puts on it in the same way as glasses, calibrates it with a strap fixed, starts singing with a signal. The participant child was measured by eyetracking system until the end of singing. It is a 1/50 second time frame at 50Hz, and audio and video are recorded at the same time. Tobii Pro Lab with I-VT fixation filter (30 degrees/ second) was utilized to identify saccade characteristics. In this study, the children in the two nursery schools experience in their musical expression every day, and each participant sang one song in major key and another one in minor key. At U nursery school, the song in major key was "Umi" (G major, Japanese lyrics: Ryuha Hayashi, composition: Takeshi Inoue), and the song in minor key was "Ureshii HinaMatsuri" (A minor, Japanese lyrics : Sato Hachiro, composition: Koyo Kawamura). At M children's facility, each participant sang the song in major key as "Kaeru no Uta" (C major, Japanese lyrics: Toshiaki Okamoto, composition: German folk song), and the song in minor key as "Teruterubozu" (A minor, Japanese lyrics: Asahara Kagamimura, composition: Shinpei Nakayama). The measurement and analysis time per person was 20 seconds for songs in major key and 20 seconds for songs in minor key.

Participant Children of Eye Tracking and the Survey Schedule
3-year-old, 4-year-old, and 5-year-old children in U nursery school(n=28) participated in eye tracking during singing from 9:30 to 11:00 on August 18 in 2020, and M facility children (n=30) from 14:30 to 16:00 on August 19 in 2020. U nursery school used a play-centered childcare form, and the M facility followed the Montessori method.

Quantitative Analysis of Acquired Data by Eye Tracking
In this study, in order to inspect how the eyeball moved, rapid eye movements from one fixation-point to another fixation-point was mainly analyzed. The author quantitatively analyzed the number of saccades that occurred during the measurement time, the moving average of angular velocity, the average of moving distance (total angle), the moving average of angular velocity of the first saccde and the magnitude of the first saccade.
Regarding the average value of those data, it was inspected whether showed a statistically significant difference by facility (U nursery school, M facility), age (3-year-old, 4-year-old, 5-year-old) and tonality (major/ minor). ANOVA (a three-way non repeated two standards as child facilities, non-repeated three standards as ages) and non-repeated two standards as tonality was used. Furthermore, a statistical difference by four songs was inspected by ANOVA (a two-way non repeated four standards as songs, non-repeated three standards as ages) regarding the total moving distance of saccade.

Results
In this paper, the author shows an example of individual calculated data of eye movement analyzed by analysis software (Tobii Pro analyzer), and quantitatively analyzes the data items presented by the method of this study.
A piano accompaniment with his back turned to the front sits on the front left side of the participant child, and other children sat on the front right side. Some of the other children sang together, moved spontaneously, and took a beat. According to the video analysis shown in Figure 1.1 and Figure 1.2, the 3-year-old girl to be measured first looked at the other children, then looked at the piano accompaniment on the left side, and then saw the other children again. Singing while watching, she saw the piano accompaniment again at the end of the singing. In particular, the circles surrounding the numbers in each figure indicate that the time spent paying attention to them was relatively long, implying this participant child looked more closely at other children than with the piano accompaniment. Figure 1.2 shows a lot of eye movement on the right side where the other children are. Regarding "Umi", the movement of the line of sight during singing tended to follow a similar moving trace for 4-year-old and 5-year-old children.

Movement of the Line of Sight When Singing "Ureshii Hinamatsuri" (minor)
The following Figure 1.3 shows the order of eye movements of a 4-year-old child (boy) when singing "Ureshii Hinamatsuri" (minor), and Figure 1.4 shows the heat map. When singing "Ureshii Hinamatsuri", a piano accompaniment with his back turned to the front was sitting on the front left side of the participant child, and other children to be measured were sitting on the front right side, the same as environment when singing "Umi". According to Figure 1.3 and Figure 1.4 and the video analysis, the order of looking at the other children on the right side and the piano accompaniment on the left side was similar to that of "Umi". In the case of "Ureshii Hinamatsuri", the child sang while watching the piano accompaniment, and did not move as much to the left and right as when "Umi" was sung, but at the end of the singing, he watched the piano accompaniment for a long time. This tendency also occurred at other ages of participant children.

Gaze Movement When
Singing "Kaeru no Uta" (major) Figure 1.5 below shows the order of eye movements of a 3-year-old child (girl) when singing "Kaeru no Uta" (major). Figure 1.6 shows a 4-year-old child, and Figure 1.7 shows the order of movement. It shows the order of eye movements in a 5-year-old child.  A three-way ANOVA (non-repeated two standards as child facilities, non-repeated three standards as ages and non-repeated two standards as tonality (major / minor)) was carried out to inspect whether a statistically significant difference was showed in the average value of the number of saccades in the participant children. As a result, a main effect / interaction of the test showed a statistically significant difference (child facility: F(1, 104)=37.185, p<.005), child facility* tonality(major/ minor): F(1, 104)=13.782, p<.005). The test regarding simple main effect test with multiple comparisons was carried out by Bonferroni method.

Number of Saccades
Concerning the facility factor/ age factor * tonality factor, the simple main effect was statistically significant in 3-year-old (F(1, 104)=17.988, p<.005), 4-year-old (F(1, 104)=10.171, p<.005) and 5year-old (F(1,104)=21.433, p<.005). As a result of multiple comparison, U nursery school was significantly larger than M facility in minor key for all 3-year-old, 4-year-old and 5-year-old children. In this way, statistically significant differences were not observed in the major songs but in the minor songs, and the number of saccades during each measurement of U nursery school children was significantly larger than the number of saccades during each measurement of M facility children.  There is no statistically difference in the number of saccades of the major "Umi" and minor "Ureshii Hinamatsuri" between 3-year-old and 4-year-old children, but in 5-year-old children, when singing a minor song, saccade indicates that the number of occurrences of is large. Figure 2.2 shows that saccades in M child facility when the children singing the song "Kaeru no Uta" in major key occur more than singing the song "Teru teru Bozu" in minor key. It shows that the frequency of occurrence in 3-year-old and 5-year-old children is characteristic.
Furthermore, a two-way ANOVA (non-repeated four standards as songs and non-repeated three standards as ages) was carried out to inspect if there was a statistically significant difference among songs. As a result of main effect / interaction of the test, a main effect showed a statistically significant difference (song: F(3, 104)=17.661, p<.005). As a result of multiple comparisons, saccades occurred significantly more frequently in 3-year-old and 4-year-old children in "Umi" and "Ureshii Hinamatsuri" than in "Teru teru Bozu". The frequency of saccades in 5-year-olds was significantly higher in "Ureshii Hinamatsuri" than in "Teru teru Bozu".

The Moving Average Velocity of Saccade (Angular Velocity)
In the case of eye movements, the moving average velocity of rapid eye movements is expressed in angular velocity (angle / second). Similarly, a three-way ANOVA (non-repeated two standards as child facilities, non-repeated three standards as ages and non-repeated two standards as tonality (major / minor)) was carried out for this acquired data. A result of main effect/ interaction of the test did not show a significant difference, but the moving average velocity of saccade in 5-year-old children in M child facility was significantly larger than that in 3-year-old children in the minor song. The moving average velocity of saccades for 3-year-olds in M child facility was significantly larger in major songs than in minor songs. In the song "Teru teru Bozu" (minor), the moving average velocity of the saccade for 5-year-olds was larger than that for 3-year-olds. As shown in Figures 3.1 in U nursery school, the moving average velocity (246.329 degree / second) when a 4-year-old child is singing a song in minor key was larger than the moving average velocity (224.392 degree / second) when singing a song in major key. For 5-year-olds, it was larger when singing a song in major key (254.645 degree / second). As shown in Figure 3.2, regarding 3-year-old in M child facility, the moving average velocity (238.011degree/ second) when singing a song in major key was larger than the moving average velocity (186.3311degree / second) when singing a song in minor key although a significant difference between major key and minor key was little observed in 4-yer-old and 5-year-old.  Figure 3.3 there is not much change by age at the "Ureshii Hinamatsuri ", and at "Kaeru no Uta", "Umi", and " Teru teru Bozu", the moving average velocity increases as the age increases. In particular, in the song "Teru teru Bozu", change of 5-year-olds were larger than 3-year-olds and 4-year-olds.

Average Data of Saccade
Saccades have occurred many times, but the average values did not differ significantly. Figure 4 shows the average value of saccade by age for each song.
As shown in Figure 4, the average saccade of "Teru teru Bozu" by 5-year-old children was larger than that of 3-year-old and 4-year-old children.

Total Moving Distance of Saccade (Total Angle)
In the case of saccade, the total moving distance is indicated by unit of angle. Concerning the result of a three-way ANOVA (non-repeated two standards as facilities, non-repeated three standards as ages and non-repeated two standards as tonalities(major/ minor)), a main effect/ interaction of the test showed a statistically significant difference (facility: F(1, 104)=18.53, p<.005), facility* tonality (major/ minor): F(1, 104)=13.382, p<.005). The test regarding simple main effect with multiple comparisons was carried out by Bonferroni method.
Concerning the facility factor/ age factor * tonality factor, the simple main effect was statistically significant in 3-year-old singing in minor key (F(1, 104)=12.725, p<.005) and 5-year-old singing in minor key (F(1, 104)=11.795, p<.005). As a result of multiple comparisons, the total moving distance of saccade regarding 3-year-old, 4-year-old and 5-year-old singing in minor key, U nursery school was significantly larger than M child facility. The average data of total moving distance of saccade when singing in minor key was larger than the average data singing in major key regarding 5-year-old in U nursery school. The average data of total moving distance of saccade when singing in major key was larger than the average data singing in minor key regarding 3-year-old in M child facility at the measurement time.
The following Figures 5.1 and Figure 5.2 show the average value of the total moving distance of saccade by age in major and minor, by facility.  Figure 5.2 shows that in U nursery school, the total moving distance in 5-year-old child is larger for songs in minor key than for songs in major key.

Discussion
In this article, the author used an eye tracker (Tobii Pro Glasses 2) to acquire data on eye movements during musical expression of participant child and to conduct quantitative analysis. A three-way ANOVA was mainly focused on saccade acquired data when singing songs in different major and minor keys by 3-yearolds, 4-year-olds, and 5-year-olds in two early childhood facilities. In general, findings show that significant differences captured in age groups and other factors suggesting change of eye movement based on accumulated experience of music educational influence. In contrast, there were some parameters which were less evident to evaluate as close relationship between eye movements and music experience to be argued for feature quantity for machine learning.
First, when the author is to understand from visual tracking movements examined individually, from heat maps, all the participant children were looking at the appearance of the other child on the front right side and the piano accompaniment on the front left side, and the line-of-sight movement of the U nursery school child was more frequent than that of the M child facility child. Furthermore, U nursery school children tend to look at the piano accompaniment with songs in minor key rather than major key, and participant children in M child facility looked at both the appearance of other children on the front right side and the piano accompaniment player on the front left side.
Next, based on statistical analysis of a three-way ANOVA based on the facility factor, major / minor factor, and age factor, the acquired data clearly showed a statistically significant difference in the number of occurrences of saccade and the total moving distance. It was found that the number of saccades in U nursery school children was significantly larger than that in M facility children when singing songs in minor key in all 3-year-old children, 4-year-old children, and 5-year-old children. Regarding the total moving distance, 5year-old children in U nursery school sang a song in minor key was larger than the measurement when singing a song in major key. The total moving distance of 3-year-old children in M child facility when singing in major key was larger than when singing a song in minor key. These results are considered to depend on the difference in childcare forms between M early childhood facility and U nursery school. M facility takes a childcare form following Montessori method. Children in M facility have everyday life experiences and musical experiences that are sensitive to the regularity and contrast of various things, so they feel the brightness in the major key of songs and the darkness in the minor key of songs. It can be said that the awareness of the difference due to the tone of the song was reflected in the eye movements during singing.
However, U nursery school takes a play-centered childcare form. The children in U nursery school responded to the meaning of the Japanese lyrics rather than the tune, and the spontaneous gestures that accompany the singing become larger. In order to share with other children to minor key of tunes such as "Ureshii Hina Matsuri", it was found that the eye movements for the songs in minor key were also increased in U nursery school.
On the other hand, among the results of researches on adults by Gingras, Marin, Puig-Waldmüller, and Fitch (2015) and Fink, Geng, Hurley, and Janata (2017) the unchanged pupillary response was the same for children in early childhood obtained in this study by the author. Although an individual difference was observed in the pupil diameter, it changed from 3 mm to 5 mm, and it was found that no statistically significant difference was observed.

Conclusion
In this way, the differences of saccade were observed in major / minor and the type of childcare forms when children in early childhood sing by using eye tracker-assisted method for detecting visionary responses. The present investigation provides evidences that music stimuli gives interaction with eye movements. This finding encourages further research with increased number of songs and the number of participant children for songs with various tunes, to asses variety of conditions affecting visual behaviours. Such exploitation of saccades, fixation or any oculomotor movements could be added to feature quantities of 3D motion capture to enhance accuracy of machine learning classification algorhythms which forecasts development degree of sophistication of musical expression.
There is also the challenge of simultaneous analysis with the motion analysis of musical expressions. Regarding research report of the combined use of eye tracker and motion capture, Saakata, Yatumura, Choku, and Yoshimura (2005) quantitatively analyzed the relationship between specific movements of Japanese dance and eye contact, and simultaneous measurement of search step movements and line of sight (Hashimoto & Sato, 2018). Burger et al. (2018) attempted a quantitative analysis of gaze and physical movements that differed according to musical characteristics in a small number of adults. The research point out the need for simultaneous measurement of eye trackers and motion capture, as human musical behavior interacts with gaze and physical movements. Those previous studies are aimed at adults, and there are no research reports on musical expressions of early childhood children so far, so it is necessary to inspect the research method.