Analysing and Interpreting Quantitative Eye-Tracking Data in Studies of Programming: Phases of Debugging with Multiple Representations

Roman Bednarik; Markku Tukiainen

While eye-tracking systems become gradually improved and easier to apply, the methodological challenges of how to analyze, inter- pret and relate the eye-tracking data to user processing remain. Studies of programming behavior are not an exception. We have conducted a reanalysis of eye-tracking data from a previous study that involved pro- grammers of two experience groups debugging a program with the help of graphical representation. Proportional fixation time on each representa- tion, frequency of visual attention switches between the representations and type of switch were investigated in relation to five consequential phases of ten minutes of debugging. Therefore, we have increased the granularity of focus on debugging process.

We found some repetitive patterns of visual strategies that were asso- ciated with less experienced programmers finding fewer errors. We also discovered that at the beginning of the process programmers make use of both the code- and graphical representations while frequently switching between them. During the process, more experienced programmers be- gan to integrate also the output of the program and finish the debugging with frequent switching between the three representations. We discuss benefits and limitations of this approach to analyzing and interpreting the quantitative eye-tracking data. As part of future research we propose to investigate the symmetries of representation switching behavior. 

Type of Publication: Paper
Conference: PPIG 2007 - 19th Annual Workshop
Publication Year: 2007
Paper #: 13
TitleAnalysing and Interpreting Quantitative Eye-Tracking Data in Studies of Programming: Phases of Debugging with Multiple Representations
Publication TypePaper
AuthorsBednarik, R, Tukiainen, M
PPIG Workshop: 
2007-07-19th