Abstract:
One of the main problems associated with empirical studies of programmers is their ecological validity. Participants are often asked to work on unfamiliar systems for a short period of time, using unfamiliar browsing environments. To compound this, they may be required to perform tasks that are unrepresentative of their daily work. This does not suggest sloppy empirical design. Ecological validity is extremely difficult to obtain, especially in the context of formal quantitative experiments where strict controls limit the variability that is normally associated with work contexts. However, this limitation does call into question the relevance of the findings for software practitioners.
The research currently being carried out at the University of Limerick aims to address this difficult ecological-validity issue, specifically in the context of software comprehension studies. We intend to observe programmers performing their job in their working environment and to see the tasks they perform. Subsequently, we intend to identify the knowledge they bring to these tasks and how they use it. Our intention is to perform one-subject studies and to form partnerships with other researchers allowing them to replicate the studies with a high degree of confidence in different contexts. This will allow the community to assess the generality of the findings.
The research currently being carried out at the University of Limerick aims to address this difficult ecological-validity issue, specifically in the context of software comprehension studies. We intend to observe programmers performing their job in their working environment and to see the tasks they perform. Subsequently, we intend to identify the knowledge they bring to these tasks and how they use it. Our intention is to perform one-subject studies and to form partnerships with other researchers allowing them to replicate the studies with a high degree of confidence in different contexts. This will allow the community to assess the generality of the findings.