By Jennifer Ferreira
This is a trip report on the 8th International Advanced School in Empirical Software Engineering that took place on 15 September 2010, as part of the Empirical Software Engineering week held in Bolzano, Italy.
The event lasted a full day and the aim was to introduce attendees to the use of ethnographic methods for studying the complexities of software practice. The topics for the day focused on conducting, analysing, presenting and evaluating ethnographic studies. The chairs for this event were Helen Sharp, Yvonne Dittrich, Cleidson de Souza and myself. While Helen and I were physically present, Yvonne and Cleidson were connected throughout the day via Skype.
The seven attendees came from various countries around the world, including Italy, Denmark, the US and Brazil. Some were already doing qualitative and/or empirical research, while others were about to begin. So experience ranged from those who were interested but not necessarily doing ethnographic research, through those who had just begun, to those who had conducted ethnographic studies before. Attendees were a combination of PhD students, post-doctoral researchers and lecturers.
Next, is my very condensed account of what was an engaging and informative day ...
The day kicked off with introductions and we made a list of the burning issues that attendees wanted to address during the day. Most were interested in finding out more about qualitative studies and ethnography in particular.
Helen's presentation started with asking "What is Ethnography?" She emphasised that the ethnographer sets out to understand the informant's point of view by being present at the research site and participating (to various degrees) in what happens there. We also considered when NOT to use ethnography. In response to this, Yvonne spoke about agendas and, as a researcher, being aware of the agendas of management especially. If you realise that you don't agree with the ethics of management's agenda (for example, if their reason for allowing you to carry out research with them is to reduce staff) then continuing to work them might not be a good idea.
After the morning coffee break, we continued with "Myths and expectations" and "Doing ethnography." As part of the discussion of the myths we discussed the unexpected aspects to doing ethnography. For example, the unexpected amount of explaining you have to do at the research site. You may have negotiated access to a research site with one person, who may or may not be part of the team that you want to observe.
Inevitably, when you arrive on site, you run into many other people who have no idea why you're there and who may not even like that you're there. This led to a discussion about the importance of trust and how that is linked with the kind of access the researcher can expect. We agreed that building trust is an ongoing process between the researcher and participants, meaning that aside from non-disclosure agreements and other confidentiality arrangements, the levels of access to the research site can vary throughout the study depending on whether the researcher is trusted. In the few minutes before lunch the attendees were given a practical exercise to have a go at during the lunch break: to find a group of people to observe and spend approximately 10 minutes collecting ethnographic data.
After lunch, attendees were invited to report back and reflect on their experiences. The waiters at the sit-down lunch turned out to be popular "targets" for observations. Participants had collected various types of data – some had taken photographs, conducted short interviews while others covertly made field notes. Consensus at the end was that next time, they would like to inform their participants of what they are doing before they launch into observations. However, the point the exercise conveyed was that the challenges of ethnographic studies are the (often social) challenges that come with working with real people in real settings. Next, we considered ways of analysing and presenting qualitative findings, using the data collected for the practical exercise.
One of the issues that came up was how to deal with documenting the researcher's reflections while in the field and how to deal with that during the analysis. Yvonne recommended a strategy that can be applied to written field notes. Each page in your notebook can be divided into three sections: one for noting down the observations (what you see happening), one for your reflections on what you're observing and a third for noting emotions or mood. Some attendees made the observation that mood could affect how we perceive what is going on in the field. Therefore, along with issues such as bias, the key is to be as self-aware as possible during the collection as well as analysis stages - a self-awareness that prevents us from asking leading questions during interviews, or carrying out emotionally-charged analyses.
After the final coffee break we moved on to dealing with the "so what?" question. We agreed that our findings could have implications for tools, processes and company strategy; that our findings can bring to light the problems, conflicts and successes of software engineering in practice; and that our ultimate aim is to improve software engineering.
As the final bit of excitement for the day, we projected Yvonne on to the big screen so that she could present to the group via Skype. Her presentation dealt with applying an action research approach in combination with ethnography. The action researcher intervenes and changes practice to solve real world problems. As an example, an action researcher may identify a lack of communication between designers and developers at a research site, introduce a new method to aid the problem and then evaluate whether their communication improved. Clearly, this approach aligns well with our overall intention of improving software engineering practice.
By the end of the day everyone felt that their burning issues and questions had been addressed and they had gained plenty of food for thought. Thanks to the enthusiasm of the attendees, their interesting questions and lively discussions this year's advanced school was a great success. The chairs also had excellent support from the conference organisers Barbara Russo, Bruno Rossi and Andrej Gavrilov.