Lessons from conducting a distributed quasi-experiment

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)22_Publication in policy or professional journal

8 Scopus Citations
View graph of relations

Author(s)

  • David Budgen
  • Barbara Kitchenham
  • Stuart Charters
  • Shirley Gibbs
  • Amnart Pohthong
  • Pearl Brereton

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number6681347
Pages (from-to)143-152
Journal / PublicationInternational Symposium on Empirical Software Engineering and Measurement
Publication statusPublished - 2013

Conference

Title2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2013
PlaceUnited States
CityBaltimore, MD
Period10 - 11 October 2013

Abstract

Context: Due to the lack of suitably skilled participants, software engineering experiments often lack the statistical power needed to detect the levels of effect that may be encountered. Aim: To investigate whether this can be remedied by running an experiment across multiple sites, organised as a single study rather than as a set of replications. Method: We performed a 'trial' of the idea using a topic (structured abstracts) that some of us had studied previously and which required no participant training. We used five sites, each with 16 participants. Results: We were able to demonstrate the benefits of increased statistical power (and of structured abstracts). We report on our experiences with designing and conducting the study and identify some key lessons about how future studies of this form might be organised. Conclusions: The distributed model offers a flexible, robust form that is capable of delivering better statistical power than would be achieved by running a set of parallel replicated studies. © 2013 IEEE.

Research Area(s)

  • Documentation, Review and Evaluation, Software Engineering

Citation Format(s)

Lessons from conducting a distributed quasi-experiment. / Budgen, David; Kitchenham, Barbara; Charters, Stuart; Gibbs, Shirley; Pohthong, Amnart; Keung, Jacky; Brereton, Pearl.

In: International Symposium on Empirical Software Engineering and Measurement, 2013, p. 143-152.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)22_Publication in policy or professional journal