Replicated Computations Results (RCR) report for "reusing Search Data in Ranking and Selection: What Could Possibly Go Wrong?"

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalNot applicablepeer-review

View graph of relations


Original languageEnglish
Article number19
Journal / PublicationACM Transactions on Modeling and Computer Simulation
Issue number3
Early online date9 Aug 2018
Publication statusPublished - Aug 2018


"Reusing Search Data in Ranking and Selection: What Could Possibly Go Wrong?" [2] by Eckman and Henderson rigorously defines the statistical guarantees for ranking-and-selection (R&S) procedures after random search, and points out that the simulation replications collected in the search phase are conditionally dependent given the sequence of returned systems. Therefore, reusing the search data for R&S may affect the statistical guarantees. The authors further design random search algorithms to demonstrate that the correct selection guarantees of some ranking-and-selection procedures will be compromised if reusing the simulation replications taken during the search. This replicated computation report focuses on the reproducibility of the experiment results in the aforementioned article.

Research Area(s)

  • Random search, Ranking and selection, Simulation optimization