Can an objective measurement of design protocols reflect the quality of a design outcome?

In this paper we develop a method for measuring the design process and correlate it with the assessment of the design outcome. We used linkography - a technique used in analyzing design protocols - which is reported to reveal the quality of design ideas and to capture the interconnectivity of ideas in the design process. Previous studies found that Shannon's entropy as a measure of a linkograph might indicate design outcomes. This paper reports on an empirical verification that design outcomes can be correlated to the entropic interpretations of a linkograph. Experimental data from protocol studies of six designers performing two similar design tasks in two different conditions were analysed. In one condition the designers were blindfolded during designing, and then given a relatively short time to draw their design at the end of the session. In the other condition the designers were allowed to sketch. We focus on the three highest scored and the three lowest scored design sessions. Their verbalization during the design session were segmented and coded to produce linkographs. The entropy and the change of entropy over time of these linkographs were studied in relation to the design outcomes. Our results demonstrate that entropy measurement of linkographs provides a way of to study design protocols and show that the design outcome is more related to the rate of change of entropy than to the entropy alone.