Evaluation techniques for systems analysis and design modelling methods - a review and comparative analysis

Even though hundreds of modelling methods are in existence today, practitioners and researchers are zealously 'producing' new modelling methods. The 'blooming' production of modelling methods is not the problem; the lack of standardized techniques for evaluating them is. To complicate the matter even further, most of the modelling methods have been introduced based on common sense and intuition of the methods' developers. Many of these methods lack theoretical foundations and empirical evidence to demonstrate their worthiness. With the current state of affairs, studies on evaluation of modelling methods have become necessary and critical. Comparing modelling methods provides us with the necessary knowledge and understanding of the strengths and weaknesses of each method. This knowledge can also guide us in our quest for better modelling methods. This paper reviews various evaluation techniques used by both researchers and practitioners. The evaluation techniques are categorized into three classifications: feature comparison, theoretical and conceptual evaluation, and empirical evaluation. This research also analyses the underlying philosophies and assumptions of these evaluation techniques. © 2007 The Authors. Information Systems Journal © 2007 Blackwell Publishing Ltd.