Definitional aspects of the Delphi method

From FORwiki

Jump to: navigation, search

The basic Delphi concept is the design of a collaborative communication structure and process that is tailored to the nature of the problem and the nature of the group[1]. Although it was used largely in the early days for predicting future technical breakthroughs, it has been used to address a wide range of complex problems that are often current and it has been used subsequently to try to understand the past as well as the future. There have, for example, been a number of examples of experts in a given field using a Delphi to establish the most significant contributions to their field.

Contents

Participation

Anonymity of the responses is one fundamental property so that people will feel free to express themselves and to be able to expose ideas that could turn out to be stupid as well as brilliant. However, in some current online approaches it is possible to allow the participants, if they chose, to put in a comment with their true name, or when they want to be anonymous, or when they want to use a pen name. An advantage of the pen name is that they can develop a series of comments to express a coherent viewpoint. In some cases, the respondents are told who is participating so they will feel they are part of a peer group of people they would like to communicate with about the particular topic. Usually those acting as the design team will commit to the fact that who said what would never be divulged to the other participants or to the sponsor.

Delphis that are well done usually try to capture and seed the process with the material that can be found in the literature on the subject, so that those participating realize that they are not being asked to educate the design team on what should be obvious. The material to be asked of the respondents is what would be difficult to find in the literature and what is not obvious. Too many poor Delphis have attempted to give people a blank piece of paper that says "tell me what I should know about this problem!" reference needed.

Motivation factors

Associated with the above is that people have to be motivated to put in the effort to participate in a Delphi exercise. The sort of motivation factors that have to be considered and made clear to the participants are:

  • Is this an important problem that should be addressed by a larger group of experts who will all have an equal opportunity to contribute?
  • Is this the right group to undertake this effort?
  • Is someone or some organization going to make real use of the results of this effort?
  • Is it worth it for me to spend the necessary time to make a good contribution?
  • Will I learn things I should learn from those in other professional areas that are participating?
  • Is it clear to me what the process is and what I will be committing to in time and effort?
  • If some of the above is not true, am I going to be paid, and what is my time is worth to participate?

Round structure

The typical view of Delphi is that it has a round structure and goes through at least three phases:
1. Exploring the problem and exposing new insights and additional relevant material.
2. Gaining a collective understanding of the material generated.
3. Evaluating the material and hopefully reaching a consensus.

This is usually what leads to a three round exercise for Delphi processes done via pencil and paper. Sometimes it does suffer because the design may lead to a premature consensus when there is not an adequate structure to expose hidden disagreements. Sometimes the pressure is to get just quantitative subjective estimates of variables such as costs, likelihood of success, effectiveness, etc. without a sufficient design in the structure to expose hidden or underlying disagreements. Voting is often used as a conclusion rather than for its real purpose, which is to expose potential disagreements and get rid of possible ambiguities so that true uncertainties can be dealt with. This leads to a number of other requirements that when done with paper and pencil can require five rounds for the complete process. They add the following phases after phase one above.
1.1 Initial voting on generated material to expose disagreements.
1.2 Exploration of the underlying reasons for disagreements.

Underlying the above is the requirement to have a morphological structure for the information that is contributed that allows the participants to input their knowledge into appropriate categories that will organize and cluster information. Today this is referred to as a knowledge structure and it is exhibited in many of the Delphis that deal with complex problems.

References

  1. ↑ Linstone, H. and Turoff, M., the Delphi Method: Techniques and Applications by Harold Linstone and Murray Turoff, 1975. [1]

Further reading

  1. Arrow K. J., (1963), Social choice and individual values, 2nd edition, Yale University Press.
  2. Bañuls, V. A. B., Turoff, M., and Silva, J. L., Clustering Scenarios using Cross-Impact Analysis, accepted ISCRAM 2010, Seattle, Washington, May. Contact an author for a copy.
  3. Bañuls, V.A. and Salmeron, J.L. (2007) Benchmarking the Information Society in the Long Range, Futures, 39, 1, 83–95.
  4. Bañuls, V.A., and Salmeron, J.L. (2007), A Scenario-based Assessment Model - SBAM, Technological Forecasting and Social Change, 74, 6, 750-762
  5. Chermack, T.J., (2004), Improving Decision-making with Scenario Planning, Futures, 36, 3, 295-309.
  6. Chermack T.J. (2007) Disciplined imagination: Building scenarios and building theories, Futures, 39, 1, 1-15.
  7. Cho, H. K., Murray Turoff, and Starr Roxanne Hiltz, The Impact of Delphi Communication on Small and Medium Sized Asynchronous Groups: Preliminary Results, HICSS 36, January 2003, IEEE Computer Society Press.
  8. Cho, K.T., and Kwom, C.S. (2004) Hierarchies with dependence of technological alternatives: a cross impact hierarchy process, European Journal of Operational Research, 156, 2, 420–432.
  9. Choi, C., Kim, S. and Park, Y. (2007), A patent-based cross impact analysis for quantitative estimation of technological impact: The case of information and communication technology, Technological Forecasting and Social Change, 74, 8, 1296 -1314. Dalkey, N. C., Brown, H., and Cochran, S. W., The Delphi Method IV: Effect of Percentile Feedback and Feed-In or Relevant Facts, RM-6118 PR, March 1970.
  10. Dalkey, N. (1975) Toward a Theory of Group Estimation, in the Delphi Method: Techniques and Applications, eds. H. A. Linstone and M. Turoff, Addison Wesley, page 236-261.
  11. Duval, A., Fontela, E., and Gabus, A. (1974) Cross Impact: A Handbook of Concepts and Applications, in Portraits of Complexity, Application of Systems Methodologies to Societal Problems, Battelle-Geneva, Geneva.
  12. Futures Group, (1994), Scenarios, UNU´s Millennium Project Feasibility Study.
  13. Godet, M. (1994) From Anticipation to Action: a Hand Book of Strategic Prospective, UNESCO Publishing, Paris.
  14. Goldstein, N., A Delphi on the future of the steel and Ferroalloy Industries, Part III, C3, in the Delphi Method, edited by Lindstone and Turoff, 1975.
  15. Harries, C. (2003) Correspondence to What, Coherence to What? What is Good Scenario-based Decision Making? Technological Forecasting and Social Change, 70, 8, 797-817.
  16. Hendela, A., Turoff, M., Hiltz, S. R. Cross Impact Security Analysis using the HACKING Game, accepted for ISCRAM 2010, Seattle Washington, May.
  17. Hopkins, R.H., K.B. Cambell, and N.S. Peterson, Representations of Perceived Relations among the Properties and Variables of a Complex System, IEEE Transactions on Systems, Man and Cybernetics, (SMC-17:1), January/February 1987, 52-60.
  18. Hsu, E. Y. P., Hiltz, S. R., and Turoff, M. (1992). Computer-Mediated Conferencing System as Applied to a Business Curriculum: A Research Update. In V. S. Jacob and H. Pirkul, eds., The Impact of Information Technology on Business Schools: Research, Teaching and Administration, Proceedings of the 20th Annual North American Conference of the International Business School Computer Users Group, pp. 214- 227. Awarded "Best Paper- Teaching."
  19. Kane, J., A Primer for a New Cross-Impact Language—KSIM (with Examples Shown from Transportation Policy), in the Delphi Method Book, editors Linstone and Turoff.
  20. Lendaris, G. G. (1980), "Structural Modeling - A Tutorial Guide," IEEE Transactions on Systems, Man, and Cybernetics SMC-10(12): 34.
  21. Plotnick, Linda, Elizabeth Avey Gomez, Connie White, Furthering Development of a Unified Emergency Scale Using Thurstone's Law of comparative Judgment: A progress Report, Proceedings of ISCRAM 2007, 4th International Conference on Information Systems for Crisis Response and Management, Delft, the Netherlands, May 13-16, Brussels University Press
  22. Turoff, Murray (2009), The Past, Present, and Future of Delphi, In: FUTURA, The Quarterly Journal of the Finnish Society for Futures Studies, ISSN 0785-5494 Helsinki, 32-44.
  23. Turoff, M., and S. R. H., Information Seeking Behavior and Viewpoints of Emergency Preparedness and management professionals concerned with Health and Medicine, Final Report for the National Library of Medicine, March 6, 2008, available at http://is.njit.edu/turoff .
  24. Turoff, M., S. R. Hiltz, X. Yao, Z. Li, Y. Wang, and H. K. Cho, Online Collaborative Learning Enhancement Through the Delphi Method, Turkish Online Journal of Distance Education-TOJDE April 2006 ISSN 1302-6488 Volume: 7 Number: 2 Article: 6, Publisher: Anadolu University, Eskisehir, Turkey, http://tojde.anadolu.edu.tr/index.htm
  25. Turoff, M., Hiltz, R., Bieber, M., Rana, A., Fjermestad, J., Collaborative Discourse Structures in Computer Mediated Group Communications, HICSS 32, 1999. Reprinted in Special Issue of Journal of Computer Mediated Communications on Persistent Conversation, Volume 4, Number 4, 1999, http://www.ascusc.org/jcmc/vol4/issue4/
  26. Turoff, Murray and S. R. Hiltz, (1995), Computer Based Delphi Processes, in Michael Adler and Erio Ziglio, editors., Gazing Into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health, London, Kingsley Publishers, pp. 56-88. Turoff, M., The Design of a Policy Delphi, Technological Forecasting and Social Change 2, No. 2 (1970), also in the Delphi Method book, 1975.
  27. Turoff, Murray, (1970) Delphi Conferencing: Computer Based Conferencing with Anonymity, Journal of Technological Forecasting and Social Change 3(2), 1970, 159-204.
  28. Turoff, M. (1991), Computer-mediated communication requirements for group support, Journal of Organizational Computing, vol. 1, number 1, pp 85-113.
  29. Warfield, J. N., Societal Systems, Wiley, New York, 1976.
  30. Warfield, J. N., Annotated Mathematical Bibliography for Interpretive Structural Modeling, May, 1992, http://www2.gmu.edu/depts/t-iasis/biblio/biw40.htm
  31. Weimer-Jehle, W. (2006) Cross-impact balances: A system-theoretical approach to cross impact analysis, Technological Forecasting and Social Change, 73, 334-336.
  32. Worrell, W., Hiltz, S. R., Turoff, M. and Fjermestad, J., (1995), An experiment in collaborative learning using a game and a computer-mediated conference in accounting games, Proceedings of the 28th Annual Hawaii International Conference on System Sciences, Vol. IV, pp. 63-71. Los Alamitos, CA: IEEE Computer Society Press, 1995.
  33. Yao, X., Turoff, M., and Chumer, M. J. (2009), Designing Collario for continuous reviewing and practicing of emergency plans to ensure complex system safety, Proceedings of the 6th Conference on Information Systems for Crisis Response and Management (ISCRAM'09), Gothenburg, Sweden.
  34. Yao, X., Turoff, M., and Hiltz, S. R., (2010), A field trial of a Collaborative Online Scenario Creation System for Emergency Management, accepted for ISCRAM 2010, Seattle, May, Iscram.org

See also

Delphi survey

Retrieved from "http://forwiki.eu/wiki/Definitional_aspects_of_the_Delphi_method"
Personal tools