Etusivu › artikkeli › 4. Delphi Panel

4. Delphi Panel

Kirjoittanut , • ( 0 )

Toni Stubin, Jukka Tikkanen & Hannu Linturi (updated 29.2.2024)

”Socrates aimed to demonstrate that although people present themselves as experts who know all sorts of things and persuade others to accept their views, their claims are poorly substantiated because they are in contradiction with their other beliefs and thus irrational to accept. Socrates declared himself an opponent of the sophists and the teachers of rhetoric. He accused them of focusing solely on winning others over to their side, even though what is essential is whether the things they are convincing about are true.” (Sihvola 1997)

Delphi is characteristically an expert method. Its panel, named for the purpose, is made up of experts who control the phenomenon under study from different directions. They are brought into interaction with both the themes of the subject under investigation and with each other in a manner where the emphasis is on factual arguments instead of the respondents’ status. Since the panels are anonymous, there is no need to consider anything other than the matter itself in the discussion and presentation of opinions. Titles, positions, and other power relations lose their significance, allowing for an o§pen-minded focus on future questions.

The Delphi method is based on the idea that the assessments of a structured group about future developments are more precise than those of an unstructured group. The assumption is that the selected group of experts, or panel, knows more about the future of their specialty than ordinary people do, and that they are also willing to present their best knowledge in the survey. An expert, in a nutshell, is defined as a person who can make better evaluations and forecasts than a non-expert. Experts should be selected so that together they represent a diverse range of practices within the subject area being studied.

According to Kuusi (1999), a Delphi-qualified expert should:

  • be at the forefront of their field of knowledge
  • be interested in different fields of knowledge
  • be able to see connections between national and international, current and future developments
  • be capable of looking at problems from an unconventional perspective
  • be interested in creating something new. This description reflects a modern understanding of expertise and can be considered a description of the panel’s qualities as well.

This description reflects a modern conception of expertise, and it can also be considered as a description of the entire panel’s characteristics.

To ensure a plurality of voices, it is good to also select experts from fields related to the subject under study. Expertise can be a trap if it acts as a ’recycler of past knowledge.’ In that case, the panel’s thoughts on the future are too clearly mirrored against the current situation and the maintenance of the status quo, when future-oriented research should aim to bring out different and even surprising opinions. But most often, the best knowers and doers in their field are ahead of others in penetrating the future because of their understanding.

The success of Delphi research critically depends on the right composition of the expert panel. Therefore, time and effort should be spent on planning the composition of the panel. The clear goal of assembling the panel is to create a composition that has a diverse range of expertise related to the subject area and panelists who can and dare to express their opinions.

Good panelist qualities include:

  • The ability to discuss, interact, and be open to alternative views
  • Foresight, imagination, and creativity
  • The ability to see patterns where others see random elements
  • Willingness to embrace the surprising future possibilities offered by Delphi.

Hines, Gary, Daheim, and Van Der Laan (2017) have identified a total of six foresight competencies. These include the ability to see signs of change and possible futures, as well as the ability to perceive likely futures based on current trends and, alternatively, those challenging them. All these are qualities that can be associated with a good panelist.

How many experts should be recruited to the panel? Delphi panel sizes can vary from a small group of experts to thousands of respondents. Typically, about 20-100 expert panelists participate in Delphi research (Kuusi 2000). When the focus is on argumentation and qualitative data, the number of panelists is smaller. Delphi studies emphasizing qualitative data have been successfully conducted with even fewer than twenty panelists.

In technology forecasting studies, large groups of experts are favored, and in societal contentious issues, small groups. Clearly, in large groups, there is no room for dialogue; instead, the focus is on collecting the voting result of the large panel and also the justifications for different positions. In a small group, there is a completely different opportunity to concentrate on the Delphi process, also allowing participants to become acquainted with other panelists’ thoughts and build dialogues on that basis. At its best, the expert group may evolve into a learning community, where positions and justifications live through interaction. This Delphi blog series emphasizes the communicative and learning Delphi technique of an expert group, often referred to as argumentative Delphi, in terms used by Osmo Kuusi.

Through the Delphi process, existential narrative knowledge accumulates, for example, in the form of scenarios. Two other terms used by Bakhtin, ’polyphony’ and ’dialogism,’ are also interesting in terms of expertise. They relate static skills to the process capabilities of knowledge production. By polyphony, Bakhtin means the diversity and independence of voices, speakers, and opinions relative to each other. In the wisdom of crowds, the terms diversity and independence address the same issue. (Surowiecki 2004)

Using the jazz metaphor, in the ideal panel, each member has the ability to perform solo. The number of players cannot be too large for enjoyable ensemble playing to occur; when a soloist can also step back to support other players’ instruments. At the heart of dialogism is the ability to listen actively and inspiringly. Such ensemble playing is unfortunately rare in Delphi studies.

As a learning community, the achievements and creativity of the Delphi panel depend on more than just the ability to know a lot and in-depth. Alongside factual knowledge, the panel needs systemic knowledge (Bakhtin’s narrative knowledge), requiring a generalist expertise. The importance of dialogic and communication skills increases as the opportunities for direct interaction between panelists grow. This includes the ability to ’irritate’ others and provoke panelists to discuss and justify their opinions.

The lessons of networks and the wisdom of crowds create pressure to dismantle collegial expert institutions that standardize the behavior of their members. Ivan Illich (1973) warned of such an over-institutionalization of society a couple of generations ago. The anonymity of Delphi helps, in part, to ensure that important intuitive knowledge can be included in creative processes.

A diverse panel has good prerequisites for solving problems related to the phenomenon under investigation, especially if the Delphi rounds produce sufficient alternatives, justifications for the alternatives, and solution models. Applying the decentralization idea of the wisdom of crowds (Surowiecki 2004), panels should reserve places for all types of experts: experts, specialists, generalists, and laypeople.”

Panel Building

According to the well-known definition by Linstone & Turoff: ’The Delphi technique can be characterized as a method for structuring a group communication process, which aims to help a group formed by individuals as a whole deal with a complex problem.’ This learning-capable group formed by individuals is not chosen at random, but through a dual-criteria selection process.

In the Delphi method, it is essential to recruit a panel so that it covers the essential expertises (knowledge or cognitive abilities) and stakeholder interests (interests or social abilities) relevant to the phenomenon under study and its future, in order for the argumentation of theses and themes to be diverse and comprehensive.

The previous part of the blog series discussed the characteristics of the Delphi method and the role of experts as well. When selecting a panel, the aim is to assemble as knowledgeable an expert group as possible about the subject being studied. In addition, it is often good to have people in the panel who look at the big picture, i.e., see the forest for the trees.

When choosing panelists, a division into so-called internal and external panels can also be considered. The internal panel includes experts directly related to the future under study, while the external panel might include, for example, representatives of stakeholder groups. A multi-round Delphi can also be structured so that the first round is conducted by the internal panel, and the external panel is included in the subsequent rounds. Thus, material from the first round is provided for the external panel to process, which brings new perspectives and may challenge the experts’ opinions in an interesting way.

For example, in the Future of Learning Barometer, the internal panel invited experts involved in learning, teaching, and education in different ways, while the external panel sought people from different sectors and positions of society so that interests and knowledge related to learning, teaching, and schooling were diversely considered. (Linturi & Rubin 2011).

An important step in forming a Delphi panel is building an expertise matrix according to the different interests and competencies of the panelists to be invited. In this, each panelist is placed in a two-dimensional table where their expertise (competence) and their background group (interest) meet. The matrix is an essential tool to ensure that the panel composition is diverse and different perspectives are taken into account as accurately as possible. If experts seem to cluster on one side of the matrix or if there are several empty spaces, it may be necessary to reconsider the panel composition or the chosen interest-competence combinations. However, perfect balance is not an absolute requirement, and in some cases, the research may want to emphasize a certain area more. There can also be empty cells in the panelist matrix. Sometimes this is a ’logical necessity.’ Even then, the expertise matrix provides a view on whether the panel has been well assembled.

Environmental Panel Expert Matrix (Petri Tapio Delphi Workshop, April 15, 2016). Instead of the term ’stakeholder interest,’ this uses the criterion of social expertise or status. The rows of the matrix describe the subject of expertise, and the columns depict the stakeholder status.

Recruiting Panelists

Recruiting panelists is most effective when the contact is personal. Panelists who meet the criteria can be contacted via telephone, email, or different social media platforms. A potential panelist usually wants to know why their particular expertise is needed in the research. Participation in a recognized expert panel, the opportunity to argue and discuss, and the personal significance of the research topic are typically motivating factors to consider during recruitment. It is also good to emphasize the specific expertise of the panelist being recruited. Additionally, promising that the research results, report, or publication will be provided to the panelist after the study ends can influence participation.

When considering the panel as a community intended to produce considered thoughts about the future, it is important that panelists find participation beneficial for themselves. More important than the number of participants in the expert panel is that the panelists are committed to their task and are able to discuss the subject under study in a thought-provoking manner.

A pre-interview is an excellent way to involve panelists already in the planning phase. It increases their interest and makes the topic more relevant to them. Through interviews, insights are also gained into the future statements of the actual Delphi study, and in some cases, these can form the first round of the Delphi. This is the most recommended method for recruiting and engaging panelists. If it is not possible to interview panelists in advance, personal invitations should be used in their recruitment, highlighting the purpose, significance, and benefits of the panel to the participant. Experts’ time is limited, and participation is more motivating when the goals and benefits are clear. Especially in argumentative Delphi and Policy Delphi surveys, where consensus is not the initial aim, the success of the panel is almost imperative, and agreement on participation, contents, and objectives must be made separately with each panelist. Otherwise, a committed and conversational panel cannot be expected.

By selecting a committed, diverse, and knowledgeable group of panelists, a solid foundation is built for the success of the Delphi study. However, this is just the initial phase of the research. The following posts in the blog series will discuss the role and tasks of the Delphi manager during the survey and the processing and analysis of the results produced by the panel.

Literature

‹
›

Kategoriat:artikkeli, Artikkelit, blogi, Tie

Avainsanat:, , , ,

Jätä kommentti

This site uses Akismet to reduce spam. Learn how your comment data is processed.