Focus groups is a research technique that collects data through group interaction on a topic determined by the researcher. In the research process, focus groups can be used at different points. Some use it at the preliminary or exploratory stage of a study, as a preparation for decisions about the precise objectives and design of the study. Others use it for the collection of the main data or in the closing period of a project to interpret findings or to generate further perspectives for research. Below we shall describe different types of focus groups, the elements of the research design and factors that could damage validity and reliability.

Focus groups in research

Morgan (1996: 130) defines focus groups as “a research technique that collects data through group interaction on a topic determined by the researcher”. The term “focus” refers to the topic that the researcher has in mind. He focuses the group discussion on such a topic through specific questions to be answered by the group. The group is often relatively small, Morgan (1997) suggests for most focus groups six to ten people. He also observed that the majority of the published articles about focus groups refer to the usage of this method along with other methods (Morgan 1996). Most frequently focus group research was combined with either individual in-depth interviews or with surveys. In the research process, focus groups can be used at different points. Some use it at the preliminary or exploratory stage of a study, as a preparation for decisions about the precise objectives and design of the study. Others use it for the collection of the main data. Often it is also used in the closing period of a project to interpret findings or to generate further perspectives for research.

Below we shall describe different types of focus groups, the elements of the research design and factors that could damage validity and reliability. Our description is based on literature, on our experience with doing an international focus group research project and an attempt to assess validity and reliability of our own project by comparing it to other focus group research projects in the same field (adult education). However, we will follow an inclusive approach and consider focus groups as a “set of central tendencies, with many useful variations that can be matched to a diversity of research purposes” Morgan (1996:132).

Typology of focus groups

There are two main types of focus groups:

lay groups and
expert panels.

Below we will make within these two main groups some further differentiation.

Focus groups as lay groups

Focus groups as lay groups, can be used as a data gathering method. Recently, focus groups, as a “management tool”, have been ‘largely associated with market research’ (Field 2000: 323). But marketing research is not the only application of this type of focus group research; despite the fact that in this area it is heavily represented (Greenbaum 2000). There are indeed many applications of the focus groups methodology in the field of applied sociology (Gamson 1992, Cable 1992, McKinlay 1993, Bobo et al. 1995 refereed to in Morgan 1996, Johnson 1996). You find also applications in communication studies (Staley 1990, Albrecht et al. 1993 refereed to in Morgan 1996), in education (Vaughn, Schumm and Sinagub 1996, Walker and Tedick 2001, Field 2000), in psychology (Vaughn, Schumm and Sinagub 1996) and public health (Kidd and Parhsall 2000).

Another alternative in this category, of focus groups as lay groups, is to employ focus groups as a tool in the development and evaluation of social programs (Morgan 1996). In this respect focus groups research can be designed as an instrument for empowering disadvantaged people or, like in emancipatory research, to “give a voice” to marginalised groups. For instance many feminist researchers therefore prefer this kind of focus groups (Wilkinson 1998). The primary purpose of this action-oriented approaches is “not to develop academic theories; nor is it to produce theoretical or empirical knowledge that can be applied in action, it is to liberate the human body, mind and spirit in the research for better, free world” (Reason and Bradbury 2001: 2).

Focus groups as expert panels

The second main category of focus groups is the expert panel. Participants of an expert panel are not just clients or “well informed” citizens, as in focus groups as lay groups, but professionals in a particular field of study (Bloor et al. 2001). It is not a kind of data gathering, but here researchers involve through expert panels practitioners in the scientific research (Emerson 1981, Kruger 1998).

The basic form of focus groups as expert panel is a co-operative inquiry (Heron 1996), based on participants’ professional experience, in the context of group interaction while searching for valid knowledge. These focus groups tend to orient toward explaining why a particular problem exists or how it develops (Ford 1977).

The second type of focus group as an expert panel is to employ it as an element in engaging experts in a policy project. The purpose of such a research design is to involve experts not only in finding explanations for a particular problem, but also to motivate them either to find solutions in their own practice or to collaborate in collective projects to solve the problem. Within such a design focus groups are at the same time a co-operative inquiry in a particular problem and a start for a collaborative action to formulate and if possible to implement solutions.

The third type of focus group as an expert panel is the use of it as a method to reach consensus among experts. It can be for producing social scientific knowledge as well as for developing policy action knowledge. A typical example is the Group Delphi method. The original Delphi method aims at reaching a consensus between experts on expected future developments by a carefully designed program of sequential individual interrogations, usually conducted by questionnaires, interspersed with information and opinion feedback (Helmer 1965, Linstone and Turoff 1975). Assuming an inclusive approach that treats most forms of interactive group interviews as variants of focus groups, expert panels employed within Delphi technique can be considered part of our typology. In this kind of Group Delphi approach the questionnaires are replaced or accompanied by a series of expert panels, which employ techniques of interactive focus groups discussion. An important intervention by the researchers is the in-between session that summarises on which points there already has been reached consensus and what remaining disagreements still have to be solved (Zolingen, van & Klaassen 2003).

Focus groups as qualitative research

Focus group research is qualitative research. There are three main epistemological traditions in qualitative research and all three apply to focus groups as well.

Firstly there is a radical hermeneutic position, a philosophical tradition, which develops insights about how people make sense of, or give meaning to, the world with the help of dialogue and discourse (Dilthey 1976). Habermas (1976, 1984) and Ricoeur (1979), strengthen Dilthey ’s distinction by noting that in the human sciences the subject of investigation and the investigator are interlinked in a communicative way. Empirical research is ultimately a discourse on the meaning of action.

Secondly, beyond hermeneutics we find a tradition that on the one hand still departs from the idea that the purpose of research is to find the truth about an objective reality, but on the other hand argues that the standards for such a truth-finding process in quantitative research do not fit with qualitative research. Lincoln and Guba (1985) proposed concepts as trustworthiness, credibility, transferability, dependability and confirmability to discuss the truth-value of their findings. Even when qualitative researchers agree on objectivity as an ultimate norm, many stress that in fact we can only hope for inter-subjectivity. But again, there are different perspectives on the concept of methodological inter-subjectivity (Smaling 1992). The perspective of consensus is the most evident one, and it can be recognised in different methodological ideas: inter-subjective verifiability, conformability, testability, repeatability, reliability, or reproducibility; inter-observer agreement, reliability, univocally. Two other traditional perspectives are inter-subjectivity by regimentation and by explicitness. Those two are more focused on the process of doing research. Regimentation refers to regulation needed in order to assure the virtual repeatability, controllability, correctability and criticizability. This leads almost inevitably to the second traditional requirement of being as explicit as possible about the design, specific procedures, and interpretation strategy.

A third tradition in qualitative research holds that there is not such a strict division between quantitative and qualitative research. Within this perspective qualitative researchers apply the central concepts of quantitative research methodology adapting definitions and more concrete standards to the specific characteristics of qualitative research. Kirk and Miller (1986: 19) states that qualitative research can be evaluated in terms of objectivity and moreover, ”the partitioning of objectivity into two components: validity and reliability” is adequate for qualitative research. For Hammersley (1987) the research results are valid or true if they represent accurately those features of the phenomena, which it is intended to describe, explain or theorise. Reliability is the extent in which a measurement procedure produces the same answer however and whenever it is carried out. Silverman (2001) adopts this perspective in his plea for the claims to validity or truth status of qualitative research studies. A meta-analysis of the key words related to validity and reliability has been done by Winter (2000). This analysis revealed that the aggregated definition of validity could be that of accuracy, and the definition of reliability that of replicability. What is important in searching for understanding of nature of validity and truth is to enquire whether the research is measuring what it was intended to measure.

An example of this third, rather pragmatic approach to validity is Maxwell’s attempt (1992) to develop a typology of validity categories, nevertheless it is important to mention that he avoided just applying or adapting the definitions developed for quantitative research. According to Maxwell validity in qualitative research refers primarily to accounts and not to data or methods. This is consistent with the position expressed by Hammersley and Atkinson: “data in themselves cannot be valid or invalid; what is at issue are the inferences drawn from them” (Hammersley and Atkinson 1983 referred to in Maxwell 1992: 283). Three of the five types of validity Maxwell mentions in his typology of the kinds of understanding are directly connected with the research process itself and will be used below to assess the validity of focus groups method:

Descriptive validity refers to the factual accuracy of the account. An important question here is “Was there a problem of miss-transcription or miss-remembrance?” This type of validity can refer to issues of omission as well as of commission.

Interpretative validity refers also to aspects of accounts; it is the fact that these accounts must be grounded in the language of the participants and relies as much as possible on their own words and concepts. It is a matter of inference from the words and actions of participants to the accounts constructed by researchers.

Theoretical validity refers to the accounts’ function as an explanation and more exactly to the inference from data to report. Also, it applies to the postulated relationships among the concepts. Therefore it is also called internal or concept validity.

While validity refers primarily to the accounts, reliability refers primarily to the data and the method. How can we guarantee that a replication of the method leads to the same results? In quantitative research the solution is sought in a strict regimentation (representative sample, closed questions). In qualitative in-depth research the problem is much more to optimise the chance that all relevant data will again show up in the replication. The method should be designed in such a way that it delivers the greatest variety of data. It is a not matter of limiting data, as in quantitative research, but of striving for completeness.

In the next section, on the different elements of focus group research, we will develop a rather pragmatic approach with respect to these epistemological questions. In developing standards for good focus group research, we will apply mainly concepts of validity and reliability as defined in the realistic, third tradition. Nevertheless we found it helpful to apply occasionally also methodological standards developed in the second tradition. But when it comes to moderating and reporting discussions in the focus group we acknowledge that at the end focus group research is, as the hermeneutic tradition underlines, also a discourse on the meaning of action.

Elements of focus group research

In order to judge whether a (specific) focus group research design is valid and reliable, we have to look now to the separate elements of the focus group method. For each element we shall consider the question how it influences the validity and reliability.

Examples of expert panels of adult educators

We developed the methodological framework above while preparing and assessing our own focus group research project, which was part of a bigger research program called Education and Training for Governance and Active Citizenship in Europe (Etgace). The data for this project are collected in six European countries: Finland, the United Kingdom, the Netherlands, Belgium, Slovenia and Spain.

The Etgace research had four phases.

It started with a critical review of the relevant literature in the six countries and in Europe as a whole. This part of the project has been reported in Holford & Edirisingha (2000).

In the second phase 96 life histories of active citizens, 16 in each participating country, has been analysed. The results have been reported in Snick, Stroobants & Wildemeersch (2001).

In the third phase, focus groups of local experts in each country discussed forms of education and training of active citizens. So, focus groups in our project were typically expert panels. The focus group results are reported in Van der Veeen and Van Raak (2001)

In the final phase we disseminated the results trough both scientific publications and various activities for workers in the field of education and training in each of the six countries. In this final phases we engaged again our experts, who also had been members of the focus groups.

The focus groups in the Etgace project had three main functions. Firstly to provide evidence about current good practice in citizenship and governance education. Secondly to help to transform the results of the earlier phase of the research, the life histories with active citizens, into effective intervention strategies. Thirdly, to form relationships with professionals which can be useful in the next, fourth phase of dissemination of our findings. So focus groups were not just an interesting research activity in itself, but it was also helpful in building a relation with both the earlier and the later phase of the project as a whole.

Mostly focus group just last for about two hours or so. In order to get more reliable results, i.e. a representation as complete as possible of all relevant arguments and facts, we wanted experts to spend more time together. That’s why we embarked on a rather ambitious project to bring, within each country, national experts together for two full days (six effective working hours each day) with an intermission period of a couple of weeks. Also, research questions were divided over separate discussion sessions during these two days. Ironically, after these two days experts had still the feeling that there was a lack of time for a full discussion of all issues!

After completing the focus groups sessions in each country, the Etgace international team discussed their experiences and it turned out that in many ways we had not succeeded in carrying out the project in the perfect way that the handbooks on focus groups expect from us. We had also the feeling that this was not just our failure, but that there are some fundamental weaknesses in the method, which are hardly discussed in the methodological literature.

We decided to compare our experience with that in another big European project in the field of adult education, called Balancing Competencies and carried out from 1998 till 2000, that used focus groups as one of its research strategies. The overall objective of this project was to generate a more comprehensive understanding of the potentialities and limitations of existing approaches to post school education and training for unemployed youth and young adults (Weil 2000). The countries involved in this project were Belgium, Denmark, Germany, Portugal, The Netherlands, and the United Kingdom. The Balancing Competencies project used in each country two types of focus groups. One type was typically an expert panel of professionals in the field. The other type of focus group in the Balancing Competencies project was typically emancipatory research with young adults. We collected data on the first type of focus groups in this project, the expert panels, in order to get comparable data with Etgace.

We were also able to collect some comparable data for two other international projects in the field of adult education, which were based on the Delphi research methodology, combining questionnaires with group sessions designed as colloquies. (Leirman 1995, Aguettaz 2000). In the Eurodelphi project (1993-1995) the main objective was to compare different models for adult education in Europe. Sixteen countries were involved, but we concentrated for very practical reasons on the experience with focus groups in two countries. The fourth and last project is Saeda (1998-2000). The main objective of Saeda was to identify the strategies for the improvement of adult education in bicultural regional settings in Europe. Four regions participated in this project, from France, Greece, Italy, and Spain.

Below you will find some results of this comparison of four projects. Firstly, we mention some elements in the research design of focus groups that are problematic in the sense that validity and/or reliability is endangered. Secondly, in our concluding section, we present what our spokesmen think are more in general strong and weak points of focus group research. More details about the assessment of focus groups by co-ordinators and moderators in these four projects can be found in Chioncel et al. (2003)

Vulnerable elements

You may make a detailed plan for focus group research, but in practice participants are sometimes unwilling to do what is requested. In the Etgace project the co-ordinating staff developed a precise time schedule for the two daylong meetings of the focus groups in each country. Nevertheless three out of six countries made some changes. The most important changes refer to the unwillingness of focus groups in these three countries to split up in gender homogenous subgroups to discuss gender questions in active citizenship. This could have damaged the theoretical validity of the research.

The data on participation of experts in all projects demonstrate two crucial questions. Firstly, many experts can or will not accept an invitation. Secondly, also experts that accepted the invitation in first instance in fact did not attend the focus group meetings. There are differences between programs and countries, but the overall figures of the huge Eurodelphi Project (16 countries) give a good indication of what you may expect. From the respondents in the questionnaire round, 65% accept the invitation for colloquia (the focus groups), but only 55% from the experts who accepted an invitation did attend indeed. There is also a danger of a serious bias, because some types of experts have a greater tendency to be absent. In the four projects we compared, particularly experts working in the profit sector and experts working as high officials for the government were most often absent.

There has been a discussion in all projects how much information should be presented to participants to focus the discussion on the research questions, which can enhance the theoretical validity and reliability of the discussion. In the Etgace project there have been sent in advance about 15 pages or more, up to 50 pages in one country (including a national report of life histories of active citizens produced in an early phase of the research); and also a report on the discussion during the first meeting, between the two focus groups sessions. National co-ordinators indicated that they have the impression that many participants did not read the documents thoroughly and some did not read them at all. During the discussions the experts hardly referred to the documents. Etgace also took care of introductions on the subject at the beginning of each meeting and/or short introductions at the beginning of each separate session. The Balancing Competencies co-ordinators mentioned that they need a lot of time during the meeting to inform professionals in order to have a focused discussion. As said, Eurodelphi and Saeda used a mix of questionnaires and focus groups. One respondent makes the remark that “it is not effective to give participants more than four pages information in advance (with summary of questionnaire results)”. So, the general conclusion of the focus group co-ordinators seems to be, that in order to focus the discussion, you may send some written information in advance, but it is more effective to rely on introductions during the focus group meeting itself, even when this takes a lot of time.

With respect to the question whether the focus groups were reliable in the sense that discussions were open and critical we have sent a questionnaire to all moderators in the Etgace project (response rate 40.7 %) and the Balancing Competencies project (response rate 83 %). Most group moderators (94.4 %) indicated that there was an animated and positive atmosphere, but these moderators are more cautious in their answer on the question whether participants were critical about each other’s ideas (only 63 % agree). Other items in the questionnaire for the moderators referred particularly to their own role. In general, moderators interfered to complete specific tasks during the meeting (87.1%), they tried to get everyone to participate equally in the discussions (96.3%), encourage those experts who might otherwise say little (70 %) and try to limit those experts who dominate the discussions (77.1%). Looking closer, these answers do not say much about what really happened. How often did moderators intervene? How successful were they? Only a precise observation can disclose what really happened.

With respect to the recording of the discussions in the focus groups, we formulated above what seems a conceivable hypothesis that for securing descriptive and interpretative validity anything less than an audio tape, which is subsequently transcribed literally, is in fact unacceptable. Our data indicate that in practice researchers hold a much more differentiating view. In the Etgace project only three out of six countries taped the discussion. These three countries used the tape in different ways. One country made literal transcripts of the tape, one country used the tape to make detailed minutes, and one country made a preliminary report on the base of the minutes and subsequently checked it scrupulously by listening to the tapes. Two of the three countries that made an audio tape, reported technical problems leading to a loss of part of the discussion, which is an extra argument to make always detailed minutes. In the Balancing Competencies project five out of six countries have tapes of the discussion and also five countries made transcripts. In generally, in the Eurodelphi and Saeda projects both minutes and transcriptions of tapes were made for significant colloquies.

In writing the report there is an enormous tension between on the one hand descriptive validity (precise transcripts) and interpretative validity (precise coding) and on the other hand theoretical validity. For the quality of the report the real problem is not producing precise transcriptions and codes. That is a not very difficult although time-consuming activity. But the real challenge is the development of a theoretical framework that makes sense of all these data. What researchers need in the production of the report has been described by co-ordinators as “interpretative imagination”, “contextualising the group” and ”locating the focus group in the wider context of research”. Or, to put in our own words, a practice of just summarising the results of a focus group does not make much sense. It is the competence of the researcher to understand the results in a broader context that makes the difference between a correct and an excellent report.

The dialogue between researchers and experts in the panels, important in a hermeneutic perspective on focus group research, was organised in two ways. Firstly all four projects had a second meeting of each focus group that was organised partly or fully as feedback on the results of the first meeting. All researchers experienced this as fruitful. Secondly researchers sent their draft reports to all participants for comments. Participants reacted rarely, so this made hardly sense. As with the preparation of the focus groups, we see here reluctance in participants to read much, either before or after the group meeting(s). Whatever you want to do to involve partners should mostly happen in the focus group meeting(s) itself.

Conclusion: strong and weak points of focus groups

We also asked the co-ordinators of all national projects within Etgace and Balancing Competencies to mention three strong and three weak points of the focus group method. We combined the answers in some broad categories that are reported in the two tables.

TABLE 1

TABLE 2

In judging the popularity but also effectiveness of focus groups it is important to assess the advantages of the method as well as its limitations. In adult education focus groups offer maybe a more concrete opportunity for participants and researchers to learn from the process (Field 2000), being a powerful way of engaging with professionals, policy makers and end-users. The benefits of using such an iterative process in which “people’s views and understandings are shared, debated, challenged and changed” (Field 2000: 324) are more important than the potential disadvantages. Nevertheless, further research into focus groups methodology is needed to unravel the importance and dynamics of all presented elements. There is a need for in-depth interviews with invited lay people or experts about their motives to participate or not. There is also an urgent need for a more close observation of the group dynamics in focus groups.
Chionce, N.E.& van der Veen R.G.W. 2001. Introduction to Focus Groups. http://www.metodix.com. Menetelmäartikkelit

References

Agar, M. and MacDonald, J., 1995, Focus groups and ethnography, Human Organization, 54, 78-86.

Aguettaz, P., 2000, Stratégies d’Amélioration de l’Education des adultes dans des contexts régionaux (Aoste).

Aldag, R.J. and Fuller S.R., 1993, Beyond fiasco: a reappraisal of the groupthink phenomenon and a new model of group decision process. Psychological Bulletin, 113, 533-552.

Bloor, M., Frankland, J., Thomas, M. and Robson, K., 2001, Focus group in social research (London, Sage).

Chioncel, N.E., Veen, R., van, R., Wildemeersch, D. and Jarvis, P. (2003) Validity and Reliability of Focus Groups as a Research Methods in Adult Education. International Journal of Lifelong Education, 22, in press.

Dilthey, W., 1976, Selected Writings. (Cambridge: Cambridge University Press).

Emerson, R., 1981, Observational fieldwork. Annual Review of Sociology, 7, 315-378.

Field J. (2000) Researching Lifelong Learning Through Focus Groups. Journal of Further and Higher Education, 24, 324- 335.

Flowers, M.L., 1977, A Laboratory test of some implications of Janis’s groupthink hypothesis, Journal of Personality and Social Psychology, 35, 888-896.

Ford, T.R., 1977, The Production of Social Knowledge for Public Use. Social Forces, 56, 504-518.

Greenbaum, L.T., 2000, Moderating focus group. A practical guide for group facilitation (London: Sage).

Habermas, J., 1976, Some Distinctions in Universal Pragmatics, Theory and Society, 3, 155-167.

Habermas, J., 1984, The Theory of Communicative Action (Boston: Bacon).

Hammersley, M., 1987, Some notes on the terms ‘validity’ and ‘reliability’. British Educational Research Journal, 13, 73-81.

Helmer, O., 1965, Analysis of the future: the Delphi method (Santa Monica: Rand Corporation).

Heron, J., 1996, Co-operative inquiry. Research into the human condition (London: Sage).

Holford, J., and Edirisingha ,P. (eds) (2000) Citizenship and Governance Education in Europe: A Critical Review of Literature: Guildford: University of Surrey.

Johnson, A., 1996, It’s good to talk’: The focus group and the sociological imagination, Sociological Review, 44, 517-538.

Johnson D.W. and Johnson, F.P., 2000, Joining together – Group theory and group skills (London: Allyn & Bacon).

Kidd, P.S. Parhsall M.B., 2000, Getting the focus and the group: enhancing analytical rigour in focus group research. Qualitative Health research, 10, 293-308.

Kirk, J. and Miller, M.L., 1986, Reliability and validity in qualitative research (Newbury Park: Sage).

Kruger, R.A., 1998, Analysing and reporting focus group results (London: Sage).

Leirman, W., 1995, Euro-Delphi a comparative study on the future of adult education in 14 countries between 1993 and 1995. International Journal of Lifelong Education, 15, 125-138.

Lincoln, Y.S. and Guba, E.G., 1985, Naturalistic Inquiry (Newbury Park: Sage).

Linstone, H. and Turoff M., 1975, The Delphi Method, Techniques and Applications (London: Addison-Wesley).

Maxwell, J.A., 1992, Understanding and validity in Qualitative Research. Harvard Educational Review, 62, 279- 300.

McCauley, C., 1989, The nature of social influence in groupthink: compliance and internalisation. Journal of Personality and Social Psychology, 57, 250-260.

Morgan, D.L., 1996, Focus groups. Annual Review of Sociology, 22, 129-152.

Morgan, D.L., 1997, Focus groups as qualitative research (London: Sage).

Morgan, D.L., 1998, The focus groups guidebook (London: Sage).

Reason, P. and Bradbury, H., 2001, Inquary and Participation in Search of a World Worthy of Human Aspiration. In P. Reason, and H. Bradbury (eds), Handbook of Action Research: Participative Inquiry and Practice (London: Sage), pp. 1-14.

Ricoeur, P., 1979, Main Trends in Philosophy. (New York: Holmes and Meier).

Silverman, D., 2001, Doing Qualitative research – A Practical Handbook (London: Sage).

Smaling, A., 1992, Varieties of methodological intersubjectivity – the relations with qualitative and quantitative research, and with objectivity. Quality and Quantity, 26, 169-180.

Snick, A, Stroobants, V, and Wildemeersch D. (eds) (2001), Learning Citizenship and Governance in Europe: Analysis of Life Histories. (Catholic University of Leuven: Leuven).

Tetlock, P.E., 1979, Identifying victims of groupthink from public statements of decision makers. Journal of Personality and Social Psychology, 37, 1314-1324.

Vaughn, S., Schumm, J.S., and Sinagub, J., 1996, Focus group interviews in education and psychology (London, New Dehli: Sage).

Veen, R., van and Raak, J., van (eds), 2001 Focus Groups: Intervention Strategies for Citizenship and Governance – Analysis of Focus groups (Katholieke Universiteit Nijmegen: Nijmegen).

Walker, C.L and Tedick D.J. 2000, The complexity of immersion education: teachers address the issues. Modern Language Journal, 84, 5-27.

Weil, S., 2000, Twinning theoretical and methodological concerns: an emerging paradigm from (youth) research (UP: Northampton).

Wilkinson, S., 1998, Focus groups in feminist research: Power, interaction, and the construction of meaning. Women Studies International Forum, 21, 111-125.

Winter, G., 2000, A comparative discussion of the notion of validity. Qualitative and Quantitative Research, 4, available on line http://www.nova.edu/ssss/QR/QR4-3/winter.html.

Zolingen, S., van, 1995, Gevraagd: Sleutelkwalificatie: een studie naar sleutelkwalificaties voor het middelbaar beroepsonderwijs (KUN: Nijmegen).

Zolingen, S., van, Klaassen, C.A., 2003, Selection process in a Delphi study about key qualifications in Senior Secondary Vocational Education, in Technological forecasting and social change, 70, 317-340.

Kategoriat:artikkeli, Artikkelit