This paper was presented at the British Educational Research Association conference in 2008. It examines the labels attached to particular research paradigms, and examines the historical development, construct and concept validity and potential tendency towards bias of ‘mixed method' research.
The aims of the resource:
The paper states that research which uses mixed methods is evolving into a dominant design in education research. The definitions of ‘mixed methods' which it cites all use "both quantitative and qualitative approaches on one or more of the levels of epistemology, methodology and methods" (p.4). The paper posits that ‘mixed methods' is now considered as one of three research paradigms, along with ‘qualitative' and ‘quantitative', but that mixed methods research has philosophical rather than empirical underpinnings, and it may act against its own principles by inhibiting the development of new research methods. Through examining the development and uses of these research paradigms, it aims to demonstrate that mixed methods research may weaken research studies and skew their outcomes, because researchers do not consider how and why the distinctions between the research paradigms exist, and that it may be time for researchers to drop the labels altogether and for a "rebirth of real-life research from the ashes of mixed methods" (p. 2).
Key findings or focus:
The authors of this paper argue that research labels are unnecessary and can be obstructive, since "no method of analysis is fixed to any one paradigm, thus the separation of these is artificial and does not support mixed methods" (p.9). However, a possible justification for mixed methods is given on page 4, which offers a philosophical, epistemological, practical and empirical argument in favour of the strength of mixed methods. The paper argues that it is the representation of the sample, the robustness and ecological validity which are the important factors.
Most of the paper argues against the purported strengths of mixed methods, arguing that mixed methods have "low construct validity...and can lead to bias against other real life forms of research" (p.5). It argues that both qualitative and quantitative paradigms can offer both objective and subjective viewpoints depending upon the type of data collected, the instrument used and the sample size, and that it is the way in which data is used which makes it essentially ‘qualitative' or ‘quantitative'. It considers that whilst many studies report using ‘mixed methods', they do not count since they are only considered ‘mixed methods' if the mixing of data happens in the actual research process. The paper also argues that mixed methods research is inherently biased since it "suggests that we should use one method to cover up the limitations of another, instead of addressing the weaknesses within the methods themselves" (p. 14). By applying labels and divisions to paradigms, we weaken the very argument that we seek to strengthen, since we do not question the assumptions related to each paradigm. The paper argues that mixed methods have a low validity in their current use and that we should forget about research ‘labels' and concentrate on improving the craft of research outside of these constraints.
The quality, authority and credibility of the resource from your subject perspective in relation to ITE:
Whilst it is interesting to read the arguments put forward about the inappropriate labelling of mixed methods in educational research, the paper is not convincing in the differentiation between ‘multiple methods' and ‘mixed methods'. Since it seeks to convince the reader that it would be more useful if they ignored all three paradigms (quantitative, qualitative and mixed methods), it is a little surprising to see the well-defined divisions between the quantitative and qualitative paradigms stated but not fully argued against, and this makes the conclusion that all three should be ignored less credible. The argument that mixed methods usually has low content validity and construct validity, as well as bias, are not always convincing, since there are many papers which support its use, and the examples used to argue against mixed methods primarily come from the authors of this paper. Nevertheless, it supports Gorard's view that "all methods have a role" (2005:162), and urges readers to give more consideration to the reasons why they employ particular research methods and their suitability to a particular purpose.
The implications for ITE tutors/mentors:
Research methods are currently part of most Masters level qualifications in education and since there is a Masters element to many ITE courses, this paper is very useful in offering an understanding of the ‘divisions' between research paradigms and the many ways in which we label research tools in educational practice. It also clearly shows the relationship between construct validity, content validity and bias, as well as highlighting the importance of reporting findings within the context which they are collected. The paper promotes the notion of ecological validity in educational research, which is something ITE tutors should encourage their students to consider when designing their own research studies.
The relevance to ITE students:
As detailed above, this paper will be useful for ITE students in helping them to understand some of the principles of research methods and leave them better informed to consider the relative strengths and weaknesses of research studies which they read and undertake. However, the content is relatively difficult to access and thus may require tutors to translate the key messages for this audience.
Reviewed by:
Dr Alison Daubney
