Advantages of Employing Mixed Methods Research in Impact Evaluation
1. Introduction
Research methods mainly involve use of both qualitative and quantitative research methods. Qualitative research methods are mainly based on the views of participants which will be subjected to analytic induction such as establishing common themes from participants’ views. Examples include interviews, focus group discussions, open-ended questionnaires, and observations. On the other hand, quantitative methods generally involve collecting numerical data that can be subjected to statistical analysis. For instance, questionnaires (with closed-ended questions or open-ended transferable to quantitative data), personality tests, performance tests, etc. There has been an increasing trend in research to systematically combine both qualitative and quantitative methods within a single research study or evaluation for the advantages discussed in this blog article.
2. Understanding Mixed Methods Research
Mixed methods research is the combination and integration of qualitative and quantitative methods in the same study for the broad purposes of breadth and depth of understanding and corroboration (Cresswell, 2012). A design might be considered mixed if it employs qualitative and quantitative approaches at any stage, including research questions development, sampling strategies, data collection approaches, data analysis methods, or conclusions (Cresswell & Garrett, 2008). According to Mason (2006), mixing methods offers enormous potential for generating new ways of understanding the complexities and contexts of social experience, and for enhancing our capacities for social explanation and generalization.
2.1 Advantages
The advantages of employing mixed research methods in impact evaluation are as follows:
- Synergy: Quantitative methods achieve objectivity whilst qualitative methods provides the explanation to a given research phenomenon. The basic premise is that integration of quantitative and qualitative approaches permits a more complete and synergistic utilization of data in providing a better understanding of research problems and complex phenomena than either approach alone (Fetters & Freshwater, 2015). Better understanding can be obtained by triangulating one set of results with another and thereby enhancing the validity of inferences.
- Complementarity: The evaluators can elaborate, clarify, or validate the results from one method with the findings from the other method. Findings from qualitative and quantitative data sources can be compared after collecting both types of data simultaneously. The data can be analysed, and results compared through side-by-side discussions, or transforming the qualitative data set into quantitative scores, or jointly displaying both forms of data. Quantitative and qualitative data can validate each other and create a solid foundation for drawing conclusions on evaluations.
- Development: The results from one method can be used to help develop the use of the other method. This explanatory sequential design typically involves two phases: (1) an initial quantitative instrument phase, followed by (2) a qualitative data collection phase, in which the qualitative phase builds directly on the results from the quantitative phase. In this way, the quantitative results are explained in more detail through the qualitative data.
- Expansion: The evaluators can seek to extend the breadth and range of inquiry by using different methods for different inquiry components. For instance, use of qualitative data to augment a quantitative outcomes study. Within an outcomes study, the researcher collects and analyses both quantitative and qualitative data. The qualitative data can be incorporated into the study at the outset to help design the intervention; during the intervention to explore how participants experience a certain programme intervention; and after the intervention to help explain the results (Palinkas, et al., 2011).
- Participatory: Mixed methods also mirror the way individuals naturally collect information by integrating quantitative and qualitative data. This stakeholder participatory approach in the research process brings about change by providing input about their needs, ways to address them, and ways to implement changes (Mertens, 2009). A reflection of participants’ point of view in mixed methods give a voice to study participants and ensure that study findings are grounded in participants’ experiences.
2.2 Limitations
Conducting mixed methods research is not easy and involves several barriers (Creswell & Plano Clark, 2011). This is because they require more work and financial resources, and they take more time in implementing the quantitative and qualitative parts of the study. In addition, researchers need to develop a broader set of skills that span both the quantitative and the qualitative descriptions which is normally difficult. However, the broader skill requirement can help motivate the evaluator/researcher to widen and extend their repertoire of methods (Mertens, et al., 2020).
3. Designing Mixed Methods Research
To determine the best mixed methods design, it is important to consider the priority and implementation of data collection. On priority, the researcher can decide to give equal priority to both quantitative and qualitative parts, emphasize qualitative more, or emphasize quantitative more. This depends on research questions, practical constraints on data collection, or the need to understand one form of data before proceeding to the next. Implementation looks at the sequence the researcher uses to collect both quantitative and qualitative data which could be either a concurrent or sequential design. In concurrent design, the researcher may seek to compare both quantitative and qualitative data to search for congruent findings. In sequential design, qualitative data collection may precede quantitative data collection when the intention is to first explore the problem being studied and then follow-up with quantitative data that are amenable to studying a large sample, so that the results can be applied to a population. Alternatively, quantitative data may precede qualitative data when the intention is to test variables with a large sample and then to explore more in-depth with a few cases during the qualitative phase.
The core characteristics of a well-designed mixed methods research study should include the following:
- Collecting and analysing both quantitative (closed-ended) and qualitative (open-ended) data.
- Using rigorous procedures in collecting and analysing data appropriate to each method’s tradition, such as ensuring the appropriate sample size for quantitative and qualitative analysis.
- Integrating the data during data collection, analysis, or discussion.
- Using procedures that implement qualitative and quantitative components either concurrently or sequentially, with the same sample or with different samples.
- Framing the procedures within philosophical/theoretical models of research, such as within a social constructionist model that seeks to understand multiple perspectives on a single issue.
4. Conclusion
The use of mixed methods research in impact evaluation has shown that this methodology provides a broader spectrum of ways to better understand complex research problems in different contexts than could be done through either quantitative or qualitative approaches alone. It is also important to note that the variety of skills required to effectively execute this methodology can researchers are motivated to increase the rigor of conceptual thinking, see new ways to answer research questions, and identify other hidden aspects when trying to solve some complex research problem.
Author: This blog article was produced by Earnest Manjengwa, a Senior Researcher at Underhill Corporate Solutions. He can be contacted at earnestm@underhillsolutions.co.za | +27 12 751 3237 | +27 73 818 3487.
REFERENCES
Cresswell, J. W., 2012. Educational research: Planning, conducting, and evaluating quantitative and qualitative research. 4th ed. Boston, MA: Pearson.
Cresswell, J. W. & Garrett, A. L., 2008. The “movement” of mixed methods research and the role of educators. South African Journal of Education,, 28(3), pp. 321-333.
Creswell, J. W. & Plano Clark, V. L., 2011. Designing and conducting mixed methods research. 2nd ed. Thousand Oaks, CA: Sage.
Fetters, M. & Freshwater, D., 2015. The 1+1=3 integration challenge. Journal of Mixed Methods Research, Volume 9, pp. 115-117.
Mason, J., 2006. Mixing methods in a qualitatively driven way. Qualitative Research, 6(1), pp. 9-25.
Mertens, D. et al., 2020. The future of mixed methods: A five year projection to 2020, s.l.: Mixed Methods International Research Association (MMIRA).
Mertens, D. M., 2009. Transformative research and evaluation. New York: Guilford.
Palinkas, L. A., Aarons, G. A. & Worwitz, S., 2011. Mixed methods designs in implementation research. Adm Policy Ment Health, 38(1), pp. 44-53.