Category: Unit 7

Melchar and Bosco’s (2010) study is evaluated through Plano-Clark and Creswell’s (2015) seven criteria for evaluating the use of mixed methods in a research report (p. 405).

1. The rationale for needing mixed methods research is appropriate and justified (0/3) – Melchar & Bosco (2010) do not explicitly articulate why a mixed method was chosen instead of a quantitative or qualitative study alone. The study states “qualitative interviews were conducted a priori in order to determine whether there were main themes that reflected the senior leader’s “servant leader” orientation” (Melchar & Bosco, 2010, p. 80), but no justification for the quantitative surveys were given.
2. The choice of the mixed methods design is appropriate and justified (1/3) – The sequential exploratory mixed methods design was chosen for this study. As mentioned, Melchar & Bosco (2010) collected qualitative data from the senior leaders first to analyze whether the leaders reflected the servant leader (SL) qualities (p. 80). To understand the influence of senior leaders’ use of SL, it was crucial to interview the senior leaders. The timing of the study and the unequal priority of the quantitative study is justified, but once again, there is no explicit reasoning from the researchers of their specific design. Plano-Clark & Creswell (2015) state, “At a minimum, all mixed methods studies should “mix” the quantitative and qualitative components in the final conclusion section” (p. 389); the study fails to do this.
3. The quantitative methods are of good quality based on the standards of quantitative research (1/3) – The study only received 59 respondents to their survey that consisted of 91% male sample group, when 350+ individuals are expected for a survey (Plano-Clark & Creswell, 2015, p. 238). Melchar & Bosco (2010) provided  coefficient alphas (p. 79) and an ANOVA (p. 80) to enhance the reliability, but this does not justify the small sample. No example of the survey or its questions were included.
4. The qualitative methods are of good quality based on the standards of qualitative research (2/3) – The qualitative methods incorporated three interviews that provided purposeful information to identify main themes, and a few examples of the open-ended questions were provided. More information on the qualitative process is needed.
5. The quantitative and qualitative components of the study are meaningfully mixed (1/3) – While the need to interview the senior leaders is explained, the study does not further interpret or mix the interview results with the quantitative survey aspect in the discussion or results. The interview is treated like a test of whether the leader practices SL, and is then discarded for the overall results.
6. The study used a rigorous application of mixed methods research to address the purpose (1/3) – Addressing the question of whether the incorporation of servant leadership at the senior levels influences a culture of servant leadership at lower levels of management (Melchar & Bosco, 2010, p. 77), the use of interviews and surveys is logical, but the lack of analysis and mixing of the two elements make it seem unnecessary. Either incorporating the interview process or solely relying on surveys would have worked.
7. The use of mixed methods produced a good understanding of the research purpose (1/3) – I am not convinced the use of mixed methods produced more complete, valid, or in-depth answers to the study’s research questions. The quantitative and qualitative aspects did not enhance one another.

References

Melchar, D. E., & Bosco, S. M. (2010). Achieving High Organization Performance through Servant Leadership. Journal Of Business Inquiry: Research, Education & Application, 9(1), 74-88.

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

The heart of the quantitative-qualitative debate is whether the mixed-methods research approach should be used due to the differences inherent to the quantitative and qualitative approaches. As Plano-Clark and Creswell (2015) state, “[There are] important differences between quantitative and qualitative research [that] range from they types of research problems to the types of purposes, to the forms of gathered data, and even to the role of the researcher” (p. 385). The argument against using the mixed-methods approach is: “How can the result be similar if the two paradigms are supposedly looking at different phenomena? Achieving similar results may be merely a matter of perception” (Sale, Lohfeld, and Brazil, 2002, p. 47).  Conversely, Plano-Clark & Creswell (2015) provides several rationals for using mixed methods: 1) to combine quantitative and qualitative data strengths, 2) to build from one type of data to the other, and 3) to answer two questions (pp. 385-386).

Agreeing with the reasoning for mixed methods, Stentz, Clark, and Matkin (2012) state, “Researches can maximize the strengths of each approach while making up for the weaknesses of the approaches, develop more complete and complementary understanding, increase validity of results, use one form to build on the results of the other, and/or examine contextualized understanding, multi-level perspectives, and cultural influences” (p. 1174) in studying leadership. Quantitative studies have limitations such as limited understanding of the participants’ context and being researcher driven, while qualitative studies are limited by the small sample and being highly interpretive (Creswell, 2013, What is Mixed Methods Research), but applying both methods into leadership research can “promote the development of new understanding about existing theories while advancing our theoretical thinking in ways that both deeper and broader” (Stentz et al., 2012, p. 1173).

Considering the pros and cons of the mixed-methods approach, I agree mixed methods is a powerful way to collect and analyze data. Plano-Clark & Creswell (2015) offer a list of design intents (p. 391) and I believe if used correctly, mixed methods can provide valid and insightful information that connects the personal stories and numbers together to provide a well-rounded research. I take most interest in how mixed methods can enhance a qualitative or quantitative study and also how it can “examine multiple levels” (Plano-Clark & Creswell, 2015, p. 391). While I enjoy seeing the connections between different levels a study asks for, my concern returns to the concern raised above: “Achieving similar results may be merely a matter of perception” (Sale et al., 2002, p. 47).

Question: How do you identify and evaluate the validity of a study’s use of multiple levels knowing the threat of forced perception in analyzing the different levels? Do you think there is a definite boundary in between a valid use of multiple levels and a forced compilation of different levels to create a single finding?

References

Creswell, J. (2017, June 1). What is mixed methods research? [Video file]. Available from http://johnwcreswell.com/videos/

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Sale, J.E.M., Lohfeld, L.H., Brazil, K. (2002). Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Quality & Quantity, 36(1), 43-53.

Stentz, J. E., Plano Clark, V. L., & Matkin, G. S. (2016). Applying mixed methods to leadership research: A review of current practices: Corrigendum. The Leadership Quarterly, 27(4), 711. doi:10.1016/j.leaqua.2015.12.006

Link to blog question