Category: LDRS591

Posted on

LDRS591, Unit 7, Activity 7.2

Melchar and Bosco’s (2010) study is evaluated through Plano-Clark and Creswell’s (2015) seven criteria for evaluating the use of mixed methods in a research report (p. 405).

  1. The rationale for needing mixed methods research is appropriate and justified (0/3) – Melchar & Bosco (2010) do not explicitly articulate why a mixed method was chosen instead of a quantitative or qualitative study alone. The study states “qualitative interviews were conducted a priori in order to determine whether there were main themes that reflected the senior leader’s “servant leader” orientation” (Melchar & Bosco, 2010, p. 80), but no justification for the quantitative surveys were given.
  2. The choice of the mixed methods design is appropriate and justified (1/3) – The sequential exploratory mixed methods design was chosen for this study. As mentioned, Melchar & Bosco (2010) collected qualitative data from the senior leaders first to analyze whether the leaders reflected the servant leader (SL) qualities (p. 80). To understand the influence of senior leaders’ use of SL, it was crucial to interview the senior leaders. The timing of the study and the unequal priority of the quantitative study is justified, but once again, there is no explicit reasoning from the researchers of their specific design. Plano-Clark & Creswell (2015) state, “At a minimum, all mixed methods studies should “mix” the quantitative and qualitative components in the final conclusion section” (p. 389); the study fails to do this.
  3. The quantitative methods are of good quality based on the standards of quantitative research (1/3) – The study only received 59 respondents to their survey that consisted of 91% male sample group, when 350+ individuals are expected for a survey (Plano-Clark & Creswell, 2015, p. 238). Melchar & Bosco (2010) provided  coefficient alphas (p. 79) and an ANOVA (p. 80) to enhance the reliability, but this does not justify the small sample. No example of the survey or its questions were included.
  4. The qualitative methods are of good quality based on the standards of qualitative research (2/3) – The qualitative methods incorporated three interviews that provided purposeful information to identify main themes, and a few examples of the open-ended questions were provided. More information on the qualitative process is needed.
  5. The quantitative and qualitative components of the study are meaningfully mixed (1/3) – While the need to interview the senior leaders is explained, the study does not further interpret or mix the interview results with the quantitative survey aspect in the discussion or results. The interview is treated like a test of whether the leader practices SL, and is then discarded for the overall results.
  6. The study used a rigorous application of mixed methods research to address the purpose (1/3) – Addressing the question of whether the incorporation of servant leadership at the senior levels influences a culture of servant leadership at lower levels of management (Melchar & Bosco, 2010, p. 77), the use of interviews and surveys is logical, but the lack of analysis and mixing of the two elements make it seem unnecessary. Either incorporating the interview process or solely relying on surveys would have worked.
  7. The use of mixed methods produced a good understanding of the research purpose (1/3) – I am not convinced the use of mixed methods produced more complete, valid, or in-depth answers to the study’s research questions. The quantitative and qualitative aspects did not enhance one another.

References

Melchar, D. E., & Bosco, S. M. (2010). Achieving High Organization Performance through Servant Leadership. Journal Of Business Inquiry: Research, Education & Application, 9(1), 74-88.

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Posted on

LDRS591, Unit 7, Activity 7.1

The heart of the quantitative-qualitative debate is whether the mixed-methods research approach should be used due to the differences inherent to the quantitative and qualitative approaches. As Plano-Clark and Creswell (2015) state, “[There are] important differences between quantitative and qualitative research [that] range from they types of research problems to the types of purposes, to the forms of gathered data, and even to the role of the researcher” (p. 385). The argument against using the mixed-methods approach is: “How can the result be similar if the two paradigms are supposedly looking at different phenomena? Achieving similar results may be merely a matter of perception” (Sale, Lohfeld, and Brazil, 2002, p. 47).  Conversely, Plano-Clark & Creswell (2015) provides several rationals for using mixed methods: 1) to combine quantitative and qualitative data strengths, 2) to build from one type of data to the other, and 3) to answer two questions (pp. 385-386).

Agreeing with the reasoning for mixed methods, Stentz, Clark, and Matkin (2012) state, “Researches can maximize the strengths of each approach while making up for the weaknesses of the approaches, develop more complete and complementary understanding, increase validity of results, use one form to build on the results of the other, and/or examine contextualized understanding, multi-level perspectives, and cultural influences” (p. 1174) in studying leadership. Quantitative studies have limitations such as limited understanding of the participants’ context and being researcher driven, while qualitative studies are limited by the small sample and being highly interpretive (Creswell, 2013, What is Mixed Methods Research), but applying both methods into leadership research can “promote the development of new understanding about existing theories while advancing our theoretical thinking in ways that both deeper and broader” (Stentz et al., 2012, p. 1173).

Considering the pros and cons of the mixed-methods approach, I agree mixed methods is a powerful way to collect and analyze data. Plano-Clark & Creswell (2015) offer a list of design intents (p. 391) and I believe if used correctly, mixed methods can provide valid and insightful information that connects the personal stories and numbers together to provide a well-rounded research. I take most interest in how mixed methods can enhance a qualitative or quantitative study and also how it can “examine multiple levels” (Plano-Clark & Creswell, 2015, p. 391). While I enjoy seeing the connections between different levels a study asks for, my concern returns to the concern raised above: “Achieving similar results may be merely a matter of perception” (Sale et al., 2002, p. 47).

Question: How do you identify and evaluate the validity of a study’s use of multiple levels knowing the threat of forced perception in analyzing the different levels? Do you think there is a definite boundary in between a valid use of multiple levels and a forced compilation of different levels to create a single finding?

References

Creswell, J. (2017, June 1). What is mixed methods research? [Video file]. Available from http://johnwcreswell.com/videos/

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Sale, J.E.M., Lohfeld, L.H., Brazil, K. (2002). Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Quality & Quantity, 36(1), 43-53.

Stentz, J. E., Plano Clark, V. L., & Matkin, G. S. (2016). Applying mixed methods to leadership research: A review of current practices: Corrigendum. The Leadership Quarterly, 27(4), 711. doi:10.1016/j.leaqua.2015.12.006

Link to blog question

Posted on

LDRS591, Unit 6, Response

This is a response to Simarjit’s post question: Since the qualitative process is ‘explorative’ and ‘evolves’ along the way, the results could be interpreted in different ways. Since the reader doesn’t always have access to the analysis tools or programs used by the researchers, how does the reader reconcile their interpretation with that of the researchers? Seems like trust in the researchers is an important element to the consumer of qualitative research.

Simarjit,

You are right that qualitative research is exploratory and evolves. I see the evolution of the research in the contradicting results found from one study to another that find differing results due to culture, the social beliefs in the time of history they conduct the study, or other factors. For example, I explored studies analyzing female leadership this past week and found a range of data that found women were equally capable or less capable than men in leading. Studies in the past reflected the social beliefs of leadership being a masculine characteristics, while more recent studies reflect the growing acknowledgement of women’s capabilities. Research is continually building and responding to one another, but I believe research always has a best answer. It is up to the reader to find the most accurate interpretation of a study, and that means not basing her interpretation solely on herself or the single research she read. To answer your question specifically, I have two suggestions on how readers can reconcile their interpretation with the researchers’ below:

  1. Reading different studies: Similar studies that produce contradicting results or similar findings help consumers of research reports find credible and accurate findings. Considering different results helps readers find a more well-rounded understanding of the study results, and similar findings confirm the accuracy of the researchers’ findings.
  2. Understanding the research and data analysis process: Plano-Clark & Creswell (2015) state, “A reader of research […] need[s] to understand both the process researchers use to analyze qualitative data and how the results of this process are reported to fully appreciate and evaluate the findings presented in a qualitative research report” (p. 353). Evaluating the processes the researchers use can help readers find credible processes that will yield reliable results. Basing one’s judgement on the credibility of the described processes is much more reliable than basing it on the researchers’ or one’s own interpretations.

References

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Posted on

LDRS591, Unit 6, Activity 6.4

As a consumer of research reports, the most important things for me in the methods and results section of a high-quality qualitative research report are…

  1. A credible sample that provides well-rounded and accurate results. The researchers’ attention to finding the “right” participants where their characteristics such as race, sex, age, and culture is crucial to providing good data. As Plano-Clark & Creswell (2015) state, “[Purposeful sampling] is best suited for qualitative research because the researcher is able to select the individuals who are most appropriate for a study of the central phenomenon” (p. 332). There are limited participants, and finding the right participants to provide rich and meaningful information provides greater accuracy!
  2. Clear and rigorous procedures that offer credible data and analysis. Providing a clear framework of the study’s procedure not only helps readers understand the study, but also shows the attention the researchers implemented to provide consistency in their data collection. Also, their use of rigorous procedures that adhere to credible standards of qualitative studies that involves processes such as audits, triangulation, and bracketing reveals credibility in their results.
  3. A good presentation of data that clearly outlines and compares the data collected. Due to the nature of data collection in qualitative studies, there are lots of words as participants are interviewed or asked to complete a questionnaire and this can cause an overwhelming sight on a study. Providing a well structured presentation of the data with subtitles, charts, tables, and the researchers’ interpretations help me understand the participants better and the data the researchers focused on and considered.

Question: I personally find it difficult to understand a study fully without a full documentation or example of the questions used to collect data from their participants – I like the details! Do you think it is important for researchers to provide all these questions and responses from participants, or do you find this unnecessary information?

References

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Link to blog question: https://create.twu.ca/ldrs591-sp18/unit-6-learning-activities/

Posted on

LDRS591, Unit 6, Activity 6.3

E. Russell, Maxfield, and J. Russell’s (2017) study is evaluated based on Plano-Clark & Creswell’s (2015) seven criteria for evaluating the data analysis of a qualitative research report (p.  301) below:

  1. The analysis process used rigorous qualitative procedures (2/3) – Russell et al. (2017) sorted and analyzed the data using a hand coding process (p. 86) that involved “a lot of time reading and rereading the data” (p. 86), but failed to provide “verbatim transcripts and electronic scans of all gathered data” (Plano-Clark & Creswell, 2015, p. 377).
  2. Strategies were used to validate the findings (2/3) – Russell et al. (2017) used triangulation to validate their results, and “had another researcher preform an analysis for comparison” (p. 86). Despite this, there is not a clear indication of the researchers using at least three of the four strategies Plano-Clark & Creswell (2015) identified as crucial to validating results: bracketing, triangulation, member checking, and audits.
  3. The findings include a description of people, places, or events in the study (1/3) – The study provides details on each participants’ organizational position and for-profit sector as shown in Table 2 (Russell et al., 2017, p. 85), and general information on the age range, sex, and location of the study. Plano-Clark & Creswell (2015) state, “A good qualitative description […] should transport you into the context of the research and help you get to know the people, setting, or events that are important in the study” (p. 366). Due to the researcher’s emphasis on anonymity, there are broad but not many narrow descriptions. While I feel the study still provides a general idea of its participants, further details on the people and setting of the organizations would have benefited the study.
  4. The findings include appropriate themes about the central phenomenon (2/3) – There are seven meaningful themes identified in the two attributes found in the results that are supported by multiple quotes and perspectives. A comparison table and further interpretation from the researchers’ voice would have benefited this area.
  5. The findings relate multiple themes to each other (3/3) – The themes are linked through the two attributes identified: 1) validation as a leader, and 2) freedom from management. There are multiple layers of categories within the two attributes that connect consistently to the central question of how “senior leaders interpret the personal benefits derived from serving the needs of followers” (Russell et al., 2017, p. 76).
  6. The data analysis represents a good qualitative process (2/3) – The data analysis involved a dynamic coding process that involved open codes, axial codes, and selective codes that were interpreted based on the data collected. The researchers reveal a credible process with relatively accurate results despite a low sample number and limitations created by the questionnaire such as “discovering how these leaders arrived at their position of leadership” (Russell et al., 2017, p. 93).
  7. The findings provide a good exploration of the central phenomenon (2/3) – The findings provide detailed information in understanding “the benefits to one’s self-interest from being a servant leader” (Russell et al., 2017, p. 93). Overall, the findings were well focused on the study’s research question and added to the discussion of servant leadership.

References

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Russell, E. J., Maxfield, R. J., & Russell, J. L. (2017). Discovering the self-interest of servant leadership: A grounded theory. Servant Leadership: Theory and Practice, 4(1), 75-97. Retrieved from http://www.sltpjournal.org/uploads/2/6/3/9/26394582/06russell_vol_4_issue_1.pdf

Posted on

LDRS591, Unit 6, Activity 6.2

Plano-Clark & Creswell’s (2015) evaluation of participants and data collection in a qualitative research report is implemented on E. Russell, Maxfield, & J. Russell’s (2017) study below:

  1. The sampling strategy is appropriate and justified (2/3) – Russell et al. (2017) select a variety of leaders from “organizations headquartered in the Western United States” (p.  84) which provide perspectives from numerous for-profit sectors such as education, legal services, and finance. This proposes a well selected variety of participants. While the sampling strategy that used questionnaires was purposeful, interviews could have better served the purpose of the study; the researchers also acknowledge this by recommending the use of interviews for future studies.
  2. The sample size is appropriate and justified (1/3) – Plano-Clark & Creswell (2015) state a grounded theory study should include 20 to 30 individuals (p. 336), but Russell et al. (2017) only use 14 senior level leaders for their study. Russell et al. (2017) acknowledge this is a limitation, but state,”Researchers relied on data saturation to determine the study’s sample size” (p. 85).
  3. The data types are appropriate (2/3) – The open-ended questions asked on the questionnaires provide a good venue for participants to share their experience through a framework that allows the researchers to collect quantitative information, but as Plano-Clark & Creswell (2015) mention, the data collected is limited “because participants often write very little […] and researchers cannot ask the participants for clarifying information” (p. 340).
  4. The data are gathered using rigorous qualitative procedures (2/3) – Although not all the questionnaire questions are provided, Russell et al. (2017) “developed a script consisting of open-ended questions to be used as the study questionnaire” (p. 84) and provide examples of the questions in the results section of the page. The benefit of the questionnaires is they can keep a good collection of their detailed questions and answers. The researchers also conducted a pilot project to refine the questions, but due to the nature of questionnaires, no follow-up questions were asked to further probe for data.
  5. Data collection issues are handled ethically and thoughtfully (3/3) – Russell et al. (2017) indicate their respect of their participants’ rights by reinforcing the volunteer aspect of the participation and protecting the anonymity of their participants through their procedure. “The researchers removed any personal identifiers” (p. 85) and ensured trustworthiness by “triangula[ting] data sources from multiple participants, had another researcher preform an analysis for comparison, and presented the data as in-depth rich descriptions” (p. 85).
  6. The selected participants are information rich (3/3) – As Table 2 (Russell et al., 2017, p. 85) indicate, the sampling was well rounded in age and gender from a variety of sectors in senior level leadership. The participants are a clear fit for the study.
  7. The database provides extensive and credible information about the central phenomenon (2/3) – Russell et al. (2017) provide great examples of their data that include open-ended questions and answers that relate to the central research question collected via different questionnaires. It is unknown of how long the overall process took and I believe further details of the information gathered could have been provided.

References

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Russell, E. J., Maxfield, R. J., & Russell, J. L. (2017). Discovering the self-interest of servant leadership: A grounded theory. Servant Leadership: Theory and Practice, 4(1), 75-97. Retrieved from http://www.sltpjournal.org/uploads/2/6/3/9/26394582/06russell_vol_4_issue_1.pdf

Posted on

LDRS591, Unit 6, Activity 6.1

l will evaluate E. Russell, Maxfield, and J. Russell’s (2017) study based on Plano-Clark & Creswell’s (2015) seven criteria for evaluating the research design of a qualitative research report (p.  301).

  1. A research design guides the conduct of the qualitative study (3/3) –  Russell et al. (2017) use the qualitative grounded theory “to discover how senior level leaders perceive personal benefits derived from serving the needs of their followers” (p. 75); this study researched the action of perceiving personal benefits through online questionnaires that questioned 14 senior level leaders. As the grounded theory approach seeks to explain “a process, action, or interaction [….] built from the experiences and perspectives of participants,” Russell et al. (2017) seems knowledgeable about the design and use accurate terms to describe the grounded theory process.
  2. The choice of the research design is appropriate  and justified (3/3) – Russell et al. (2017) justify their choice of using the qualitative grounded theory by stating: “The design allows for an analysis of data using a constant comparative method” (p. 84). Beyond this reasoning, the nature of the study that considers the experience of its participants’ interpretation of their personal benefits derived from serving followers requires personal assessments of leaders “who have experienced the process” (Plano-Clark & Creswell, 2015, p. 298) that the grounded theory offers.
  3. Good qualitative data collection procedures are used (2/3) – Plano-Clark & Creswell (2015) state, “You often find that the grounded theory researcher interviewed people” (p. 298). While Russell et al. (2017) do not interview their participants directly, they choose to collect their participants’ experience and perspectives through online questionnaires to protect their anonymity which equally serves the study’s purpose. The researchers ran a pilot study to refine the questions, and “established a secure database for data collection and storage to ensure data reliability” (p. 85).
  4. Good qualitative analysis procedures are used (3/3) – A good grounded research analyses “data using multiple stages of coding” (Plano-Clark & Creswell, 2015, p. 298). Russell et al. (2017) adhere to this characteristic by using “a tiered process for the sorting and analysis of the data” (p. 86) which included open codes and axial codes.
  5. Good qualitative results and interpretations are reported (3/3) – The results are organized well into two categories (or attributes) and a sample of the questionnaire answers are presented to support the two attributes. Overall, the conclusion and discussion of the data interpret the generated theory that leaders benefits from serving followers.
  6. The study used a rigorous research design (3/3) – The study’s process and analysis fit together logically and coherently. The article begins with a clear framework of the characteristics of servant leadership, and explore its research question thoroughly.
  7. The use of the qualitative research design addressed the study’s purpose (3/3) – The research design fits the study’s intent and answers the study’s intent of understanding how senior leadership perceives personal benefits of serving their followers; “leaders realize personal benefits from serving the needs of followers” (Russel et al., 2015, p. 92).

References

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Russell, E. J., Maxfield, R. J., & Russell, J. L. (2017). Discovering the self-interest of servant leadership: A grounded theory. Servant Leadership: Theory and Practice, 4(1), 75-97. Retrieved from http://www.sltpjournal.org/uploads/2/6/3/9/26394582/06russell_vol_4_issue_1.pdf

Posted on

LDRS591, Unit 5, Response

This is a response to Layla’s post.

Hi Layla,

You asked: How do you evaluate the results of a quantitative study when the information of the statistics or instruments is not clear enough? For me, if the statistical information and details of the instruments are not clearly explained, I dismiss the study as valid.

The importance of quality and credibility of a study has been discussed time and time again, and as our learning activities emphasized this week, a lack of indicators that explain the use of an instrument or lack of analysis of the results indicate low quality results and data collection processes. Beyond failing Plano-Clark & Creswell’s (2015) evaluation process, I would not use an article with more explanations because I  would simply not understand the study in full.

If I cannot understand the results or the instruments they used to produce the results, I do not have faith in the study or my interpretation of the study. Without understanding the process and the resulting statistics, I would find a hard time connecting the results with the purpose of the study. Also, I have a short attention span and would not find it worth my efforts to dig into this study further if the writing of the article poorly explains these important details.

How do you handle the ambiguity of unclear procedures and statistical results?

Posted on

LDRS591, Unit 5, Activity 5.4

As a consumer of research reports, the most important things for me in the methods and results section of a high-quality quantitative research report are…

  1. A clear procedure that provides statistically valid and credible results. The procedure of a study must explicitly explain why a research design was chosen and appropriate for the study. Readers can only understand the overall study if the steps and variables considered are explained for contextual reasoning. Clarity in the procedure also helps readers understand the findings and correlations between the variables. The procedure must also produce valid and credible results that consider all possible variables, statistics, and instruments used.
  2. Detailed explanation of the results that include descriptive statistics. While descriptive statistics are important, the mass of numbers, symbols, and statistic names at the end can be overwhelming for me. A detailed interpretation of these results and the analysis of the results in relation to the original hypothesis helps me focus on the study’s purpose.
  3.  An ethical procedure is highly important as I believe every participant deserves respect and the assurance that their best-interest is considered. With past experience reading studies in psychology, it is horrifying to read how studies can turn inhumane even with the initial innocent intentions. Ethics must be carefully examined as all possible outcomes – good or bad – are plausible.

Question:

Do you have an example of an ethically controversial study you participated in or read? What was the result of the study?

Posted on

LDRS591, Unit 5, Activity 5.3

Jenkins & Stewart’s (2010) study will be evaluated through Plano-Clark & Creswell’s (2015) seven criteria for evaluating data analysis and results in a quantitative research report on a three-point scale below:

  1. The data were rigorously scored and prepared (2/3) – Jenkins & Stewart (2010) explain their scoring process and data preparation for each variable thoroughly which consists of using previously tried survey instruments and scales such as Sherman’s scale (p. 50). There is a general consistency in the distribution of the survey, but because the exact questionnaires are not provided, it is unknown whether the questions are good indicators of the variables.
  2. Good descriptive analyses were conducted (3/3) – The tables provided reveal good descriptive analyses and consideration of the central tendency and variability in the research; the study outlines the mean, correlation matrix, and standard deviation of each variable (Jenkins & Stewart, 2010, p. 51). Z-scores are also used to “compensate for restriction of range on the basis of the Likert scale” (p. 50) used in the survey. The descriptive analyses were appropriately calculated for each variable.
  3. Good hypothesis testing procedures were used (2/3) – Although a null hypothesis is not stated, three alternative hypotheses of the relationship between managers’ leadership and nurses’ job satisfaction. The researchers also provide an Alpha level, collects data, computes the selected variables and p value, and makes a decision to confirm the original hypotheses. Due to the lack of a null hypothesis, no rejection or failure to reject the null is available.
  4. The results are comprehensive (3/3) – Table 1 of the study outlines the demographic characteristics of the participants’ ages, ethnicity, and age (Jenkins & Stewart, 2010, p. 50), and the reliability of the instruments are tested (p. 50). The results and discussion at the end of the study adequately report the outcome for each research question and hypotheses proposed at the beginning.
  5. The results include sufficient information (3/3) – Jenkins & Stewart (2010) reports the statistic, its value, the associated p value, and significance in Table 3 and data analysis. The table is clear and consistent. The analysis explain the results clearly and its relation to the hypotheses.
  6. The data analysis represents a good process (3/3) – The data analysis provided a linear and objective explanation of the data collected that discuss each considered variable and the correlation or lack of correlation between them.
  7. The results provide a good explanation of the study’s purpose (3/3) – Readers are provided a clear understanding of the study’s purpose and the relationship between the results and original intent of the study provided in the introduction. The statistics reported are all reasonable responses to the study’s consideration of the relationship between a manager’s commitment to serve and use of role-inversion with nurse satisfaction” (Jenkins & Stewart, 2010, p. 49).

References

Jenkins, M., & Stewart, A. C. (2010). The importance of a servant leader orientation. Health Care Management Review, 35(1), 46-54.

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Link to activity question: https://create.twu.ca/ldrs591-sp18/unit-5-learning-activities/