Category: LDRS591

https://create.twu.ca/ldrs591-sp18/unit-6-learning-activities/

As a consumer of research reports, the most important thing(s) for me in the methods and results section of a high-quality qualitative research report is the explanation and justification of a research design. I found it difficult to determine the research design when it was not explicitly stated. Plano-Clark and Creswell (2015) offer many suggestions for keywords to look for to assist in recognizing the research design used. Chan and So (2017) did not have any of these keywords.

It is also important for a research study to be rich in detail, support a full explanation of the research, and provide a discussion of  the final results. This can be done by including tables and figures that provide more detail and clarification of connections between themes. Plano-Clark and Creswell suggest that a the findings should be “rich and detailed so that the reader feels like they were present within the setting.” (Plano-Clark and Creswell, 2015, P 379). I felt that this research report was somewhat lacking. I did not feel this presence in my reading and found that I had a lot of unanswered question about the research design itself. Chan and So (2017) did provide several themes, sub-themes and an adequate explanation of each theme. I feel that this is important because it helps me to further understand the purpose of the study.

For further comments and discussion, I would like to know if the flexibility of qualitative research procedures impacts the validity of the research?

 

References

Chan, K. W. C, & So, G. B. K. (2017). Cultivating servant leaders in secondary schooling. Servant Leadership: Theory and Practice, 4(1), 12-31.

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

https://create.twu.ca/ldrs591-sp18/unit-6-learning-activities/

Chan, K. W. C, & So, G. B. K. (2017). Cultivating servant leaders in secondary schooling. Servant Leadership: Theory and Practice, 4(1), 12-31.

• Did the researcher(s) use at least three strategies to validate the findings?

Chan and So did not provide evidence of Triangulation, member checking or an outsider audit to validate their findings. They used focus group interviews and then categorized and coded the results. If my understanding of the research is correct, they did not use sufficient strategies to validate the findings.

Rating: 1

• Did the findings include a good description of the people, places, or events in the study?

Chan and So (2017) provide a clear description of themes that were outlined in the results. Plano-Clark and Creswell suggest that in a higher quality qualitative research design, “the important context of the study is described in rich detail using a broad-to-narrow form, factual information, action words and modifiers, participant quotes and tables or figures that provide additional details.” (Plano-Clark and Creswell, 2015, P 377). Chan and So provide the reader with several participant quotes, and use factual information. They provide one table. They could strengthen their research in this area by adding more tables and figures with information to back their findings.

Rating: 2

• Did the findings include appropriate themes about the central phenomenon?

Chan and So (2017) identified seven themes. They were (a) listening, (b) empathy, (c) empathy and healing, (d) awareness, (e) planning: conceptualization and foresight, (f) persuasion, and (g) community building. Plano-Clark and Creswell (2015) suggest five to seven themes. (Plano-Clark and Creswell, 2015, P 377).

Rating: 3

• Did the findings provide a good exploration of the central phenomenon?

Chan and So (2017) also broke the 7 themes into sub-themes which Plano-Clark and Creswell suggest as an indicator of high quality research. “The theme of awareness was subdivided into purpose, reflection, stewardship, development of self and development of others. The theme of community building was also subdivided into perseverance, teamwork, and relationship.” (Chan and So, 2017, P 22). They used several participant quotes as evidence, however they did not use comparison tables.

Rating: 2

My overall rating for evaluating the data analysis and findings is 8. This places the research by Chan and So (2017) in the category over an overall low quality article in terms of Data Analysis. I feel that there are several components missing from this research  preventing it from being adequate or high quality.

References

Chan, K. W. C, & So, G. B. K. (2017). Cultivating servant leaders in secondary schooling. Servant Leadership: Theory and Practice, 4(1), 12-31.

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

https://create.twu.ca/ldrs591-sp18/unit-6-learning-activities/

Chan, K. W. C, & So, G. B. K. (2017). Cultivating servant leaders in secondary schooling. Servant Leadership: Theory and Practice, 4(1), 12-31.

• What sampling strategy is used and is it appropriate and justified?

Plano-Clark and Creswell suggest that Theory or Concept Sampling is used to “generate a theory or explore a concept.” (Plano-Clark and Creswell, 2015, P 234). I believe that Chan and So used this method for sampling in their research. They chose a specific group that would help them to generate their theory, which is that servant leadership qualities can develop in adolescents, as well as to develop specific concepts within that theory. I believe that this sampling strategy is appropriate for the research. Plano-Clark and Creswell do state that in “a well-written qualitative report, the researchers should describe and defend the specific type of purposeful sampling strategy that they used in their study.” (Plano-Clark and Creswell, 2015, P 335). Chan and So (2015) do not justify or provide a clear explanation of the reasoning for the sampling method they have chosen.

• Is the sample size appropriate?

Plano-Clark and Creswell (2015) suggest that the sampling size should be as few as one and as many as 30 when studying a classroom as a case study. Chan and So had 11 participants in their study. I believe that this is an appropriate sample size for the classroom as a case. (Plano-Clark and Creswell, 2015, P 336).

• Is the data collected appropriate?

The data was collected in this study through focus-group interviews. The data was then recorded into audio recordings. I feel that the validity and significance of the data collected would be more valuable if Chan and So collected data in different ways as well. They conducted interviews by a main researcher and then the results were coded and categorized. It would be beneficial to include a formal observation analysis or other documented data that would strengthen the results of the research.

• Are the data gathered ethically and thoughtfully?

The data for this research was collected ethically. Chan and So (2017), state “prior to each focus-group interview, interviewees gave their written consent, acknowledging their understanding of this research, agreeing to participate in the interview and giving permission for the audio recording.” (Chan and So, 2017, P 21). Chan and So secured appropriate permissions in their research and treated participants respectfully.

References

Chan, K. W. C, & So, G. B. K. (2017). Cultivating servant leaders in secondary schooling. Servant Leadership: Theory and Practice, 4(1), 12-31.

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

https://create.twu.ca/ldrs591-sp18/unit-6-learning-activities/

Chan, K. W. C, & So, G. B. K. (2017). Cultivating servant leaders in secondary schooling. Servant Leadership: Theory and Practice, 4(1), 12-31.

 

• What research design was used to guide the study? Was it justified? (2)

Plano-Clark and Creswell suggest that a case study research design is “a set of qualitative procedures used to explore a bounded system in depth.” (Plano-Clark and Creswell, 2015, P 292). I believe that the qualitative research study conducted by Chan and So (2017) is a case study research design. In this study, Chan and So are providing an in-depth exploration on the effect of the delivery of a program, by a teacher, on the development of servant leadership traits on a group of high school students. The system is bounded, which Plano-Clark and Creswell (2015) suggest as a quality of a case study research design. (Plano-Clark and Creswell, 2015, P 293). It is bounded, as it is set within a school environment over time.

To my knowledge, Chan and So (2017) did not justify their research design. I was slightly torn when deciding whether or not this was a case study research design or a grounded theory research design. Plano-Clark and Creswell suggest tips or keywords to look for in a grounded theory research design. Chan and So (2017) did not use these terms. This led me to believe that it is a case study research design.

I rate this research a 2 in this category as the research design was not justified.

• Does the qualitative research design address the study’s purpose? (3)

Chan and So (2017), used focus group interviews to “explore a possible pathway to cultivate servant leadership attributes among learners in secondary schooling.” (Chan and S0, 2017, P 19). “The results of this study showed that the co-curricular program provided opportunities for learners to develop the attributes of servant leaders.” (Chan and So, 2017, P 25).

The purpose of the study was to determine if servant leadership qualities could be developed in adolescent youth. The results indicated that the participants in this study developed servant leadership traits.

The research design conducted provides a rich exploration that satisfies the purpose of the study, which Plano-Clark and Creswell (2015) suggest as an indicator of a higher quality research design.

Rating: 3.    

References

Chan, K. W. C, & So, G. B. K. (2017). Cultivating servant leaders in secondary schooling. Servant Leadership: Theory and Practice, 4(1), 12-31.

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

https://create.twu.ca/ldrs591-sp18/unit-5-learning-activities/

 

As a consumer of research reports, the most important thing(s) for me in the methods and results section of a high-quality quantitative research report are: to understand the purpose of the study and how the study will be conducted, and for a clear discussion of the results, including the interpretation of the data.  As a person with limited knowledge of statstics, it is easier for me to understand the results when they are presented as a discussion in the conclusion section, rather than being presented in a Table.

Can one read the abstract and the data analysis and conclusion sections only, to have a basic understanding of the research?

Unit 5 Learning Activities

 

1. The data were rigorously scored and prepared (3/3) – Cerit provides a clear explanation of the scoring process and results of the questionnaire. The servant leadership scale (OLA), developed by Laub in 1999, was used in this research. It is a measurement scale that uses six factors of servant leadership. Respondents were required to respond to questions using a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). (Cerit, Y. 2009, P 608). Plano-Clark and Creswell suggest that clear and consistent manners should be used to score the data. (Plano-Clark and Creswell, 2015, P 278). The scale used is clearly described and it is consistent. The author also clearly explains how the data was prepared.
2. Good descriptive analyses were conducted (2/3) – The author outlines the correlation matrix, the standard deviation, standardized regression coefficients (β ), and variance inflation factors (VIF). The tables provide information on the central tendency as well as the mean. The tables were somewhat confusing to interpret as not all of the statistical variables used align with the statistical variables in Plano-Clark and Creswell. For a person with no statistics courses in their background, it is difficult to understand what each variable represents.
3. Good hypothesis testing procedures were used (1/3) – Upon my review of the article, I cannot see that a null hypothesis is stated. Cerit states an alternative hypothesis.The author does provide Cronbach’s alpha coefficient but does not provide the alpha level. The researcher did collect, analyze and interpret the data that was collected in the questionnaire.
4. The results are comprehensive (2/3) – The study refers to number of years of teaching experience, gender and level of education however they only do this to clarify that the selection of teachers is random. They do not report the results based on these different variables. Cerit does comment on the reliability of the OLA. “The OLA is a reliable instrument for measuring servant leadership. The internal consistency of the survey instrument was reliable at an acceptable level.” (Cerit Y., 2009, P 609-610).
5. The results include sufficient information (2/3) – The author provides tables and figures. As previously stated, I personally found the tables somewhat difficult to interpret as not all of the variables align with Plano-Clark and Creswells variables. Cerit does provide a p value and did provide a clear determination that significant results were found. (Plano-Clark and Creswell, 2015, P 279).
6. The data analysis represents a good process (3/3) – SPSS was used for the data analysis (Cerit Y., 2009, P 610). The analysis provided an explanation with the correlation between variables. The analysis was deductive, linear and objective.
7. The results provide a good explanation of the study’s purpose (3/3) – “The results of this study revealed that there was a positive and significant relationship between servant leadership behaviours of principals and teachers’ job satisfaction, and that servant leadership behaviours of principals had a significant effect on job satisfaction.” (Cerit, Y., 2009, P 613). Cerit provides the reader with a clear understanding of the results and purpose of the study.

References

Cerit, Y. (2009). The Effects of Servant Leadership Behaviours of School Principals on Teachers’ Job Satisfaction. Educational Management Administration & Leadership, 37(5), 600. doi:10.1177/1741143209339650

Creswell, J. W. (2015). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 5th Edition. [Bookshelf Online]. Retrieved from https://bookshelf.vitalsource.com/#/books/9780133570847/

Plano-Clark, V., Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Unit 5 Learning Activities

 

• The sampling strategy is appropriate and justified (3/3) –  Cerit provides a clear explanation of the gender, teaching experience and levels of education for the teachers that responded to the survey. The survey was sent out to 700 teachers and 595 responded. This is an appropriate sample size for the survey. Plano Clark and Creswell suggest that a higher quality sampling strategy should be “fully described and clear” (Plano Clark and Creswell, 2015, P 249). Cerit justified his sampling by stating, “no studies on this subject have been carried out in Turkey.” (Cerit Y., 2009, P  601)
• The sample size is appropriate and justified (2/3) – The sample size for this survey was appropriate. Plano-Clark and Creswell suggest that a survey study should have at least 350 responses. (Plano-Clark and Creswell, 2015, P 238).  Cerit surveyed across 29 different schools and had 595 teachers respond to the survey. Plano Clark and Creswell state that in order to meet criteria for higher quality quantitative research, the author needs to strongly justify the sample size using a procedure. Cerit stated the sample size but did not justify it.
• High quality instruments are used to gather data (3/3) – Cerit provides an explanation of the instrument used in the research. The servant leadership scale (OLA), developed by Laub in 1999, is a measurement scale that uses six factors of servant leadership. Respondents were required to respond to questions using a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). (Cerit, Y., 2009, P 608). Plano-Clark and Creswell state that the author should indicate that the instrument was previously developed for research purposes. Cerit provides references to other servant leadership research in which this instrument was used. (Cerit, Y., 2009, P 608).
• The data gathered using ethical quantitative procedures (1/3) –  Cerit sent out 700 survey questionnaires.  Of those 585 elementary teachers responded; an 85% response rate.  Cerit does not indicate specifically what consent was formally obtained, if any, before the survey was sent out.  One can only infer that consent was specifically asked for on the survey questionnaire document, or alternatively that consent was assumed by virtue of the fact that the teacher responded to the questionnaire.
• The data are gathered using standardized quantitative procedures (2/3) – Plano-Clark and Creswell state that all participants must be asked the same questions, are given the same choices for responding, are given the same instructions for completing the survey and can complete the survey under similar conditions such as the same amount of time etc. (Plano-Clark and Creswell, 2015, P 245-246). Cerit meets all of these criteria for using standardized quantitative procedures. A criticism of the standardized quantitative procedures for this research may be that respondents may have had the opportunity to talk to others while completing the questionnaire and I believe that this could easily influence their responses. Plano-Clark and Creswell, (2015) suggest that researchers should ensure that participants cannot talk to each other while completing the survey, however, with a sample size this large, that would be difficult to control.
• The study has a high level of internal validity (3/3) – This research measured the relationship between factors of servant leadership and job satisfaction. “Results of correlation analysis revealed a positive and significant relationship between extrinsic job satisfaction of teachers and factors of servant leadership. (Cerit Y., 2009, P 611). I believe this study has good internal validity.
• The study has high level of external validity (3/3) – The study had an 85% rate of response. Plano-Clark and Creswell (2015) suggest that 80% or higher indicates high external validity. (Plano-Clark and Creswell, 2015, P 247). The researchers also examined the demographics of the participants including gender, teaching experience and education

 

References

Cerit, Y. (2009). The Effects of Servant Leadership Behaviours of School Principals on Teachers’ Job Satisfaction. Educational Management Administration & Leadership, 37(5), 600. doi:10.1177/1741143209339650

Creswell, J. W. (2015). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 5th Edition. [Bookshelf Online]. Retrieved from https://bookshelf.vitalsource.com/#/books/9780133570847/

Plano-Clark, V., Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Unit 5 Learning Activities

• What quantitative research design is used and is it justified?

The quantitative research design used in this nonexperimental research is a survey design. (Plano Clark and Creswell, 2015, P 196). The purpose of the study was to examine the effects of the servant leadership behaviour in principals of elementary schools, and determine how it impacted teacher’s job satisfaction.(Cerit, Y., 2009, p. 600). “Justifying the research problem means that you use the literature to document the importance of the issue examined in the study” (Creswell, J.,2015, p.13).The study is justified as there is a significant review of literature and current research on the subject matter, and this type of study has not ever been done in Turkey. (Cerit, Y., 2009, p. 601) The leadership scale developed by Laub (1999) was used to determine servant leadership behaviours in principals, and the job satisfaction scale developed by Mohrman et al. (1977), to determine teacher job satisfaction. (Cerit, Y., 2009, p. 600). The study method is high quality in that it provides trends to variables in a given population: in this case principals and teachers. The study was conducted by giving questionnaires out to 700 primary school teachers in 29 different primary schools in Turkey. They had an 85% response rate, as 595 teachers responded. (Cerit, Y., 2009, p. 608) The reason for the research was very clear and was justified.

• Are good quantitative procedures used to select and assign participants?

The procedure for obtaining participants was appropriate.  Participants were randomly selected within the population.  Researchers sent the questionnaire to 700 primary school teachers in 29 different primary schools and out of this, 595 responded. (Cerit, Y., 2009, p. 608) Of this group, 56.3% were female and 43.7% were male.(Cerit, Y., 2009, p. 608) “21% of the teachers had a bachelor’s degree, 67.9 a college degree and 10.9% a master’s degree”(Cerit, Y., 2009, p. 608) In terms of the experience that the teacher’s had, the number of years teaching varied from 1 year to 21 years or more. (Cerit, Y., 2009, p. 608) The researchers identified a few variables to study based on the leadership scale developed by Laub (1999), which was used to determine servant leadership behaviours in principals, and the job satisfaction scale developed by Mohrman et al. (1977), to determine teacher job satisfaction. (Cerit, Y., 2009, p. 600).

• Are good quantitative data collection procedures used?

Rigorous quantitative data collection procedures were used. The instrument used for quantifying the data was a questionnaire made up of 68 questions. Of those, 60 were related to determining the servant leadership behaviour of principals and 8 were questions to measure teacher job satisfaction (Cerit., Y., 2009, p. 608) After collecting and analyzing the data, the researchers were able to draw conclusions from the results.

• Does the quantitative research design address the study’s purpose?

The basic quantitative research design format was: introduction, review of the literature, methods, results, and discussion to address the study’s purpose. The results of the questionnaire were broken down and rigorously analyzed using statistical analyses, such as: standard deviation, mean, correlation and multi-regression analysis. The researchers found that there is a positive and significant relationship between a teacher’s level of job satisfaction and the servant leadership qualities displayed by the principal. (Cerit, Y., 2009, p.613).The findings were supported by other studies that had been conducted previously, but not in Turkey. The researchers concluded the study by acknowledging that the answers to the questionnaires were based on an individual teacher’s perception and biases within their school setting, and it is recommended that in order to generalize the results, more studies would have to be conducted in different places. (Cerit, Y., 2009, p. 617). I would rate this research as: 3.

Reference:

Cerit, Y. (2009). The Effects of Servant Leadership Behaviours of School Principals on Teachers’ Job Satisfaction. Educational Management Administration & Leadership, 37(5), 600. doi:10.1177/1741143209339650

Creswell, J. W. (2015). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 5th Edition. [Bookshelf Online]. Retrieved from https://bookshelf.vitalsource.com/#/books/9780133570847/

Plano-Clark, V., Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.).
Boston, MA: Pearson.

As a consumer of research reports, the most important things for me in the introduction section of a high-quality research report are as follows:

1. That the subject matter is of interest to me;
2. That the author provides a clear and concise purpose statement in the introduction (P184 of clark and creswell);
3. That the need for the research is clear;
4. How the research will be conducted, ie. by what methodology; and,
5. The results discussion.

In the first few sentences of the introduction, the reader must become interested in the topic and must understand why there is a need for the research to be conducted in the first place. What exactly does the research hope to find or explain? Basically, the “so what?” question must be answered. The researcher must convince the reader that the study is important and is worthwhile.The reader must then understand what methodology will be used. There has to be a logical fit between what the purpose of the research is, and how the research will be conducted. The introduction should also identify who would benefit from the results of the study. Finally, the results should be presented as a brief summary.   

Within the sample that the researchers used, they noted that servant leadership is most prevalent in education organizational settings. (Parris and Peachey, 2013, P 385).  My question is, is the servant leadership theory more applicable to organizations in certain economic sectors more than others?

References

Parris, D., & Peachey, J. (2013). A Systematic Literature Review of Servant Leadership Theory in Organizational Contexts. Journal Of Business Ethics, 113(3), 377-393. doi:10.1007/s10551-012-1322-6

• Is the study’s purpose clearly specified?

Yes the purpose of the study is clearly stated. It is to “identify empirical studies that explored servant leadership theory by engaging a sample population in order to assess and synthesize the mechanisms and outcomes and impact on servant leadership” (Parris, D., & Peachey, J., 2013, p. 377).

• Is the focus of the study appropriate?

The focus of the study is appropriate, but it is very broad as it is a Systematic Literature Review. The variables are clearly defined in the summary, but are vague:  “the researchers sought to provide an evidence-based answer of how does servant leadership work and how can we apply it” (Parris, D., & Peachey, J., 2013, p. 377).

• Is the overall intent of the study appropriate?

Yes, the intent is clearly stated. The researchers are clear on what they hope to learn. “Thus, we sought to provide an evidence-informed answer to how does servant leadership work, and how can we apply it?”(Parris, D., & Peachey, J., 2013, p. 377).

• Are the participants and the sites appropriate?

Yes the participants and the sites are appropriate. “A disciplined screening process resulted in a final sample population of 39 appropriate studies” (Parris, D., & Peachey, J., 2013, p. 377).

• Is the purpose of the study narrowed through appropriate research questions and/or hypotheses?

In terms of what literature was selected, the study was definitely narrowed by virtue of the fact that it was a systematic review of literature rather than a narrative review.  “The purpose of this study was to systematically examine and organize the current body of research literature that either quantitatively or qualitatively explored servant leadership theory in a given organizational setting. In this SRL we only included empirical studies that investigated servant leadership in an organizational context and excluded studies with a primary focus on model development or testing measurement instruments.(Parris, D., & Peachey, J., 2013, p. 378).

• Does the purpose of the study follow logically from the statement of the problem and the literature review?

Yes the purpose of the study is to assess empirical studies that explored theories of servant leadership in organizations to determine the impact of servant leadership on organizations (Parris, D., & Peachey, J., 2013, p. 377).

• Is the purpose consistent with the study’s overall approach?

Yes the purpose is consistent with the study’s overall approach. I would rate the research study as 3.

References

Parris, D., & Peachey, J. (2013). A Systematic Literature Review of Servant Leadership Theory in Organizational Contexts. Journal Of Business Ethics, 113(3), 377-393. doi:10.1007/s10551-012-1322-6