Unit 5, Assessment

Part A    

In the guidebook, Understanding Research by Plano-Clark and Creswell (2015), they provide the following explanation to describe the differences between Quantitative and Qualitative research:

“In quantitative research the research problem tends to call for:

• An explanation of the relationship that exist among variables,
• A measurement of trends in a population” (Plano-Clark and Creswell, 2015, p. 59).

“In qualitative research problems tend to call for:

• An exploration because little is known about the problem,
• A detailed description and understanding of a phenomenon” (Plano-Clark and Creswell, 2015, p. 59).

“In quantitative research, researchers identify research problems that call for an explanation of the relationships among variables” (Plano-Clark and Creswell, 2015, p. 59).

“Qualitative Research is best suited for research problems that call for the need to explore and learn from participants because important variables are unknown or insufficient for describing a phenomenon” (Plano-Clark and Creswell, 2015, p. 59).

Assessment of qualitative research

Research Report select written by Van Winkle, Allen, DeVore & Winston (2014).

1. The choice of the research design is appropriate and justified – Rating 3 – the research design was based on previous collection method and were tested against previous results for accuracy. The researchers utilized previously tested collection methods and resulted were tested against previous results for accuracy. As stated “The ESLB scale (Winston & Fields, n.d.) was chosen for its design, brevity, and reliability” (Van Winkle, Allen, DeVore & Winston, 2014, p. 74). Also chosen was Laschinger et al.’s (2010) CWEQ II, because it “extends an existing model of workplace empowerment and integrates Kanter’s (1977, 1993) theory of structural power in organizations and Spreitzer’s (1995) notion of psychological empowerment” (Laschinger et al., 2010, P.5). So yes the research design is appropriate and justified.
2. Good quantitative procedures are used to select and assign participants – Rating 2 – the researchers checked with an outside review board to receive approval prior to the data collection. The researchers selected adults 18 years of age, employed at small businesses. They also worked with a college and received permission to work with adult business students. While the procedure as to selecting participants was approved, it could have been improved by including a larger population base, participants from across the country and also diverse populations.
3. Good quantitative data collection procedures are used – Rating 2 – A survey link was sent to 156 participants and 130 of the surveys were received. Of the 130, 116 were useable. My reasoning for a score of 2 is because it is not clear how the survey was controlled to ensure that it was in fact the employees and not supervisors responding. For example if the survey was passcode protected it could have ensured the further step in ensuring the data collected was in fact from the intended participants. It was also not described why some of the respondent’s survey results were not useable.
4. Good quality data analysis procedures were used – Rating 2 – as indicated in the research paper “The Pearson Product-Moment (PPM) correlation coefficient (r) was used to measure the relationship between the participant’s perceptions of their supervisor’s servant leadership and overall empowerment, as well as each empowerment subscale” (Van Winkle, Allen, DeVore & Winston, 2014, p. 75). The data could have been better controlled by using a unique passcode to ensure the participants were the correct participants that were requested to participate in the study. A survey link was sent to participants, likely for efficiency but the report does not address control measures that were in place for accuracy.
5. Good quantitative results and conclusions are reported – Rating 3 – the researched provided a table indicating the result of their study and then further went on to discuss the results of the scores in more detail.
6. The study used a rigorous research design – Rating 2 – The research design could have been improved by including a larger population based, samples from different parts of the country and more information on the demographics of those that participated in the research.
7. The use of the quantitative research design addressed the study’s purpose – Rating 3 – As stated in The Relationship Between the Servant Leadership Behaviors of Immediate Supervisors and Followers’ Perceptions of Being Empowered in the Context of Small Business “The purpose of the study was to measure the relationship between followers’ perceptions of the servant leadership of their immediate supervisor and followers’ sense of empowerment in the context of small business” (Van Winkle, Allen, DeVore & Winston, 2014, p. 70). Also as quoted under the heading of Small Business, according to Wang and Pouziouris (2010) “There has been little written about small business leaders and servant leadership, and according to Wang and Poutziouris (2010), “research on the leadership in the small business domain remains immature” (p. 350). So yes the research addressed the studies purpose.

Overall score is 17/21 which indicates an adequate quality overall quantitative research report.

Part B

Quality criteria for data collection utilizing research by Van Winkle, Allen, DeVore & Winston (2014)…

1. The sampling strategy is appropriate and justified – Rating 2 – The researcher’s strategy was to utilize email to send the surveys out. There were no details about controls in place to ensure the surveys were completed by the right group. The strategy was not very detailed and appeared like the researchers were looking for a quick turnaround.
2. The sample size is appropriate and justified – Rating 3 – “The sample size of 116 resulted in a power of 100%, (using Howell, 2013) suggesting the sample size was more than adequate to support the correlation found” (Van Winkle, Allen, Devore & Winston, 2014, p. 76).
3. High quality instruments are used to gather data – Rating 1 – the instrument used it appears to gather data is an emailed survey. The survey was sent to business owners and employees recruited on campus. There are a number of concerns with this method in that there is not mention on how the surveys were controlled and that steps were put in place to ensure employees were able to complete the survey as opposed to business owners. There is also no mention of anonominity and so the participants may have not been truthful in their responses out of fear of owner retribution.
4. The data are gathered using ethical quantative procedures – Rating 3 – there does not appear to be any issues related to ethics.
5. The data are gathered using standardized quantative procedures – Rating 3 – yes the data included previously recognized quantative procedures and the results shared in a table.
6. The study has a high level of internal validity – Rating 3 – The study utilized numerous other quantitative studies and build on former knowledge. Also the study utilized former knowledge in reviewing the data and comparing result of previous data.
7. The study has a high level of external validity – Rating 3 – the authors are qualified with credential suitable for this type of report. The report is also published by a respectable and well known Journal publication which means the study would have undergone a peer review.

Overall score 18/21 indicating a high quality collection of quantitative data.

Part C

Evaluation of the data analysis and results utilizing research by Van Winkle, Allen, DeVore & Winston (2014)…

1. The data was rigorously scored and prepared – Rating 3 – The data was collected using a 5-point Likert scale and so the score was outlined prior to the participants responding to the survey.
2. Good descriptive analysis were conducted – Rating 3 – The analysis was well written and shared in the results, discussions and findings section. References were providing drawing a correlation to previous known knowledge and newly found knowledge.
3. Good hypothesis testing procedures were used – Rating 0 – It does not appear that the authors provided any hypothesis.
4. The results are comprehensive – Rating 1 – I felt the authors had the opportunity to research and provide more analysis then they did. The results shared were minimal and perhaps without having a number of hypothesis to be researched, it did not require extensive results to be shared. I feel like this is a missed opportunity.
5. The results include sufficient information – Rating 1 – again I did not feel the report provided sufficient information and left me wanting to know more such as how the results might have differed by gender, age and race.
6. The data analysis represents a good quantitative process – Rating 3 – the author’s analysis in the discussions and findings did correlate back to the quantitative process that was followed.
7. The results provide a good explanation of the study’s purpose – Rating 3 – the results did indicate that the study’s purpose was met.

Overall quality 14/21 indicating adequate quality results for the data analysis and results, likely pointing back to gaps in collection of data.

Part D

As a consumer of research reports, the most important aspects needed to consider in the methods and results section of a high quality quantitative report is citing literature that includes models of how to design a study and how to collect and analyze data. If a research paper utilizes previous successful methods, the information and results are more likely to be credible and measurable. Detailed information about how the information is collected and what controls are in place to ensure credible data is also equally important.

The collection of data appears to be time consuming and difficult for the average person to have access to collection models that ensure anonominity, security and reliability of the data in a quantitative report. The other skillset that is needed of researchers is mathematical skills that can explain the data in an anonymous way. Depending on the field of study this could potentially prohibit individuals from participating in a research report if not comfortable in a mathematical setting.

One thing that is evident in creating a credible qualitative research report is that it takes time to set up collection methods and to think through potential gaps in the research. Also taking time to think about a hypothesis in advance may help to shape what data is collected. In my current role as a leader, having the time to create a research report is not likely however having the ability to read research reports on industry trends is essential. In particular in reading research reports on leadership styles and trends that effect employee morale and productivity.

What kinds of research tools are available and accessible to an amateur researcher, for the collection of qualitative data?

References

Howell, D.C. (2013) Statistical methods for psychology (8^th ed.). Belmont, CA: Cengage.

Kanter, R.M. (1977) Men and woman of the corporation. New York, NY. Basic Books.

Laschinger, H., Gilbert, S., Smith, L., & Leslie K. (2010). Towards a comprehensive theory of nurse/patient empowerment: Applying Kanter’s empowerment theory to patient care. Journal of Nursing Management, 18, 4-13

Plano-Clark, V., & Creswell, J. (2015). Understanding research: A consumer’s guide (2nd ed.). Boston, MA: Pearson.

Spreitzer, G.M. (1995). Psychological empowerment in the workplace: Dimensions, measurement and validation. Academy of Management Journal, 38(5), 1442-1465. doi:10.2307/256865

 

Van Winkle, B., Allen, S., DeVore, D., & Winston, B. (2014). The relationship between the servant leadership behaviors of immediate supervisors and follower’s perceptions of being empowered in the context of small business. Journal of Leadership Education, 13(3), 70-82.

Wang, Y., & Poutziouris, P. (2010). Leadership styles, management systems and growth: Empirical evidence from UK owner-managed SMEs. Journal of Enterprising Culture, 18(3), 331-354.

Winston, B.E., & Fields, D. (n.d.) Development and evaluation of a new parsimonious measure of servant leadership. Manuscript under review.