LDRS 591 Activity 591
As a consumer of research reports, the most important thing(s) for me in the methods and results section of a high-quality quantitative research report is/are…
After reading the required chapters of the textbook for this unit, I realized that my criteria for evaluating a study have been vague. I now have a better idea what to look for in a high quality quantitative research. I have combined the criteria in chapters 5,6,7 of Plan-Clark and Creswell (2015) along with some personal criteria tat I feel strongly about.
Research design: The authors should clearly state their reasons for the choice of study and data collection methods.
Participants: For me this is the most important component of the study. The validity of the study depends to a large extent on how the researchers choose their participants. For a high-quality study it must be clear what attempts the researchers made to identify and eliminate any bias or sampling error, an explanation that the sample is representative of the population, and that the size of the sample is appropriate for the type of study conducted.
Data collection: The instruments that the authors use to collect data must be validated through prior research. If the authors use an instrument created by them, there should be a clear explanation by the authors how that instrument would help collect data for the variables they plan to study.
Data analysis and results: I will admit that my understanding of statistics is very basic, even after having read chapter 7 of the above textbook. I usually read the results and discussions section to understanding the outcome of the study. The results should clearly address the research questions and hypotheses. The results and discussion section should also be arranged in a comprehensive and orderly manner, addressing the research questions in a linear way.
Question: I find myself instinctively trusting studies that use validate instruments. Is it fair to judge a study as falling short if the authors create their own instrument/s , as long as they attempt to include all the variables they are studying ?
Reference
Plano-Clark, V., Creswell, J. (2015). Understanding research: A consumer’s guide. (2nd ed.). Boston, MA: Pearson.
I do the same thing. When the author uses a tool or strategy for research that was developed by another author and the author shows that at least one other research study has used that tool, I just assume it is a valid tool for research. Where as, when I see a tool/questionnaire created by the author, I am more likely to question its bias/validity. This practice is not the best practice, we should question the validity of all research methodology in order to understand variables that might have been missed that would alter the research, regardless of whether the tool, strategy, questionnaire, etc has been used in multiple research studies or is new to the one we are reading.
Excellent post Simarjit and great question! To provide my two cents on your question I think you always have to use your critical evaluation skills when examining the measures used in a study. If the authors have created a questionnaire for the study they should always provide a rationale for why it was created and describe how they are using it and provide the questions in an appendix. That way you can be the judge (as the reader) as to the validity of their measure. I think this information is just as important as the validity and reliability scores. If there is no rationale or purpose stated, then I would be more cautious about the results. Does that make sense?
Dr. Strong
Totally makes sense , Dr Strong. I guess critical thinking is key.
Thanks for the encouraging words!
Thanks for the comments Layla. I Agree with you when you say that we have to use critical thinking in deciding if the instruments used in a research study are credible or not, regardless of whether they are pre-validated or created by the authors themselves. I think context is also key here.