Learning Activity 5.4
As a consumer of research reports, the most important thing(s) for me in the methods and results section of a high-quality quantitative research report is clarity. It is important to note that I will be discussing more the methods section than the results section as it is this one that confuses me more.
The methods section of an article is the area that is most at risk of confusing readers. It is easy to get bogged down in numbers and statistical wordiness. I will be the first to admit that before this course I often simply skipped this section and went on to the conclusion to find what the results were. Needless to say, it can be difficult not to get lost in this section! Even the article I looked at for this unit had some areas that were a bit too muddled for me “Laub (1999) and Miears (2004) reported that the OLA had Cronbach’s alpha coefficients for the six subscales of 0.90 or above” (p. 609). This is why clarity in the methods section is so important. Authors and researchers should strive to speak in lamens terms and clarify terms that are not seen in the field. In this case, the authors should have clarified what mathematical significance the words ‘alpha coefficient’ and ‘subscales’ mean.
The question I would like to ask is the following: Is it absolutely necessary to comprehend the methods section when the conclusion has a better chance of clearly stating what the findings were and their implications? For years I have done this and while it might make me a bad student I certainly don’t lose any sleep at night when I don’t understand the exact procedures. Even if understanding the methods section means being able to critique the work I find that I fear running the risk of looking stupid if I criticize words I’m not even sure mean.
Source:
Cerit, Y. (2009). The effects of servant leadership behaviours of school principals on teachers’ job satisfaction. Educational Management Administration and Leadership, 37(5), 600-623.
February 5, 2018 @ 10:10 am
I totally agree, with your thoughts/question, more often than not I do not understand the procedure they use to analyze the accuracy of their data and I just assume it is valid. I feel like the fact that this is academic research predicates a duty on the part of the author to analyze their own research to the best of their ability with the most honest answers possible. The fact that it is peer reviewed helps to keep them honest in their procedures even if we can’t, that being said, what happens when we are the ones that are reviewing them. It is our duty to understand the research methodology to the best of our ability. I have googled so many statistical analysis tests and types of questionnaires. In the last activity I had to look up the SPSS. So to answer your question, in general, No I don’t think we need to understand all the tools used, but we should have a better than basic understanding of them just in case we are called to double check their work.
February 7, 2018 @ 1:28 am
I agree with Rob. I found myself looking up a lot of terminologies this past unit. My background is certainly not in statistics or mathematics so some of the technical data analysis terminology I struggle with. As I mentioned in a previous comment I think the key way you discern the level of analysis you are going to undertake on an article is mostly based on its relevance to your own research interests. If there is an only minor correlation with your research question then your analysis may not be as thorough.
February 7, 2018 @ 10:21 am
Hi Jamie,
Thank you for your response. I would like to add a few more thoughts to Rob’s.
I think it is very important to read the methods section of a quantitative research study. The purpose of a quantitative study will vary depending on the research question being asked. For example, quantitative research is primarily used to 1) test the effect of a treatment (differences), 2) describe relationships among variables or 3) examine trends in a population. Quantitative studies are only valuable “if the methods can be replicated in future studies and/or expanded upon. Therefore, it is critical that quantitative studies include a meticulous procedural and statistical accounting from which future researchers could plausibly replicate the findings” (Munoz, 2011). If you are not evaluating the methods section of a quantitative study then you could potentially miss some “red flags” that address the credibility of the research itself. For example, participant selection, selection of instruments, missing data, missing steps in a procedure.
The results section should be explained in the discussion section and yes it is helpful if the authors interpret the results of a statistical procedure in the results section in plain language to help the reader understand what they have done. Not everyone does this in practice. In cases where you don’t understand what the researchers have done, I would recommend consulting the Plano-Clark and Creswell (2015) text. The authors have done an excellent job explaining the evidence that a researcher has used a good measure or instrument on pages 241 and 242, and also explains how to understand the procedures that researchers use to collect quantitative data on page 244. The text also provides an excellent table for recognizing common statistics for comparing groups (on page 266) and also for relating variables (on page 267). These tables provide you with the statistic name, symbol used, and how to interpret the results. I would also recommend page 265 under the bold title “Alpha level” to help you understand the criterion that researchers use to obtain a statistically significant result.
Dr. Strong