Description
Read the final research article provided by your instructor, and discuss it with your peers. Discuss the research question posed, the authors’ hypotheses, the research methodology, and the statistical concepts used. Consider the implications of the study. How did the authors interpret the results? Discuss the meaning and importance of the study.
Create a list of questions for those areas that you know little about, and save them for discussion in class. Include statistical questions, concepts, terminology, or any other area that may be confusing to you.
Complete the Research Evaluation Form.
Note: There may be questions you are not able to answer at this time, but do your best. Confer with your Learning Team for clarification and assistance.
Research Evaluation Form
Use this form for both your individual and Learning Team article reviews.
Name(s): ______________________________________________
Quantitative study: __________ Qualitative study: _____________
Location of Researcher’s Observation or Experiment
None _________ Home ________ Clinic _________ Lab __________
Residential facility _________ School _________ Hospital _________ Other __________
Were children involved? Yes _____ No ______ If your answer is yes, how many? _________
Introduction
Research hypothesis or research questions/problem behaviors: ________________________
_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________
__________________________________________________________________________________________________________________________________________________________
Operational definitions:
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Consider the following questions:
Is the need for the study clearly stated in the introduction?
_____________________________________________________________________________
Were the research questions and hypothesis clearly stated? Note that research questions are often presented implicitly within a description of the purpose of the study section.
Review of literature as relevant to the study: _________________________________________
_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________
_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Method
Sample size (total): ________________ Size per group or cell: _______________
Consider the following questions:
_______ Were the methods described so that the study could be replicated without further information?
Subjects
_______ Were subject recruitments and selection methods described?
_______ Were subjects randomly selected? Were there any biases in sampling?
_______ Were the samples appropriate for the population to which the researcher wished to generalize?
_______ Were characteristics of the sample adequately described?
_______ Are two or more groups being compared? Are they shown to be comparable on potentially confounding variables? If they are not comparable, is this incomparability handled properly?
_______ Was informed consent obtained?
_______ Was the size of the sample large enough for the number of measures and for the effect being sought?
Research Design
_______ Single group, time series study
_______ Multiple baseline (sequential) design: ______________
_______ Single group, no measurement
_______ Single group with measurement: Pre ______ During _____ Post _____
_______ Two groups classic experimental versus control group, randomly assigned
_______ (Quasi-experimental) two groups experimental versus control group,
not randomly assigned
_______ Correlation research, not manipulated, degree of relationship
_______ Descriptive research (qualitative study)
_______ Natural observation
_______ Analytical research
_______ Interview research
_______ Historical study
_______ Survey research
_______ Legal study
_______ Ethnography research
_______ Policy analysis
_______ Fieldwork research
_______ Evaluation study
_______ Phenomenology
_______ Grounded theory
_______ Protocol analysis (collection and analysis of verbatim reports)
_______ Case study, no measurement
_______ Case study, with measurement: Pre _________ During _______ Post _________
_______ Developmental research
_______ Longitudinal (same group of subjects over a period of time)
_______ Cross-sectional (subjects from different age groups compared)
_______ Cross-sequential (subjects from different age groups, shorter period of time)
_______ Correlation, more than two groups: control, treatment, and other treatment comparisons
_______ Factorial design, two or more groups: other treatment differences, no untreated controls
_______ Two or more dependent variables (MANOVA)
_______ Other design: __________________________________________________________
Consider the following Yes/No questions:
_______ If appropriate, was a control group used?
_______ Was the control method for the study appropriate?
_______ For what variable was being controlled?
_______ In the case of an experimental study, were subjects randomly assigned to groups?
Measures
Type of dependent measures or instruments used: ____________________________________
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Measurement or instrument validity information: _______________________________________
__________________________________________________________________________________________________________________________________________________________
Measurement or instrument reliability information: _____________________________________
__________________________________________________________________________________________________________________________________________________________
Consider the following Yes/No questions:
_______ For all measures—measures to classify subjects, dependent variables, and so forth—was evidence of reliability and validity provided, either through summarizing the data or by referring the reader to an available source for that information?
_______ Do the reliability and validity data justify the use of the measure? Specific evidence is particularly important if a measure is created just for this particular study.
_______ Do the measures match the researchers questions and hypothesis being addressed?
_______ In the case that different tasks or measures are used, was their order counterbalanced? Do the researchers analyze for potential order effects?
_______ Are multiple measures used, particularly those that sample the same domains or constructs but with different methods, such as self-report, rating scales, self-monitoring, or direct observation?
_______ If human observers, judges, or raters were involved, was inter-observer or inter-rater agreement (reliability) assessed? Was it obtained for a representative sample of the data? Did the two raters do their ratings independently? Was their reliability satisfactory?
Independent and Dependent Variables
Independent variables: ________________________________________________________
_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Dependent variables: __________________________________________________________
_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Data Analysis
Scales of Measurement:
Nominal _______ Ordinal _______ Interval _______ Ratio _______
What type of statistical techniques are used?
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
What type of tables and graphs are used?
_____________________________________________________________________________
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Consider the following Yes/No questions:
_______ Do the data fulfill the assumptions and requirements of the statistics?
_______ Were tests of significance used and reported appropriately, such as with sufficient detail to understand what analysis was being conducted?
_______ In correlation studies, did the researchers interpret low but significant correlations as though they indicated a great deal of shared variance between the measures? Are the correlations limited by restricted ranges on one or more measures? Are means and standard deviations provided so that you may determine this?
_______ Do the researchers report means and standard deviations, if relevant, so that the reader can examine whether statistically significant differences are large enough to be meaningful?
Limitations of the study: ________________________________________________________
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Discussion
Summary and conclusions (usefulness): ____________________________________________
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Consider the following Yes/No questions:
_______ Do the researchers discuss marginally or insignificant results as though they were significant?
_______Do the researchers over-interpret the data? For example, do they use casual language to integrate correlation findings or consider self-report of behavior to be equivalent to direct observation?
_______Do the researchers consider alternative explanations for the findings?
_______Do the researchers have a humility section that describes the limitations of the research?
_______Do the researchers point out aspects of subject selection, procedures, and dependent variables that limit generalizability of the findings?
Rate the writing style on a scale of 1 to 10:
Overall clarity? 1————-5————-10
Easy to read? 1————-5————-10
Organized? 1————-5————-10
Describe what you learned from the study:
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
__________________________________ ________________________________
Group member Group member
___________________________________ ________________________________
Group member Group member
___________________________________ ________________________________
Group member Group member
*Adapted from form created by Dr. Randy Buckner, University of Phoenix instructor