Assignment - Evidence Review

General Guidelines

This assignment requires students to evaluate and summarize research evidence on a specific policy topic assigned by the instructor. Using an evidence hierarchy (or levels of evidence), students will assess the strength of the evidence—such as the effectiveness of policies, programs, or actions—based on empirical research. The findings must be communicated in a concise and accessible format intended for a non-expert audience, such as policymakers or community stakeholders. The target (hypothetical) audience for your report and presentation is the legislators and the Governor of North Carolina. Your work should be tailored to this audience, focusing on clarity, relevance, and actionable insights.

Assignments % of Final Grade Due Date
01 Evidence Review Report
 15% 02/24
02 Evidence Review Presentation
 10% 02/27
03 Group-to-Group Feedback
 5% 03/06

# Part 1

Evidence Review Report

Working in groups of 3–4, students will review 7–11 empirical, peer-reviewed papers and produce a 2–4 page, single-spaced report documenting evidence on a specific state policy. A group of 4 students will be required to review at least 10 papers. The policy topic will be provided by the instructor during class. Students are not allowed to discuss this assignment outside of their group.

Components of the Evidence Review Report
  • Title and Introduction: Include a title and the names of all team members. The introduction outlines the policy context and scope of the evidence review. Clearly define the purpose of the review and provide relevant background information.
  • Results: Summarize the key findings of the evidence review, including the types of outcome measures examined, the effects and outcomes of the policy, and an evaluation of the methodological quality of the reviewed studies. While these components do not need to be separate sections, organizing findings in tables is highly encouraged, as illustrated in Tables 1 and 2 in Graham-Rowe, Skippon, Gardner, and Abraham (2011).
  • Discussion: Summarize the overall methodological rigor of the evidence base and discuss the implications for policy formulation, implementation, and development.
  • References: Provide a complete list of all cited studies at the end of the report. Footnotes are not allowed for the references.
  • Supplementary Materials: ubmit a separate directory containing PDFs of all reviewed papers, labeled as 'LastNameoftheFirstAuthor_YearofPublication'
Evaluation Criteria
  • Formats (10%): Includes the required number of reviewed papers (9–12). (2) Provides a separate directory containing PDFs of all reviewed studies, labeled correctly as 'LastNameoftheFirstAuthor_YearofPublication'. (3) Adheres to the specified length (3–5 pages, single-spaced) and formatting requirements.
  • Content Quality (30%): (1) Clearly defines the policy context and purpose of the review. (2) Summarizes the types of outcome measures examined in the reviewed papers accurately. (3) Evaluates the methodological quality of the reviewed studies with accurate and appropriate evidence. (4) Provides a concise assessment of the overall methodological rigor of the evidence base. (5) Offers insightful and well-supported implications for policy formulation, implementation, and development.
  • Diversity of Evidence (10%): The review incorporates a balanced mix of studies, including those with varying methodological quality (including experimental or quasi-experimental designs).
  • Use of Evidence (30%): (1) Selected studies are relevant to the policy topic and aligned with the review’s objectives. (2) Critically evaluates evidence rather than simply summarizing it. (3) Findings are accurately presented and supported with appropriate citations and references.
  • Clarity, Organization, & Professionalism (20%): (1) Submission is completed on time and reflects effort in preparation. (2) Writing is clear, precise, and well-organized, ensuring readability for a non-expert audience. (3) Report is logically structured, free of grammatical and typographical errors, and exhibits a professional tone.
Criteria Excellent (95%+) Good (75%+) Fair (40%+) Needs Improvement Weight
Formats Meets all formatting requirements, includes the required number of papers, and provides a well-organized directory of labeled PDFs. Mostly meets formatting requirements with minor deviations or minor issues with the directory of PDFs. Some formatting requirements met, but several key elements are missing or incorrect. Fails to meet most of formatting requirements or provide a directory of labeled PDFs. 10%
Content Quality Clearly defines policy context, accurately summarizes outcome measures, evaluates methodological quality thoroughly, and offers insightful implications. Defines policy context and summarizes outcome measures well but lacks depth or clarity in methodological evaluation or implications. Provides a basic policy context and outcome summary but lacks detail or clarity in evaluation and implications. Fails to define policy context clearly, inaccurately summarizes findings, and lacks meaningful evaluation or implications. 30%
Diversity of Evidence Demonstrates a balanced selection of studies, including those with varying methodological quality, and incorporates 3 or more papers employing experimental or quasi-experimental designs. Includes a mix of studies with varying methodological quality, with at least 2 papers using experimental or quasi-experimental designs. Shows some diversity in methodological quality but lacks sufficient representation of experimental or quasi-experimental designs. Relies heavily on a single methodological approach, with minimal or no inclusion of experimental or quasi-experimental studies. 10%
Use of Evidence Studies are highly relevant and critically evaluated, with findings presented accurately and supported by appropriate citations. Studies are relevant and evaluated adequately, with findings mostly accurate and supported by citations. Studies are somewhat relevant but lack critical evaluation, with limited accuracy in findings or incomplete citations. Studies are irrelevant or poorly evaluated, with findings inaccurate or unsupported by citations. 30%
Clarity, Organization, & Professionalism Writing is clear, precise, and free of errors; the report is logically structured, professional, and accessible to a non-expert audience. Writing is mostly clear with minor errors; the structure is logical but could be improved, and the tone is generally professional. Writing has noticeable errors; structure and tone lack consistency or professionalism, reducing readability. Writing is unclear, contains many errors, and is disorganized or unprofessional. 20%

# Part 2

Evidence Review Presentation

Overview
  • Each team will deliver a 12-minute presentation of their review, followed by a 5-minute Q&A session. The presentation must clearly and effectively communicate the team’s findings and analysis to the hypothetical audience of legislators and the Governor of North Carolina.
  • Teams may present directly from their report. Creating a separate slide deck is optional. But using visual aids such as charts, tables, or infographics are recommended to illustrate key findings and engage the audience effectively.
  • The presentation should address the policy context and objectives, summarize key findings, evaluate the methodological quality of the evidence, discuss the policy implications, and conclude with the key takeaways. The emphasis should be on the findings of your review rather than the background.
  • Teams are encouraged to focus on clarity and engagement, tailoring their language and explanations to ensure they are accessible to non-experts. Rehearsing as a team is highly recommended to manage time effectively and ensure smooth transitions between speakers. During the Q&A session, all team members should be prepared to respond to questions with clarity, confidence, and a deep understanding of the reviewed evidence.
Evaluation Criteria
  • Content and Evidence (40%): The presentation summarizes key findings from the reviewed papers, critically evaluate the methodological quality of the evidence, and explain the policy implications and recommendations. The discussion should be accurate, thorough, and directly relevant to the policy context.
  • Demonstrating Knowledge of Reviewed Papers (20%): Show a clear understanding of the reviewed papers and respond to audience questions confidently and competently during the Q&A session. Answers should reflect a deep analysis of the evidence.
  • Audience Engagement and Accessibility (25%): Present findings clearly and effectively, drawing connections to policy implications and tailoring the content for a non-expert audience. Use clear language. Include visuals such as charts, tables, or infographics to enhance understanding. Ensure the presentation flows logically with smooth transitions between sections.
  • Respectful Communication and Professionalism (15%): Maintain professional, respectful, and inclusive language throughout the presentation. Foster a constructive atmosphere and adhere to the designated time limit.
Criteria Excellent (95%+) Good (75%+) Fair (40%+) Needs Improvement Weight
Content and Evidence Summarizes findings accurately, evaluates methodological quality thoroughly, and explains policy implications clearly. Mostly accurate and thorough but lacks some detail in methodological evaluation or policy implications. Somewhat accurate but lacks depth in evaluation and relevance to policy context. Findings are inaccurate, discussion lacks relevance, and methodological evaluation is poor. 40%
Demonstrating Knowledge Clearly understands reviewed papers, responds confidently, and reflects deep analysis of evidence during Q&A. Understands reviewed papers well, responds adequately, but with minor gaps in analysis during Q&A. Some understanding of reviewed papers; responses are vague or lack depth during Q&A. Poor understanding of reviewed papers; unable to answer questions effectively. 20%
Audience Engagement and Accessibility Presents findings clearly with effective visuals and accessible language tailored to a non-expert audience. Presents findings well but visuals or language could be improved for accessibility and engagement. Somewhat engages audience; visuals and language are not consistently accessible. Fails to engage audience; visuals and language are poorly executed and inaccessible. 25%
Respectful Communication and Professionalism Maintains professionalism, inclusive language, and adheres to the time limit perfectly. Professional and respectful with minor issues in inclusivity or time management. Some professionalism, but noticeable issues in respect, inclusivity, or adherence to time limits. Lacks professionalism, inclusivity, or adherence to time limits. 15%

# Part 3

Group-to-Group Feedback

Overview
  • Each team will receive another team’s evidence review report and the evaluation rubric 2 days prior to the presentation. Teams are expected to evaluate the report and presentation using the provided evaluation rubric, focusing on key areas such as content clarity, accurate assessment of the methodological quality of reviewed papers, and overall professionalism.
  • Each team will submit their evaluation of the assigned report and presentation within 7 days of the presentation. Feedback must be constructive, specific, and aimed at helping the presenting team improve their work.
  • The evaluation will include two components:
    1. A structured (numerical) evaluation based on the rubric.
    2. Written feedback addressing:
      • Usefulness of the evidence review: (a) Was the evidence review clear, accurate, and directly relevant to the policy context? (b) Did the report and presentation effectively communicate the findings talored to the policymakers or community stakeholders?
      • Areas for improvement: (a) Identify specific opportunities to enhance the report or presentation, focusing on improving content clarity, analytical depth, or professional delivery. (b) Highlight any missing elements or weaknesses that, if addressed, could improve the overall quality and impact of the evidence review.
    The written feedback should be concise yet detailed, approximately 2–4 paragraphs (within one page).
  • Evaluations should be written from the perspective of a hypothetical audience member (e.g., NC legislator or the governor). Reviewer identities will remain anonymous, but evaluations and comments will be shared with the presenting team.
Evaluation Criteria
  • Thoroughness (45%): Evaluators critically examine the presenting team’s findings, with particular attention to the methodological quality of the reviewed papers. The assessment is comprehensive and accurate, reflecting a deep understanding of the work. Numerical evaluations are consistent with the criteria outlined in the rubric and align with the written feedback.
  • Clarity and Professionalism (30%): Feedback demonstrates a clear and comprehensive understanding of the report and presentation. Written feedback is well-organized, concise, and free of grammatical or typographical errors. The tone is respectful and professional, maintaining the perspective of a hypothetical audience.
  • Constructiveness (25%): Feedback provides clear, specific, and actionable suggestions for improvement. It effectively addresses both the strengths and weaknesses of the report and presentation, offering meaningful insights for the presenting team.
Criteria Excellent (95%+) Good (75%+) Fair (40%+) Needs Improvement Weight
Thoroughness Comprehensive, insightful evaluation of findings and methodological quality; numerical evaluations align perfectly with written feedback. Thorough evaluation with minor gaps; numerical evaluations mostly align with written feedback. Evaluation lacks depth or misses key aspects; numerical evaluations partially align with written feedback. Evaluation is superficial, incomplete, or inconsistent with rubric criteria. 45%
Clarity and Professionalism Feedback is exceptionally clear, well-organized, and professional, with no errors. Feedback is clear and professional, with minor organizational or grammatical issues. Feedback lacks clarity or professionalism; noticeable grammatical or organizational issues. Feedback is unclear, disorganized, or unprofessional, with frequent errors. 30%
Constructiveness Feedback is highly actionable, offering specific, meaningful, and balanced suggestions for improvement. Feedback is actionable and offers useful suggestions, but may lack specificity or balance. Feedback provides limited actionable suggestions, with vague or generic comments. Feedback lacks actionable suggestions and fails to address key strengths or weaknesses. 25%