目录
C1 Evidence Standards for Education Research
- C1.1 [描述性拆分] Evidence to be Produced by Early-Stage or Exploratory Studies
- 页码：1-1
- C1.2 [描述性拆分] Table B-3: Design and Development Research
- 页码：2-2
- C1.3 [描述性拆分] Table B-4: Impact Research — Purpose and Three Study Types
- 页码：4-4
C1.1 [描述性拆分] Evidence to be Produced by Early-Stage or Exploratory Studies
第1段（P0001）
原文复述
Table titled “Evidence to be Produced by Early-Stage or Exploratory Studies.”
Row 1 — Project Outcomes: The project’s outcomes should include (1) empirical evidence regarding (a) the malleable factors’ association with education or learning outcomes and/or (b) evidence on whether the factors and conditions moderate and/or mediate the relations between the malleable factors and the learner outcomes; (2) (a) a well-specified conceptual framework that supports a theoretical explanation for the link between the malleable factors and the education or learning outcomes and/or (b) a theoretical explanation for the factors’ and conditions’ moderation and/or mediation of the relationship between the malleable factors and learner outcomes; and (3) a determination, based on the empirical evidence and conceptual framework, of whether there is a basis for pursuing a Design and Development Project or an Efficacy Study, or whether further Foundational, Early-Stage, or Exploratory Research is needed before proceeding to efficacy or effectiveness testing.
Row 2 — Research Plan: The research plan should include the following: (1) a set of hypotheses or research questions that are derived from the theoretical and empirical rationale for the study; (2) a detailed research design that is appropriate for the hypotheses or research questions; (3) a justification for the proposed research context and sample; (4) a description of the data sources if secondary analyses are proposed; (5) a detailed description of data collection procedures and instruments, as well as evidence of and strategies for ensuring reliability and validity; (6) if applicable, a plan to study the opportunities for interventions (i.e., programs, policies, practices or technologies) to address education and learning challenges; and (7) a detailed description of data analysis procedures and the reporting plan.
Row 3 — External Feedback Plan: The project should be subject to a series of external, critical reviews of its design and activities (including theoretical framework, data collection, analyses, and reporting). These review activities may entail one or more of the following: peer review of the proposed project; ongoing monitoring and review by the grant-making agency’s personnel; external review panels or advisory boards proposed by the project and/or the agency; a third-party evaluator; and peer review of publications and conference presentations resulting from the project. The external critical review should be sufficiently independent and rigorous to influence the project’s activities and improve the quality of its findings.
中文讲解
这一节是一张规范表，列明早期/探索性研究项目在立项与执行中必须交付的三类证据：项目产出（Project Outcomes）、研究计划（Research Plan）以及外部反馈计划（External Feedback Plan）。它实际上在告诉申请者：评审方不是只看一个想法，而是要看到一套结构化、可检验的承诺。
第一行"项目产出"的核心是三角证据结构：(1)要求经验证据，要么证明"可塑因素"（malleable factors，即可被干预改变的因素）与教育/学习结果之间的关联，要么证明这些因素/条件是否起调节或中介作用；(2)要求配套的概念框架或理论解释，使经验发现具有理论意义；(3)要求基于(1)和(2)做出阶段判定——是继续推进到设计与开发阶段、效能研究，还是需要回到更基础、早期或探索性研究再做一轮。这一项尤为关键，因为它把早期研究明确定位为"路径筛选器"，不允许跳过因果链条就贸然进入干预或效能测试。
第二行"研究计划"是研究方法的硬性清单：假设/问题必须来自理论或经验依据，研究设计必须与假设匹配，研究情境与样本必须有正当理由，二次分析需说明数据来源，工具与数据收集流程需说明信效度与保障策略，若涉及干预还需包含"干预机会"研究（如方案、政策、实践或技术），最后必须描述分析程序与报告计划。这一行体现了"方法透明、可重复、可被外部评议"的方法学底线。
第三行"外部反馈计划"强调独立而严格的外部评议。评议对象覆盖理论框架、数据收集、分析与报告全过程；可选形式包括同行评审、出资机构人员的持续监督、外部评审小组或顾问委员会、第三方评估人以及对出版物与会议报告的同行评审。该行的隐含主张是：早期研究的不确定性高，缺少外部审视容易让设计偏差演化为结论偏差，因此独立性（independent）与影响力（影响项目活动与产出质量）成为评审外部反馈的两条标准。
整体来看，本表把"早期/探索性研究"界定为一种理论-经验-判定三位一体的工作，并且强调研究全程的方法规范性与外部可问责性。它在本节乃至后续小节中的功能是：界定早期研究应有的产出与可审查性门槛，使评审有据可依，也使申请者知道不能仅凭初步观察就声称因果或效能。
补充说明
表内"Project Outcomes"一栏中，(1)与(2)分别对应经验层与理论层，而(3)将两者综合为阶段决策，因此三者构成递进关系，缺少任一环节都会被视为未完成早期研究任务。“Research Plan"中第(6)项以"if applicable"开头，表明"干预机会研究"只对可能涉及干预设计的项目才作硬性要求；纯描述性早期研究不必为此项提供计划。“External Feedback Plan"列举的五种形式以"may entail one or more of the following"连接，意味着项目可选择组合，但独立性（independent）与严格性（rigorous）必须足以"影响项目活动并改进产出质量”，这是不可让步的底线。
脚注 / 边栏 / 图表 / 附加信息（如有）
本节主体即一张三行两列的规范表，标题为"Evidence to be Produced by Early-Stage or Exploratory Studies”。表中无脚注、图、旁栏或其他附加材料。
💡 本节核心要点
- 早期/探索性研究的三类必备交付物：项目产出、研究计划、外部反馈计划，缺一不可。
- 项目产出要求经验证据与理论解释并重，并以二者为依据判定是继续推进、转向还是回到更基础阶段。
- 研究计划要求假设与设计匹配、情境与样本合理、工具信效度可证、分析透明，干预机会研究视情况纳入。
- 外部反馈计划要求覆盖全过程、可采用多种形式，但必须独立且严格到能实质影响项目活动与产出质量。
- 该表把早期研究明确定位为"路径筛选与基础论证"环节，禁止跳过因果链进入效能测试。
🔑 关键术语
- malleable factors：可塑因素，指那些有可能通过干预、政策或环境改变而影响教育或学习结果的因素；早期研究的目标之一就是确认哪些因素属于此类并解释其作用路径。
- moderate / mediate：调节与中介。调节指某些条件改变因素之间关系的方向或强度；中介指因素通过中间环节间接影响结果。早期研究要回答的正是这些因素/条件扮演哪种角色。
- Design and Development Project / Efficacy Study：设计与开发项目以及效能研究，分别指将研究发现转化为干预设计的阶段，以及在受控条件下检验干预效果的阶段；早期研究的判定(3)决定是否能进入这两个阶段。
- Foundational, Early-Stage, or Exploratory Research：基础性、早期或探索性研究，指尚未具备充分基础以进入设计与效能阶段的研究类型；判定(3)允许项目回到这一层继续积累证据。
- External Feedback Plan：外部反馈计划，指由出资机构、同行或第三方对项目设计、数据与分析进行持续独立评议的安排，目的是降低早期研究的不确定性与偏差。
C1.2 [描述性拆分] Table B-3: Design and Development Research
第1段（P0002）
原文复述
Table B-3: Design and Development Research
Purpose: The purpose of Design and Development Research is to develop new or improved interventions or strategies to achieve well-specified learning goals or objectives, including making refinements on the basis of small-scale testing. Typically this research involves four components: (1) Development of a solution (for example, an instructional approach; design and learning objects, such as museum exhibits or media; or education policy) based on a well-specified theory of action appropriate to a well-defined end user; (2) Creation of measures to assess the implementation of the solution(s); (3) Collection of data on the feasibility of implementing the solution(s) in typical delivery settings by intended users; and (4) Conducting a pilot study to examine the promise of generating the intended outcomes. In some cases, funders will expect all four stages to be completed within a single project; in other cases, Design and Development Projects may entail sequential projects.
Justification Guidelines for Design and Development Research
Policy and/or Practical Significance: The proposal for the project should provide a compelling rationale that (1) specifies the practical problem the proposed intervention intends to address; (2) justifies the importance of the problem; (3) describes how the proposed intervention or strategy differs from existing practice; and (4) explains why the proposed project has the potential to improve learning or education outcomes or increase efficiencies in the education system or institutional setting beyond what current practice provides.
Theoretical and Empirical Basis: The proposal should include a strong theoretical and empirical justification for development of the proposed intervention or strategy. If the theoretical basis rests on evidence related to individual features or components, the proposal should provide a compelling rationale for how combining these features or components into a new intervention is expected to achieve intended outcomes. The proposal should include a description of the initial concept for the planned investigation, including a well-explicated theory of action or logic model. The concept and logic model should identify key components of the intervention (i.e., the ingredients hypothesized to be critical to achieving the intended results) and should describe their relationships, theoretically, and operationally.
Evidence to be Produced by Design and Development Research
Project Outcomes: The project’s outcomes should include (1) a fully developed version of the proposed design-research (including all materials necessary for its implementation); (2) a well-specified theory of action, including evidence supporting or refuting key assumptions of the intervention’s original theoretical basis; (3) descriptions of the major design iterations and the resulting evidence to support or question key assumptions about the theory of action; (4) description and empirical evidence of the adjustments to the theory of action and intervention design that resulted from design testing; (5) measures with evidence of technical quality for assessing the implementation of the intervention in an authentic education delivery setting and data demonstrating the project’s success in such implementation; and (6) pilot data on the intervention’s promise for generating the intended beneficial learner outcomes.
Research Plan: The research plan should describe the (1) method for developing the intervention to the point where it can be used by the intended end-users (iterative development process); (2) method for collecting evidence on the feasibility that end users can implement the intervention in an authentic education or learning setting (evidence of feasibility of implementation); and (3) method for obtaining pilot data on the promise of the intervention for achieving the expected outcomes (pilot study).
External Feedback Plan: The project should be subject to a series of external, critical reviews of its design and activities (including theoretical framework, data collection, analyses, and reporting). These review activities may entail one or more of the following: peer review of the proposed project, ongoing monitoring and review by the grant making agency’s personnel, external review panels or advisory boards proposed by the project and/or the agency, a third-party evaluator and peer review of publications and conference presentations resulting from the project. The external critical review should be sufficiently independent and rigorous to influence the project’s activities and improve the quality of its findings.
中文讲解
本节呈现的是一张规范性表格（Table B-3），它把"设计与开发研究"（Design and Development Research）这一研究范式拆解为四个相互衔接的部分——目的（Purpose）、论证准则（Justification Guidelines）、应产出的证据（Evidence to be Produced）、外部反馈计划（External Feedback Plan）。理解这张表的关键在于：它不是描述一种理论观点，而是给申请者和评审者提供一套可操作的判定标准，因此表格中出现了大量带编号的清单和"should"这样的规范用语。
1. 目的与四个组件。作者把"设计与开发研究"的核心目标定为：发展出新方案或改进方案，以达成具体的学习目标，并允许在小规模测试基础上迭代精修。表格进一步把这一研究拆成四个组件：(1) 开发方案——可以是教学法、学习物件（如博物馆展品、媒体）或教育政策，开发时必须以"行动理论"（theory of action）为依据，并面向明确的"终端用户"（end user）；(2) 创建评估方案实施情况的测量工具；(3) 在典型应用情境中收集可行性数据；(4) 进行试点研究，检验是否真能产出预期结果。作者特意补充了一句：资助方可能要求四个阶段在一个项目内完成，也可能允许安排成连续的多项目——这表明该研究范式在项目管理上具有弹性。
2. 论证准则。这部分回答的是"什么样的立项才算合格"，包含两个维度：
- 政策/实践意义：提案必须明确指出所要解决的实际问题、证明问题的重要性、说明新方案与现行做法有何不同、并解释它为何有望带来超出当前实践的改进或效率提升。四个要点串成一条"问题—差异—价值"的论证链。
- 理论与实证基础：要求强有力的理论与实证依据；特别值得注意的是，当依据来自对单个组件的证据时，提案还须额外说明"把这些组件组合在一起"为何能产生预期效果——这是为了防止把分别有效的内容机械拼凑。同时要求提出一个well-explicated theory of action 或 logic model，并识别关键组件（即假定的"活性成分"）及其在理论与操作层面的关系。
3. 应产出的证据。这部分是表格中条目最多的一节，分三块：
- 项目产出：六项明确交付物——(1) 完整的设计研究成果（含实施所需的全部材料）；(2) 经过验证的行动理论（须有支持或反驳原始假设的证据）；(3) 重大设计迭代的描述与证据；(4) 由设计测试引发的对行动理论和方案设计的调整（含实证证据）；(5) 在真实教育情境中评估实施的测量工具（须有技术质量证据）以及实施成功的数据；(6) 关于方案能否产出预期学习者收益的试点数据。这六项实际上映射了"目的"中的四个组件，并把它们具象化为可被检查的产物。
- 研究计划：要求描述三种方法——迭代开发方法、可行性证据收集方法、试点研究方法，分别对应目的中的组件一/二/三/四。
- 外部反馈计划：强调外部同行评审的独立性。列举了五种可能的评审形式（同行评审、资助机构持续监督、外部评审小组或顾问委员会、第三方评估者、对发表成果的同行评审），并以一句规范语句收尾：评审须"足够独立和严格，以影响项目活动并提升其发现的质量"——这表明外部反馈不是装饰，而是要真正发挥作用。
4. 表格的论证逻辑。从"目的→论证准则→应产出的证据→外部反馈"的递进可以看出，该范式预设了一种实证-迭代-问责的研究观：先有清晰的行动理论，再通过小规模测试迭代，最后由外部独立评审来保证质量。它特别强调**end user（终端用户）和authentic education delivery setting（真实教育交付情境）**这两个词——表明设计与开发研究不是实验室里的纯学术活动，而是要落到真实使用者、真实场景中去检验。
易误解之处：不要把"Design and Development Research"误读为软件工程意义上的"设计与开发"。在教育研究语境下，它特指一类以学习目标为导向、以行动理论和迭代试点为方法论核心的研究范式。
补充说明
原文表内的小标题 “Justification Guidelines for Design and Development Research” 与 “Evidence to be Produced by Design and Development Research” 分别起到分节作用，把表格内容切分为"论证要求"与"产出要求"两块；它们本身不是论证内容，而是结构性标签。
表格中 “design and learning objects, such as museum exhibits or media” 的例子，是作者为说明"方案"（solution）的形式多样性所给的典型例证，意在表明设计方案不限于课程，也可以是展品或媒介；本节未提供这些例子背后的具体研究出处。
“(i.e., the ingredients hypothesized to be critical to achieving the intended results)” 是对 “key components” 的同位语解释，表明作者把"关键组件"等同于"假定的活性成分"，这是理解"理论依据是否充分"时的核心判断点。
脚注 / 边栏 / 图表 / 附加信息（如有）
本节为单一表格（Table B-3），无独立脚注、图表说明或边栏。表内含三处分节小标题（“Justification Guidelines for Design and Development Research”、“Evidence to be Produced by Design and Development Research”），仅起结构分隔作用，不构成独立论证。
💡 本节核心要点
- “设计与开发研究"被界定为以"行动理论”（theory of action）为依据、以明确终端用户为对象、通过四阶段（方案开发—测量工具—可行性数据—试点研究）达成具体学习目标的研究范式。
- 论证层面须同时具备"政策/实践意义"（问题—重要性—差异—价值）与"理论与实证基础"（含组合各组件的额外论证和明示的 logic model）。
- 应产出的证据包含六项交付物，分别对应目的中的四阶段，并明确要求"支持或反驳原始假设的证据"，意味着该范式容许甚至要求证伪式检验。
- 研究计划须显式描述迭代开发、可行性证据收集、试点研究三种方法，强调在"真实教育交付情境"中检验，而非依赖实验室或理想化条件。
- 外部反馈计划被设定为强制项，要求评审"足够独立和严格以影响项目活动"，并提供五种可组合的评审形式，使规范要求具备可操作性。
🔑 关键术语
- Design and Development Research：设计与开发研究：一种以发展或改进教育干预方案（含教学法、学习物件、政策）为目标，依赖行动理论、通过小规模迭代试点检验方案可行性与有效性、并产出可实施交付物的研究范式。
- Theory of action / Logic model：行动理论／逻辑模型：对"方案关键组件—作用机制—预期结果"之间因果关系的明示陈述；本节将其作为立项论证与项目产出的共同核心要件。
- End user：终端用户：本节特指方案最终由谁来使用（如教师、学生、馆方、政策实施者）；方案开发必须面向已界定的终端用户，而非泛化群体。
- Authentic education delivery setting：真实教育交付情境：方案实际投入使用、而非模拟或实验条件的教育场景；可行性证据与实施证据须在此类情境中获取。
- Iterative development process：迭代开发过程：通过"开发—小规模测试—调整"循环来完善方案的方法论；本节将其作为研究计划必须描述的方法之一。
- Pilot study：试点研究：在小范围内初步检验方案是否能产生预期学习者收益的研究；其数据是"项目产出"六项交付物之一。
- External critical review：外部关键性评审：由项目团队之外、具有足够独立性和严格度的力量对项目设计、理论框架、数据收集、分析与报告进行审视；本节将其设定为强制项，并列举五种可能的实施形式。
- Intervention：干预／干预方案：本节用作对所开发方案的总称，可指教学策略、学习物件或政策；论证准则要求说明其与现有做法的差异。
C1.3 [描述性拆分] Table B-4: Impact Research — Purpose and Three Study Types
第1段（P0003）
原文复述
Table B-4: Impact Research
Purpose The purpose of Impact Research is to generate reliable estimates of the ability of a fully developed intervention or strategy to achieve its intended outcomes. For an impact study to be warranted, the theory of action must be well established and the components of the intervention or strategy well specified. The three types of impact studies—Efficacy, Effectiveness, and Scale-up—differ with regard to the conditions under which the intervention is implemented and the populations to which the findings generalize. In addition, as the research moves from Efficacy to Scale-up, studies should also give greater attention to identifying variation among impacts by subgroup, setting, level of implementation, and other mediators. For all impact studies, descriptive and exploratory analyses should be sufficiently elaborated to determine the extent to which the findings support the underlying theory of action.
The purpose of Efficacy Research is to determine whether an intervention or strategy can improve outcomes under what are sometimes called “ideal” conditions. For example, these conditions may include more implementation support or more highly trained personnel than would be expected under routine practice, or in contexts that include a more homogeneous sample of students, teachers, schools, and/or districts than is typical. Efficacy studies may involve the developer in the implementation of the intervention or strategy; however, the study should include reasonable safeguards for ensuring the objectivity and integrity of the study. Sometimes Efficacy studies are used to replicate previous evaluations of an intervention, but under different conditions (e.g., with a different population or using a variant of the intervention or strategy).
The purpose of Effectiveness Research is to estimate the impacts of an intervention or strategy when implemented under conditions of routine practice. To this end, implementation should be similar to what would occur if a study were not being conducted. Importantly, an Effectiveness study should be carried out with no more developer involvement than what would be expected under typical implementation.
The purpose of Scale-up Research is to estimate the impacts of an intervention or strategy under conditions of routine practice and across a broad spectrum of populations and settings. That is, Scale-Up studies should be conducted in settings and with population groups that are sufficiently diverse to broadly generalize findings. As with Effectiveness Research, Scale-up Research should be conducted with no more developer involvement than what would be expected under typical implementation.
Justification Guidelines for Efficacy Research Policy and/or Practical Significance: The project proposal should provide a clear description of the intervention to be tested and a compelling rationale for examining its impact. The rationale should (1) specify the practical problem the intervention is intended to address; (2) justify the importance of the problem; (3) describe how the intervention differs from other approaches to addressing the problem; and (4) explain why and how the intervention will improve education outcomes or increase efficiencies in the education system beyond current practices or interventions. The proposal should justify the choice to examine the impact of the intervention under ideal implementation conditions with a well-defined sample, rather than under routine practice conditions with a relevant typical sample or under typical implementation conditions with a broad sample. It also should describe the implementation setting(s) and population group(s) relevant to current and prospective policy or practice.
Theoretical and Empirical Basis: Efficacy Research should be justified by one or more of the following: (1) empirical evidence of the promise of the intervention from a well-designed and implemented pilot study (e.g., a study conducted as part of a design and development project); (2) empirical evidence from at least one well-designed and implemented Early-Stage Research study supporting all the critical links in the intervention’s theory of action; (3) evidence the intervention is widely used even though it has not been adequately evaluated to determine its efficacy; or (4) if the intent is to replicate an evaluation of an intervention with a different population, evidence of favorable impacts from a previous well-designed and implemented efficacy study and justification for studying the intervention with the new target population.
Justification Guidelines for Effectiveness Research Policy and/or Practical Significance: The proposal for the project should provide a clear description of the intervention to be tested and a compelling rationale for examining its impact. The rationale should (1) specify the practical problem the intervention is intended to address; (2) justify the importance of the problem; (3) describe how the intervention differs from other approaches to addressing the problem; and (4) explain why and how the intervention will improve education outcomes or increase efficiencies in the education system beyond current practices or interventions. The proposal should justify the choice to examine the impact of the intervention under routine practice conditions with a relevant typical sample, rather than under ideal implementation conditions with a well-defined sample or under typical implementation conditions with a broad sample. It also should describe the implementation setting(s) and population group(s) relevant to current and prospective policy or practice.
Theoretical and Empirical Basis: Effectiveness Research should be justified by strong empirical evidence of the efficacy of the intervention, as demonstrated by statistically significant and substantively important estimates of impact, from one study that includes multiple sites or settings, or two studies that each include one site or setting, all of which meet the Evidence Standards for Impact Evaluations (Table 4) or evidence that the intervention is widely used even though it has not been adequately evaluated for efficacy.
Justification Guidelines for Scale-up Research Policy and/or Practical Significance: The project proposal should provide a clear description of the intervention to be tested and a compelling rationale for examining its impact. The rationale should (1) specify the practical problem the intervention is intended to address; (2) justify the importance of the problem; (3) describe how the intervention differs from other approaches to addressing the problem; and (4) explain why and how the intervention will improve education outcomes or increase efficiencies in the education system beyond current practices or interventions. The proposal should justify the choice to examine the impact of the intervention under typical implementation conditions with a broad sample, rather than under ideal implementation conditions with a well-defined sample or under routine practice conditions with a relevant typical sample. It also should describe the implementation setting(s) and population group(s) relevant to current and prospective policy or practice.
Theoretical and Empirical Basis: Scale-up Research should be justified by compelling evidence of the effectiveness of the intervention, as demonstrated by statistically significant and substantively important impact estimates from one study that includes multiple sites or settings, or two studies that include one site or setting, all of which meet the Evidence Standards for Impact Evaluations (Table 4). In addition, there should be no overriding evidence demonstrating a negative impact of the intervention.
Evidence to Be Produced by Impact Studies Project Outcomes: Efficacy, Effectiveness, and Scale-up reporting should include detailed descriptions of the study goals, design and implementation, data collection and quality, and analysis and findings, for example, as outlined in the What Works Clearinghouse author reporting guidelines (http://ies.ed.gov/ncee/wwc/documentsum.aspx?sid=235). The core findings from these studies should be reliable estimates of the intervention’s average impact. In some cases, it also will be possible and desirable to estimate impacts for sample subgroups defined by such characteristics as setting, population group, or cohort. Study reports should document implementation of both the intervention and the counterfactual condition in sufficient detail for readers to judge applicability of the study findings. When possible, these factors should be related descriptively to the impact findings. Study reports should discuss implications of the findings for the theory of action and, where warranted, make suggestions for adjusting the theory of action to reflect the study findings. If a favorable impact is found, the project should identify the organizational supports, tools, and procedures that were key features of the intervention implementation. If no evidence of a favorable impact is found, the project should examine possible reasons (e.g., weaknesses in the implementation, evidence that raises questions about particular aspects of the logic model).
Research Plan: The research plan should identify and justify (1) the study design used to estimate causal impact of the intervention on the outcomes of interest; (2) the key outcomes of interest for the impact study and the minimum size impact of the intervention that would have policy or practical relevance; (3) the study setting(s) and target population(s); (4) the sample, including the power it provides for detecting an impact; (5) the data collection plan, including information about procedures and measures, including evidence on and strategies for ensuring reliability and validity, and plans for collecting data on program implementation, comparison group practices, and study context; and (6) the analysis and reporting plan. Efficacy, Effectiveness, and Scale-up studies should use study designs that will yield impact estimates with strong causal validity and that, for example, could meet What Works Clearinghouse standards without reservations (http://ies.ed.gov/ncee/wwc/). Generally and when feasible, they should use designs in which the treatment and comparison groups are randomly assigned. For Impact Studies (as opposed to Design and Development Research), quasi-experimental designs, such as matched comparison groups or regression discontinuity designs, are acceptable only when there is direct compelling evidence demonstrating the implausibility of common threats to internal validity. These might include selection bias in the case of matched comparison groups, or, in the case of regression discontinuity designs, nonlinearities of treatment impacts over a policy relevant span around the “cut point.” Ideally, the study sample size and allocation to condition should be such that the minimum true impact detectable size with 80 percent power and a 95 percent confidence interval is no larger than the minimum relevant size impact for policy or practice. If that is not the case, the proposal should provide a rationale for conducting the study despite its not meeting this standard. Primary outcome measures should include student outcomes sensitive to the performance change the intervention is intended to bring about (e.g., researcher-developed measures that are aligned with the experiences of the treatment group), student outcomes not strictly aligned with the intervention, and student outcomes of practical interest to educators and policymakers (e.g., test scores, grades, graduation or dropout rates). These outcomes should be pre-specified, have been demonstrated as reliable and valid for the intended purposes, and based on data-collection methods that have been shown to yield reliable data. The project should measure the strength and qualities of implementation (sometimes referred to as “fidelity of implementation”) to address whether the intervention’s impact estimates may be linked to how it was implemented. The project should measure comparison group practices and/or conditions to support a clear characterization of the contrast between the intervention and comparison condition. Proposals for Impact Studies should identify the measures, the validity and reliability of these measures, and how data will be collected on these measures. The analysis plan should specify analytic models that reflect the sample design and maximize the likelihood of obtaining unbiased, efficient estimates of average impacts and the confidence intervals around those impacts. The analysis plan should describe additional analyses conducted to explore variability in the intervention’s impacts and possible implications for the theory of change. For example, these analyses could include (1) subgroup analyses (expected in Effectiveness and in Scale-up Studies); (2) exploration of co-variation in impact estimates and fidelity of implementation or intervention contrasts; and (3) evidence of possible moderator and mediator effects.
External Feedback Plan: The project should be subject to a series of external, critical reviews of its design and activities (including theoretical framework, data collection, analyses, and reporting). These review activities may entail one or more of the following: peer review of the proposed project, ongoing monitoring and review by the grant making agency’s personnel, external review panels or advisory boards proposed by the project and/or the agency, a third-party evaluator and peer review of publications and conference presentations resulting from the project. The external critical review should be sufficiently independent and rigorous to influence the project’s activities and improve the quality of its findings.
中文讲解
本段是一张大型表格(Table B-4)，标题为"影响研究"(Impact Research)，系统地规定了影响研究的总体定位、三种研究类型(Efficacy、Effectiveness、Scale-up)、各自的正当性论证要求，以及影响研究应当产出的证据。它是整份文件中承上启下的核心规范表：上承早期/设计开发研究的结论，下启因果推断研究与证据标准的具体操作。
一、总体定位(Purpose 总论)。影响研究的目的被严格限定为：对一个"已充分开发"(fully developed)的干预或策略产生其预期结果的能力，给出"可靠估计"(reliable estimates)。这一句话包含两层关键闸门：第一，必须先有成熟的"行动理论"(theory of action)；第二，干预的各组成部分必须被"明确界定"(well specified)。换言之，若理论和成分都不成熟，就不配做影响研究。三种类型之间的差异被精炼为两条轴线：实施条件(conditions of implementation)与可推广人群(populations to which findings generalize)。从 Efficacy → Effectiveness → Scale-up，研究推进的方向是"理想条件 → 常规实践条件 → 常规实践条件下的多样化样本"，并伴随一个附加要求：越往后，研究者越需要细致考察亚组(subgroup)、情境(setting)、实施层级(implementation level)及其他中介变量(mediators)上的影响变异。所有影响研究都必须配以足够详尽的描述性和探索性分析，以判断研究结果是否真正支撑行动理论——这是一种"事后回检"机制。
二、三种研究类型的差别。这一部分是整张表最关键的分类学。Efficacy(效力研究)在"理想条件"下判定干预"能否"改善结果。“理想"是一个具体而非抽象的概念：可能意味着比常规实践更多的实施支持、更高素质的人员、更同质化的学生/教师/学校/学区样本。允许开发者参与实施，但必须设保障以维持客观性与研究诚信。Efficacy 研究也常被用于"复制”(replicate)前次评估，但更换人群或调整干预变体。Effectiveness(效果研究)要求"实施过程与不开展研究时几乎一致"，并明确把开发者参与压到与常规实施相当的程度——它的本质是把 Efficacy 的"能不能"翻译成"在日常条件下能不能"。Scale-up(规模化研究)在常规实践条件之上又加一条：人群和情境必须足够多样，使结论可以广泛外推。它对开发者参与的限制与 Effectiveness 相同。值得注意的是：从 Efficacy 到 Effectiveness 再到 Scale-up，开发者参与从"允许但需保障"逐步收紧到"必须接近零"，这与研究问题从"内部效度优先"向"外部效度优先"的迁移是一致的。
三、正当性论证(Justification Guidelines)。三种类型的论证框架高度同构：每类都设"政策/实践意义"与"理论/实证基础"两栏。“政策/实践意义"栏的四条要求几乎相同——描述干预、说明它针对的实际问题及其重要性、说明它与其他途径的区别、说明它为何能改善教育结果或提升系统效率；其差异仅在于一句话：分别要求论证为何选择"理想实施+明确样本”、“常规实践+典型相关样本”、“典型实施+宽泛样本”，而非另外两种条件。换言之，每一种研究类型都必须显式地"自我排他"，说明自己为什么要选这条而非另外两条路径，这是对方法学严谨性的强约束。“理论/实证基础"栏则呈递进关系：Efficacy 可由"试点研究、早期阶段研究、广泛使用但缺乏评估的现状、跨人群复制"四类证据之一支持；Effectiveness 要求"效力研究的强实证证据”(一项多场所或两项单场所)，且需符合 Table 4 的证据标准；Scale-up 要求"效果研究的强实证证据"，并附加"无压倒性负面证据"的"消极条件"。这条递进链揭示了文档背后的累积性知识观——不允许跳跃式证据升级。
四、应产出的证据(Evidence to Be Produced)。这是对研究实施本身的工程化要求，分三层。“项目结果”(Project Outcomes)层强调报告必须涵盖目标、设计与实施、数据收集与分析，并以 What Works Clearinghouse(WWC)的作者报告指南为模板；核心成果是"平均影响的可靠估计"，并尽可能提供亚组影响；报告必须把干预实施与反事实(counterfactual)条件的实施都描述得足以让读者判断可应用性；若结果有利，须识别关键的组织支持、工具与程序；若结果不利，须检视实施弱点或逻辑模型问题——这是"无论结果好坏都必须解释"的诚信要求。“研究计划”(Research Plan)层是一份六要素清单：研究设计、关键结果与最低相关效应量、研究情境与目标人群、样本与统计检验力、数据收集与分析报告计划；并明确要求采用"强因果效度"的设计——优先随机分配，类实验设计(匹配对照、断点回归)只有在能直接证明内效度威胁(选择偏差、围绕"切点"附近的非线性)不存在时才被接受；统计上理想状况是以 80% 检验力与 95% 置信区间能检出的最低真实效应不大于"政策相关最低效应"；结果测量需分三类(与干预契合、与干预不契合、对政策与实践者有现实意义)，且必须事先指定、证明信效度；同时要求测量"实施忠实度"(fidelity of implementation)、比较组的实践与条件；分析计划需匹配样本设计，并探索亚组、实施忠实度、调节与中介效应。“外部反馈计划”(External Feedback Plan)层要求对项目设计与活动进行独立、严格、能真正影响项目走向的外部批判性审查，形式包括同行评审、资助机构持续监督、外部评审/咨询委员会、第三方评估者、对出版物与会议报告的同行评审。
五、整体结构与功能。整张表把"研究是什么 → 凭什么做 → 必须产出什么"三层逻辑压缩进同一表格，并通过三种类型的同构化处理(同样的政策/实践意义栏、同样的理论/实证基础栏结构)凸显"研究类型选择本身就是论证的一部分"。它同时是规范性文本与操作性清单的混合体：上接"理论必须成熟"的门槛，下接 Table 4(证据标准)与 WWC 标准，并嵌入网址链接作为工具参照。读者应当把这张表视为一份"影响研究准入与运行守则"，而不是一般性方法学介绍。
补充说明
本段内的几个关键术语关系(仅基于本表信息)：“行动理论”(theory of action)与"逻辑模型"(logic model)、“变革理论”(theory of change)在本表中交替出现，作用相似——它们都是检验结果是否"解释得通"的对照框架。“反事实条件”(counterfactual condition)指未接受干预的对照组的实际发生情况，必须与干预组的实施一样被详细记录，否则"平均影响"无法被读者判断其可应用性。“实施忠实度”(fidelity of implementation)与"研究计划"中的"实施强度与质量"测量直接呼应，是把"实施变量"从背景噪音升格为可分析对象的关键概念。表内两处提及的网址(ies.ed.gov/ncee/wwc/…)对应 What Works Clearinghouse 的报告与设计标准，是本表所引外部规范的来源；表内对 Table 4 的多次引用指代本文件中独立的证据标准表。“理想条件/常规实践条件"与"明确样本/典型样本/宽泛样本"这两组对照是贯穿全表的分类坐标，决定了三种研究类型的归属。
脚注 / 边栏 / 图表 / 附加信息（如有）
本段本身就是一张大表格(Table B-4)，包含三层信息：研究类型与目的、三类研究各自的正当性论证要求、以及影响研究必须产出的证据(项目结果、研究计划、外部反馈计划)。表格中嵌入两处 What Works Clearinghouse 网址作为外部规范来源(报告指南与研究设计标准)，并多次引用 Table 4(证据标准)，提示该表在文件中与独立的证据标准表互为参照。表格内重复出现的"政策/实践意义"四要素(描述干预—指出问题—区分替代方案—说明改进机制)在三种研究类型中结构一致，仅"实施条件+样本类型"的对比措辞不同，用以强制研究者对研究类型选择本身进行论证。
💡 本节核心要点
- 影响研究被严格限定为针对"已充分开发"的干预，前提是行动理论成熟且干预成分明确，三种类型(Efficacy、Effectiveness、Scale-up)沿"实施条件"与"可推广人群"两条轴线递进。
- 从 Efficacy 到 Scale-up，开发者参与逐步收紧、样本同质化逐步让位于多样化、亚组与中介分析要求逐步上升，反映"内部效度优先"向"外部效度优先"的迁移。
- 三类研究各自要求自我排他式论证：必须显式说明为何选择本类而非另两类的"实施条件+样本"组合，否则正当性不完整。
- 证据基础呈递进链：Efficacy 可由试点/早期阶段研究/广泛使用现状/复制前次效力支持；Effectiveness 要求多项符合 Table 4 的效力证据；Scale-up 要求效果证据且无压倒性负面证据。
- 影响研究应产出的证据被工程化为三层：报告必须描述干预与反事实两端的实施以服务可应用性判断；研究计划必须使用强因果效度设计，优先随机分配并辅以统计检验力论证；外部反馈必须独立且能实质影响项目走向。
🔑 关键术语
- Theory of action / Logic model / Theory of change：本表中交替使用的近义概念，指干预"如何产生结果"的因果逻辑链条。影响研究的结果必须被回检至该逻辑链条：有利则识别关键支持与程序，不利则检视逻辑模型或实施上的弱点。
- Counterfactual condition：未接受干预的对照组的实际实施状态。本表要求研究报告把对照组的实施与干预组一样详细记录，否则读者无法判断平均影响估计在何种真实条件下成立。
- Fidelity of implementation：实施忠实度，指项目实际按设计执行的程度。本表把它从背景变量升格为可测量变量，要求分析与影响估计相关联，以判定效应究竟来自干预还是来自实施偏差。
- Efficacy / Effectiveness / Scale-up Research：本表定义的三种影响研究类型。Efficacy 在"理想条件+明确样本"下判定干预"能否"奏效，允许开发者参与但需保障客观性；Effectiveness 在"常规实践+典型相关样本"下估计干预的实际影响，要求开发者参与接近常规；Scale-up 在"常规实践+多样化人群与情境"下追求结论的广泛外推。
- Quasi-experimental design / matched comparison groups / regression discontinuity：本表允许的替代随机分配的非实验设计。前提是能直接证明内效度主要威胁(匹配对照的选择偏差、断点回归在切点附近的非线性)在所研究情境中不成立；否则须用随机分配。
- What Works Clearinghouse (WWC) standards：本表指定参照的外部研究规范来源，涵盖作者报告指南与研究设计标准。研究者应以其"无保留”(without reservations)级别作为强因果效度的参考基准。
- Evidence Standards for Impact Evaluations (Table 4)：本文件中独立存在的一张证据标准表，被本表多次引用，作为 Effectiveness 与 Scale-up 研究所需"强实证证据"是否符合规范的判定依据。