ATE Evaluation Task Validation Study

What problem does this study address?

In order for EvaluATE to do its job well, we need a comprehensive and accurate understanding of what ATE evaluation work really entails. To that end, this study identifies the core tasks that are conducted as part typical ATE evaluations.

Why is this study important?

This study builds on the work of the American Evaluation Association (AEA) and other organizations that have developed taxonomies of evaluation competencies. The AEA competencies served as the foundation for the ATE Evaluation Task Framework. This study will help translate general evaluator competencies to the specific knowledge and skills needed by ATE evaluators and project staff.

How will EvaluATE (or others) use the findings from this study?

We will use the finalized ATE Evaluation Task Framework to plan and prioritize EvaluATE’s training activities and resource materials. This framework will help us ensure that we are providing support and guidance for all core evaluation tasks.

Others engaged in evaluation capacity-building and research on evaluation can use our Framework as a starting point for similar validation work in other evaluation contexts.

How are the researchers conducting this study?

We are using two methods in this study:

• Task Tracking: We asked ATE evaluators and principal investigators to track their evaluation-related tasks for one year. We will use the task-tracking to revise the framework to ensure it’s a comprehensive and accurate accounting of what’s involved in a typical ATE evaluation.
• Delphi Study: We asked people who are recognized as experts in evaluation to review and provide feedback on the task framework. We will finalize the framework based on their feedback.

These methods will provide evidence for answer these research questions:

1. What are the key decisions and actions involved in ATE evaluation?
2. What types of professionals are involved in those decisions and actions?
3. What aspects of ATE evaluation do PIs and evaluators find especially challenging?

We’ll post study findings here when they’re available. In the meantime, you can learn more about what evaluators should know and be able to do by reviewing the American Evaluation Association’s Evaluator Competencies.