Dr. Paul Parkison, associate professor and chair of Childhood Education, Literacy, and TESOL (CELT), led his program to develop a new process of preparing evidence for the programmatic assessment process in 2017-18. CELT's Structure of Evidence Report guidelines emphasize the use of five questions that faculty use to systematically evaluate evidence for use in assessment for university, accreditation, and state licensing purposes:
Structure of Evidence Report
[Excerpted from CELT's 2017-18 ALC]
the evidence questions
After each assessment measure is developed and implemented, the Program
will be prompted to respond to five questions about each measure item. The five
- What is this item of evidence?
- How was the quality of the evidence determined
- What criteria of success have been established
on the measure, and how?
- What does the reported evidence mean?
- How is the evidence used to support improvement?
- In response to question 1,
the program will provide a description of the evidence, its features, and
other relevant characteristics or contextual information. Remember to keep
the response to question 1 focused on the most critical information
necessary to understand what the evidence contains or illustrates.
- In response to question 2,
the program will provide a statement of how the quality of the evidence is
monitored. Remember to provide accurate information, as detailed as
possible, about any of the criteria for evidence quality that are
- In response to question 3,
the program will explain for each measure it uses what level of performance
counts as ‘success’—as meeting its own expectations. The response to
question three should make clear what level of performance the program
regards as sufficient and why that standard or criterion of success makes
sense. Empirical evidence used in establishing the criterion should be
shared as part of the explanation.
- In response to question 4,
the program will provide an interpretation of the evidence. Analysis of
trends over time, comparisons within and across programs or at transition
points, or benchmarking with peer institutions or other clinical
preparation programs is encouraged. Remember that reports are a self-study
and it is an opportunity to collect and reflect upon the evidence of
candidates’ performance comparatively and over time. Let the evidence
speak, but let it speak through your interpretation and the meaning that
you are making of it.
- In response to question 5,
the program will provide a statement about how the evidence is used or
what actions have been taken as a result. Remember that the point of
collecting and interpreting the evidence is to learn from it. Take this as
an opportunity to identify challenges, acknowledge successes, and open up
more opportunities to improve the programs and candidates’ performance.
CELT will use the five questions as a litmus test for what
constitutes powerful evidence to include and what may be less critical to make our
case for meeting the standard.
Composing summary standards related to each standard
When all the Student Learning Outcome evidence is collected
and analyzed, the program will provide a holistic summary statement of how the
measuresdemonstrate that the Program Educational Standards are met.
The summary statement will provide a consolidated overview
of the evidence collection in relation to the Program Educational Standards,
not the Student Learning Outcome components. There will be no requirement to
provide a statement of how each component is met. The focus is on the wording
of the standard itself, and the program’s summary statement should emphasize
the standard’s holistic and overarching expectation.
The summary statement should be an analysis and
interpretation of the evidence collection itself.
- What do you now know about
your candidates after compiling the collection?
- Compared to what?
- Do you have clear
indicators, how do you know?
- Are there gaps, how do you
- What does the collection
tell you, tell stakeholders, about how the standard is being met?
The holistic summary statement will provide an analysis of
the collection itself."