Read about Project ESSY (Equitable Screening to Support Youth)
In 2022, CSCH affiliates and Co-Principal Investigators Sandra Chafouleas (UConn), Amy Briesch (Northeastern University), Jacqueline Caemmerer (UConn), and Jessica Koslouski (UConn) received funding from the U.S. Department of Education Institute of Education Sciences to start the four-year Project ESSY.
Jackie Caemmerer recently spoke with us about the project:
What is Project ESSY and why this study?
ESSY (which stands for Equitable Screening to Support Youth) is a research project whose goal is to develop a whole child screening assessment that provides a more complete picture of students. Information from such a screener can assist in identifying supports for issues that arise both from within the child (maybe they need glasses to better see the Smart Board) and from within their context (maybe the child is tired because their housing is unsafe and they haven’t been sleeping). The goal is to develop a universal screener that is comprehensive but efficient and easy to use.
There have been a lot of screening assessments developed in the last 20 years but because they’re usually focused in a specific area (e.g. depression, reading), schools are often faced with trying to implement too many time-consuming assessments. Screening assessments also often focus on a child’s deficits. Our goal is to develop a screening assessment that incorporates contextually relevant information about the whole child and that addresses the personal biases of school staff to enable more equitable assessment practices. As you can imagine, creating a screener that covers all of this yet isn’t too cumbersome is a challenging task!
You’re using what’s called mixed methods and consequential validity-centered approaches to develop and validate the measure (making sure it’s giving accurate results). Can you tell us what that is?
In typical measure development, the voices of those who will use the measure, or those affected by measure use, are not included. As a result, measures might be irrelevant, might not be feasible, or might have harmful consequences (bias, disproportionality, etc.). By the time intended users can give feedback, it’s too late to make changes because the measure has already been through the validation process.
In Project ESSY we are involving the people this screening assessment affects the most throughout our development of the measure. From the start of the project in 2022 we have relied on feedback from two advisory boards. A multi-disciplinary board (made up of experts from health education, neuroscience, school psychology, community partnerships, screening tools, and special education) has helped us to develop the domains and items in the measure. A school-based personnel board (made up of school and district administrators, school psychologists, behavioral and health specialists) has helped us confirm the domains as well as identify aspects of the measure that could introduce bias and has helped us think about usability. This board has also been key to developing relationships with families, community members, students, and teachers to gain additional input and feedback.
We have also been looking at the measure’s consequential validity (the intended and unintended consequences of measure use) throughout the measure development process. This is a change from typical measure development which has mostly ignored consequential validity. We have started using this ten-step approach to make sure that we are examining those consequences throughout the development process:
(Click on the image to view an accessible pdf version.)
This integrated approach provides us with multiple opportunities to revise the measure and associated guidance regarding its use.
How is it going so far?
We have gone through three intentional drafts of the measure. In addition to receiving feedback from the two advisory groups, we have completed focus groups and interviews with school-based professionals, families, and students to ask for their feedback on content, relevance and usability. It’s a lot of work but overall things are going well!
What’s Next?
As we enter year three our focus is on steps 6 through 8 of that 10-step framework. We are recruiting many teachers to rate their students using our measure. Their ratings and feedback will be used to refine the measure again. We are also planning for how the data from the measure will be used in the real world. We will test different data reports that summarize the data. We will use users’ feedback to design a data report that best supports decision making and generates positive consequences for students.
What have you learned about as part of this project?
My prior research projects have had a heavy emphasis on quantitative methods using previously collected data. Through the mixed methods approach of Project ESSY I have seen first-hand the value and richness of integrating quantitative and qualitative data. Also, I’m seeing all the effort and relationship building that is key to recruiting research participants and collecting data.
For More Info:
Check out the Project ESSY website, which includes videos and presentations about the project.