Core Considerations for Selecting a Screener
Screening is an important part of systems for early identification of students who are at risk for poor education outcomes. Screening is an important step in providing an evidence-based intervention to learners who need additional educational supports.
Screening processes typically include brief, reliable, and valid assessments that are administered to whole classrooms of students. Students are then typically classified into one of three groups based on their scores:
- those who are low risk – typically greater than an 80% chance of meeting expectations on a later assessment
- those who are moderate risk – typically a 50% chance of meeting expectations on a later assessment
- those who are high risk – typically less than a 20% chance of meeting expectations on a later assessment
There are many available screeners for reading and other education or social-emotional outcomes. What are the important things to consider when choosing and using a screener?
Choosing a Screener
The following technical and usability characteristics are important to consider when selecting a screener.
Population of Interest
It is important that the screener was designed for a population of students that is similar to yours. A well-defined population of interest is the critical foundation for evaluating whether a screener is appropriate for your students and setting.
The description of the population of interest in the assessment manual should be specific enough to tell you:
- Whether the sample that the screener was normed on reflects your intended population.
- The outcome that the screener is designed to identify. For example, the intended outcome may be to identify students with dyslexia or students with language disorders.
- The intended age range for the screener.
Scope of Assessment
It is important to think about the alignment between the outcome that the screener is designed to identify and the content measured by the screener. It is also important to consider if the screener is a timed assessment, such as curriculum-based measurements that measure speed and accuracy, or a skill-based assessment, such as computer-adaptive assessments. There are trade-offs between the two types of screeners that need to be considered, such as administration time and information provided. Finally, it is important to consider if the screener is univariate and measures one skill or multivariate and measures multiple skills.
The most basic definition of reliability is the consistency of a set of scores for a measure. Different forms of reliability that are reported in technical manuals for screeners include internal consistency, alternate-form, test-retest, split-half, inter-rater. A careful evaluation of each type of reliability will allow you to determine how consistent the scores from the screener are.
It is important to look at the different types of validity that are reported in technical manuals for screeners because they tell you the extent to which the assessment measures what it intends to measure. For example, a word reading screener should measure word reading and not receptive vocabulary. Evidence of validity might include predictive correlations with another measure.
The correct identification of students who are at risk and not at risk for poor outcomes is a critical consideration in evaluating the quality of a screener. When looking at the classification accuracy of a screener, such statistics include:
- the sensitivity of scores – the ability of the screener to correctly identify those who will not meet an expected threshold of performance on a later assessment. For a reading screener, sensitivity is the ability to identify students who are at risk for reading difficulties (true positives).
- the specificity of scores – the ability of the screener to correctly identify those who will meet or exceed an expected threshold of performance on a later assessment. For a reading screener, specificity is the ability to identify students who are not at risk for reading difficulties (true negatives).
- the false positive rates – how many children are identified as being at risk but are not actually at risk, and
- the false negative rates – how many children are actually at risk but are not detected with the screener.
Barriers and Access For Screeners
When selecting a screener for your school or district, it is also important to think about administrative and environmental considerations for use. For example, the administration format may be a barrier in choosing a particular type of assessment based on whether the screener is given on an individual or group basis, or whether it requires the use of certain technology, like computers. The choice of a screener should also be informed by the administration and scoring time and the scoring format (i.e., scoring done by school personnel or scoring done by a computer).
The components above should all be considered when choosing a screener. The number of considerations when evaluating, choosing, or using a screener can seem overwhelming. When reviewing a screener technical report, tool chart, or summary of the assessment, we recommend examining each of these core considerations to guide your discussions. Screeners lacking essential characteristics can be eliminated from consideration for use, making screener selection easier.
National Center on Improving Literacy (2022). Core considerations for selecting a screener. Washington, DC: U.S. Department of Education, Office of Elementary and Secondary Education, Office of Special Education Programs, National Center on Improving Literacy. Retrieved from http://improvingliteracy.org.
Petscher, Y., & Suhr, M. P. (2022). Considerations for choosing and using screeners for students with disabilities. In C. Lemons, S. Powell, K. Lane, & T. Aceves (Eds.). Handbook of special education research, 2. Routledge.
The research reported here is funded by awards to the National Center on Improving Literacy from the Office of Elementary and Secondary Education, in partnership with the Office of Special Education Programs (Award #: S283D160003). The opinions expressed are those of the authors and do not represent views of OESE, OSEP, or the U.S. Department of Education. Copyright © 2022 National Center on Improving Literacy. https://improvingliterarcy.org