School improvement variables are many and diverse and even more so when assessing and addressing achievement gaps across and within subgroups. Data-driven school improvement remains a priority and digging into relevant data can be daunting. While data analysis is typically focused at the school and classroom level, it can be useful to take a look at aggregate data at the ISD level. This level of data review can assist in tackling underlying or less obvious issues that impact achievement gaps for students with IEPs. In addition, identification of such issues can inform procedural and managerial improvements.
ISpecial education policies, procedures and practices are typically managed (to varying levels) at the ISD level. Reviewing ISD level data can generate questions and probes that may impact school- and district-level improvement issues and strategies. One recent review of ISD-level data generated questions regarding rates of categorical identification that may be relevant to student achievement and school improvement issues.
Categorical Eligibility and Outliers
This review of recent ISD special education data portraits identified three categories of eligibility containing statistical outliers. An outlier is a value (number) that is statistically distant or different from other values across a population. In this case, six outliers were found in three categories of eligibility: Cognitive Impairment (CI); Autism Spectrum Disorder (ASD); and Specific Learning Disability (SLD). These varying outliers were found in a total of five ISD data snapshots (one ISD had 2 outliers).
These outliers beg questioning on many different levels and across many issues and topics. When examining data, outliers should receive full consideration because they often contain valuable information about the topic of interest. It is important to understand why there is an outlier in the data. It could be due to miscounting, errors in reporting, or errors in recording. If mistakes have been ruled out, then outliers can tell us about differences in policies, practices and context that may be influencing the extremely high or low incidence.
The graphs that follow provide an easy visual display of the distribution of the rates of categorical identification at the ISD level as well as a view of the outlier rates. The following graphs are presented consecutively; questions and suggested analyses follow.
Graph 1.1: Percent of Students Identified with a Cognitive Impairment by ISD
Graph 1.1 displays the percent of students identified with a Cognitive Impairment by ISD in the 2013-14 school year. The shaded dark green area contains data representing about half of the ISDs. For these ISDs, the percent of students identified with a cognitive impairment range from 7.7% to 10.6%. Expanding the range from 3.4% to 14.9% captures data representing most of the ISDs (shaded light green and dark green areas combined). ISDs that identify students with a cognitive impairment much higher or much lower than the range are considered outliers. There are three outliers in this graph and all three outliers are ISDs that identify students with a cognitive impairment above 14.9% (approximately 15.2%, 15.5%, and 18.0%).
Graph 1.2: Percent of Students Identified on the Autism Spectrum by ISD
Graph 1.2 displays the percent of students identified on the Autism Spectrum by ISD in the 2013-14 school year. The shaded dark green area contains data representing about half of the ISDs. For these ISDs, the percent of students identified on the Autism Spectrum range from 5.6% to 9.0%. Expanding the range from 0.4% to 14.3% captures data representing most of the ISDs (shaded light green and dark green areas combined). ISDs that identify students on the autism spectrum much higher or much lower than the range are considered outliers. There is one outlier in this graph, which is an ISD that identified students on the autism spectrum above 14.3% (approximately 14.9%).
Graph 1.3: Percent of Students Identified with a Specific Learning Disability by ISD
Graph 1.3 displays the percent of students identified with a Specific Learning Disability by ISD in the 2013-14 school year. The shaded dark green area contains data representing about half of the ISDs. For these ISDs, the percent of students identified with a specific learning disability range from 29.3% to 36.4%. Expanding the range from 18.7% to 47.0% captures data representing most of the ISDs (shaded light green and dark green areas combined). ISDs that identify students with a specific learning disability much higher or much lower than the range are considered outliers. There are two outliers in this graph and both outliers are ISDs that identify students with a specific learning disability above 47.0% (approximately 48.5% and 56.1%).
Questions to Consider
In each of the three categories of outliers, the first set of questions is routine and concerns accuracy:
- Are the data accurate?
- Were the data reported accurately?
- Were the data recorded accurately in the final public reporting?
Accuracy is important not only as it pertains to a particular ISD data portrait, but as it pertains to state level reporting as well. Outlying ISD data skew the total state portrait as compared to other states and can lead to inaccurate federal observations and/or determinations of state performance. Further, inaccuracy at the ISD and district level may lead to unnecessary review and oversight/monitoring in a variety of areas, including determinations (such as disproportionality) or assessment participation.
Once accuracy is assured, the following questions can generate a level of analyses that may not have been previously considered.
Policies, Procedures and Practices:
Within the domains of policies, procedures and practices, the questions are more complex as well as subtle. Eligibility determination for special education programs and services is a function of:
- Adherence to statutory and regulatory procedures;
- Standards of practice across professional practice for classes of educators, evaluators, therapists and other providers of services to children and youth with disabilities;
- Administrative oversight;
- School & agency culture;
- Individual approaches of the various professionals and practices.
Given the numerous influencers on practice, eligibility determination can vary across schools/agencies. This results in inconsistencies as to categorical determination of eligibility for special education programs and services from district to district, ISD to ISD, and state to state. While some variation is anticipated, extreme variance requires deeper analyses. Of particular concern is the impact of wide variance on students with IEPs.
This concern is based upon differences in educational practices tied to categorical eligibility. For example, students identified as cognitively impaired may not have access to a rigorous or aligned curriculum, or they may be placed in self-contained settings more than is necessary or optimal, impacting both curriculum and instruction. Or, students identified on the Autism Spectrum may face adult mindsets or biases regarding actual potential and abilities. In addition, students identified as having a specific learning disability may be swept into resource or remedial programs that reduce opportunity for core instruction with their peers. These are just a few examples of the impact of categorical eligibility outcomes. There are numerous others.
Even deeper issues impacting eligibility determination may include unstated biases regarding class, ethnicity, race, immigrant status, family configuration, or subtle community stereotypes. Therefore, it is imperative that notable variances in data regarding categorical eligibility be examined carefully. Questions to be carefully considered include the following:
Administrative Oversight Variables:
- Is our practice for identification of categorical eligibility out of date?
- Do our evaluation tools reflect current standards and practice?
- Do we support our staff to engage in continuous learning and professional certification updates?
- Do we provide solid and up-to-date management and evaluation of our staff?
- Do we access appropriate consultation and external coaching as needed?
Administrative Convenience Variables:
- Are available (or unavailable) resources driving our identification practices?
- Do rigid approaches to finances impede thoughtful and coordinated budgeting for necessary programs and services?
- Do student outcomes and program needs drive budgeting, or does budgeting drive what is available?
- Are program needs and budget planning coordinated and aligned?
- Is administration of special education programs and service coherently aligned with general education?
- Are there inherent biases impacting our practice?
- Are we attending to our own biases (class, ethnicity, race, immigrant status, family configuration, community stereotypes, etc.)?
- Have we provided professional learning on cultural competencies?
- Are we measuring professional practice as to cultural competence?
- Are external pressures for certain programming or services driving our practice in determining eligibility?
- Can we defend our practices against professional standards?
- Do we articulate standards of practice when confronted with external pressures that do not align with such standards?
How to Check if Your ISD is an Outlier
First, check your data as it appears in MI School Data. Go to Special Education, Data Portraits, Disability, All ISDs, then by: Autism Spectrum Disorder, Cognitive Impairment, and Specific Learning Disability.
Check your percentage of students in these categories and compare to the graphs presented in this article (2013-14 data). If you are one of the outliers (or even close), seriously consider deep administrative and practitioner team discussions using the questions presented here. Also, compare your 2013-14 data against the previous 3-5 years to determine trends or anomalies. Are you confident that policies, procedures and practices in your ISD, or within individual districts, are serving students with IEPs in manner that creates high expectations, rigorous teaching and learning constructs, and optimal achievement and outcomes?
Ultimately, categorical eligibility drives subtle but real impacts on curriculum, instruction, opportunity to learn, expectations for achievement, and finally, real outcomes.