Following the ACMi debate, Ms. Martyn attempted to walk back her claim of Asian students being disciplined at five times the rates of White students. Ms. Martyn wrote, “Other than one time during the debate when I inadvertently overstated the rate at which Asian high school students are disciplined compared to white students (without notes mistakenly stating it was five times the rate rather than 2.6 times, which is the correct figure), I am unaware of any other error in my statements that evening.”
This is a case of transporting the goalposts across town in an attempt to place them in the most advantageous position. Instead of acknowledging there was no difference in district discipline data between White and Asian students, she pivoted away from district numbers to the much smaller numbers at the high school.
In graduate school, we spent considerable time debating whether a sample size of 20 or 30 was sufficient for any statistical analysis. We were also cautioned about the large statistical error inherent in an analysis of cohorts of different sizes. Unfortunately, DESE displays data for cohorts of 6 students (and sometimes even fewer), and while the state makes no direct comparisons, the publication of this data invites inappropriate conclusions.
Over the course of 180 school days, 4 (out of 157) Asian students were disciplined at Arlington High. The small numbers magnify ratios when compared to other small numbers, which are easily manipulated and misinterpreted. We can’t claim extreme disparities when just one student having a terrible, horrible, no good, very bad day at school can shift ratios and alleged conclusions dramatically higher or lower. We can’t claim extreme disparities when we repeatedly divide small numbers into smaller and smaller fractions that descend deep into statistical insignificance.
I could move the goalposts over to the Gibbs School, where 11 of 12 students disciplined were White, and 0 Asian students were disciplined. However, I must not do that if I want to make valid or ethical inferences about the school or the district’s suspensions, just as Ms. Martyn can’t justify claims of extreme disparities by pointing to just four Asian high school students. Just as we can’t claim extreme disparities, we can’t congratulate ourselves blindly when all demographic groups have very high growth scores in the Grade 3-8 Mathematics MCAS. The bottom line is that we must not cherry pick data to provide a foundation for political claims of extreme discrepancies and thousands of underserved children when that is not the case.
I do not downplay the needs of any child in the system. One student who drops out is one too many. One student suspended is one too many. One student who does not feel welcome is one too many. One unhappy child or family is one too many. One incident of bias or racism seeping into any interaction is one too many. While we must work to get these numbers to zero, we must also recognize that our small number of adverse outcomes don’t point to extreme disparities or thousands of underserved students.
Paul Schlichtman is a member of the state Board of Elementary and Secondary Education’s School & District Accountability & Assistance Advisory Council. The Council reviews and recommends changes to the state's accountability system. He spent the past 20 years analyzing school and district performance and accountability data to help principals and teachers to understand the levers of change that improve their schools.
He understands that measures like the MCAS are just indicators that inform our work, not desired outcomes that drive our decisions. Too much emphasis on MCAS and accountability scores takes time and resources away from art, music, world languages, social studies, social-emotional learning, and other essentials not tested by the state.
Return to schlichtman.org