Fact checking the ACMi School Committee debate:

 

Lynette Martyn, beginning at 34:30

“I’ve noticed that no school committee candidates have been specifically addressing the state’s data on the extreme disparity gaps for our high needs students, including 15% of our kids on IEPs, the 30% of our kids that identify as students of color, our economically disadvantaged students, and our English language learners. This amounts to thousands of children with disparities in MCAS scores, graduation rates. These students deserve better from our school system and I’d like to understand why no one is talking about the data specifically.”

 

Lynette Martyn, beginning at 40:20

“Our students of color make up 30 percent of the student population but 50 to 60 percent of our suspensions. Our Asian kids might be doing better on our MCAS scores but they are being disciplined at five times the rate of our white kids.”

 

“Arlington prides itself on a 96 percent graduation rate but they are significantly lower and as low as 79 percent for our economically disadvantaged kids. We don’t know how to talk about these uncomfortable truths. If we aren’t willing to lean into these difficult conversations, then we’re not going to be able to tackle the systemic issues…”

 

Lynette Martyn, beginning at 44:24

“If you’re a white kid who comes from an economically stable family, and is a native English speaker, and learns the same way as everyone else, Arlington Public Schools is great. We’re meeting the needs there, and we’re on a (inaudible at 45:22) kids that are being underserved are overlooked and we really need to focus on them. And these statistics are, the statistics come from the state. They are the state keeping us accountable, and so I think they are important data to look at. I think we should be more transparent about it with the community.”

 

Paul Schlichtman, beginning at 35:00

It’s a complicated picture and we cannot pigeon-hole it as certain kids are doing well and certain kids aren’t in terms of looking at big groups. To make things work, we need to do really great teaching one-on-one, classroom level, understanding the unique needs of each and every kid. Looking for things that are systemic, but also going down to the classroom level, building the relationship with each and every child to make sure we are meeting individual needs, through a tiered approach that will demonstrate that good teaching matters; and that’s what we’re looking to do.


 

A lesson in significance:

 

Amy Gallo quotes author Tom Redman in her article, A Refresher on Statistical Significance,” in the Harvard Business Review (February 16, 2016):

“Statistical significance helps quantify whether a result is likely due to chance or to some factor of interest,” says Redman. When a finding is significant, it simply means you can feel confident that’s it real, not that you just got lucky (or unlucky) in choosing the sample.

https://hbr.org/2016/02/a-refresher-on-statistical-significance


I note that, for the purpose of testing the null hypothesis that there is no difference between groups in the population, the test is to determine the probability of that group (sample) being significantly different than the population as a whole. 


A close up of a
                  coin Description automatically generatedA picture
                  containing object, photo, sitting, clock Description
                  automatically generated

 

Making sense of coin flips

We have data from a series of coin flips using two different coins, a quarter and a dollar coin. The quarter comes up heads 52% of the time, the dollar coin comes up heads 60% of the time.

Which of the following conclusions can be derived from the data?

  • The dollar coin is more likely to come up heads because it is more valuable.
  • The dollar coin is more likely to come up heads because the image is of a woman
  • The dollar coin is more likely to come up heads because the image faces to the right, where the image on the quarter faces left.
  • The dollar coin is more likely to come up heads because of the dollar coin’s golden color

 

Before coming to any conclusions, let’s look at a detailed description of the data.



 

Coin

Flips

Heads

Tails

Percent Heads

Quarter

100

52

48

52%

Dollar

5

3

2

60%

 

This brings us back to Lynette Martyn’s claim there are “thousands of children with disparities in MCAS scores, graduation rates. These students deserve better from our school system.”

Let’s look at the data, as reported on the state website www.doe.mass.edu.


Are there extreme disparities in graduation rates?

Here is a screenshot of the data reported on the state website.

http://profiles.doe.mass.edu/grad/grad_report.aspx?orgcode=00100000&orgtypecode=5&

 

The state calculates two sets of graduation rate data. There is the Cohort Graduation Rate, which describes the outcome for all the students in the graduating class. There is also the Adjusted Cohort Graduation Rate, which describes the outcome for the students who were in the cohort since the beginning of ninth grade. In this case, the Adjusted Cohort Graduation Rate includes only students who were reported as enrolled in the Arlington Public Schools on October 1, 2015.

 

`Graduation Rates from DESE

 

The analysis requires us to look at the raw numbers instead of the percentages displayed on the DESE data file. To aid in the analysis, the students in the Adjusted Cohort Graduation Rate table were subtracted from the students in the Cohort Graduation Rate to provide data for students who entered the Arlington Public Schools after October 1, 2015.

 

A screenshot of a cell phone
                  Description automatically generated

DESE does not report graduation rates for cohorts less than 6

 

Despite Lynette Martyn’s claim to the contrary, there is no evidence of extreme disparities in graduation rates for White, Black, or Asian students in the Arlington Public Schools.

 

Looking at the Four Year Cohort Graduation Rate, the differences between White (96.1%), Black (92.3%), and Asian (100.0%) graduation rates is not significant. There were 13 Black students in the cohort, so the percentage graduating (12/13 or 92.3%) cannot be seen as evidence of extreme disparities between racial groups.

 

Looking further, the Four Year Adjusted Cohort Graduation Rate yields another set of numbers where the differences between White (97.4%), Black (100.0%), and Asian (100.0%) fails a test of significance. Why is there a difference in the graduation rate for Black students? One Black student, who entered Arlington schools after the beginning of ninth grade, was still enrolled in school on October 1, 2019.

 

If we are to look for any findings from the data, the statewide trends may provide some insight. Students with disabilities, English Learners, and students who transfer in after ninth grade are more likely to take additional time to graduate. The reason is self-evident. Students may be transferring into our schools lacking sufficient credits to be on track for a four-year graduation, English Learners need to develop English proficiency in addition to meeting standard academic requirements, and Students with Disabilities often take more time to meet graduation requirements.

 

Moving away from the graduation rate, we should also look at the dropout rate. Arlington reported 5 dropouts out of 360 students (1.4%). Two of these students were in the adjusted cohort of 315 students (0.6%), and three came from the students who joined the Class of 2019 after the beginning of ninth grade (6.7%). Again, the very small number of dropouts (5) makes it impossible to assign significance to any category of students. This is why the strategy of focusing on individual students, rather than try to attribute outcomes to extreme disparities between groups.

 

Paul Schlichtman, beginning at 35:00

It’s a complicated picture and we cannot pigeon-hole it as certain kids are doing well and certain kids aren’t in terms of looking at big groups. To make things work, we need to do really great teaching one-on-one, classroom level, understanding the unique needs of each and every kid. Looking for things that are systemic, but also going down to the classroom level, building the relationship with each and every child to make sure we are meeting individual needs, through a tiered approach that will demonstrate that good teaching matters; and that’s what we’re looking to do.

 


 

Are Asian kids being disciplined at five times the rate of white kids?

 

Lynette Martyn, beginning at 40:20
“Our Asian kids might be doing better on our MCAS scores but they are being disciplined at five times the rate of our white kids.”

Fact:
According to the Department of Elementary and Secondary Education 2018-19 Student Discipline Data Report, 8 out of 803 Asian students were disciplined (0.996%) and 54 out of 4,282 White students were disciplined. (1.261%). This difference is not significant.

 

A screenshot of a social
                    media postDescription automatically generated

http://profiles.doe.mass.edu/ssdr/default.aspx?orgcode=00100000&orgtypecode=5&=00100000&

 

The very small number of students disciplined makes it difficult to assign significance to any one category, with the exception of gender. Given that 78 of 93 students disciplined are male, there is evidence of a disparity that merits a possible examination of systemic factors leading to a student being disciplined.

 

The High Needs category consists of any student that falls into any one of the three categories of English Learner, Economically Disadvantaged, or Students with Disabilities. (The same student will be counted in two or three of these categories.)

The district has a goal of zero suspensions, and Arlington has added social workers and psychologists to meet the individual needs of students with behavioral challenges. Again, the numbers in Arlington are small, so a focus on individual students is the best strategy for reducing incidents requiring disciplinary action.

 

Additionally, the numbers of students disciplined in Arlington are significantly smaller than the statewide numbers. The percentage of all students disciplined in Arlington (1.53%) is about a third of the statewide rate.

 

A screenshot of a cell
                  phone Description automatically generated

Racism is evil, and it has been a toxin infused in the American experience from the first day Europeans landed on this continent. Our schools are a better place today than in the past because of the diligence and hard work of our staff and our community. Along with the rest of the world around us, we have not reached a race-blind panacea, which is why we strive for a culture of continuous improvement in everything we do. This work requires precise, thoughtful analysis in an environment where it is safe to examine our reality and talk openly about the next steps forward. The norms required for this difficult work are violated when false or misleading data is forced into the center of our public discourse.

 

Promoting blanket outrage over shortchanging “thousands of students,” when the thousands don’t exist, blinds us to the important work of fighting racism. It blinds us to examining and evaluating the interpersonal relationships that are the foundation of our work. It is toxic. It is hurtful and defamatory to members of our school community, and counterproductive to our efforts to create a better world for all our students.