EmpowerK12 Releases DC Equity Comparison Tool

We hope everyone had the opportunity to enjoy the Memorial Day weekend and is geared up for the SY2018-19 school year home stretch! Nearly two weeks ago, OSSE posted additional student group STAR data as part of their federal reporting requirements and to align with their five-year strategic plan. The new data includes report card metric outcomes for more granular groups like "Not At-Risk Black/African-American" and "At-Risk Students with Disabilities" that help us further define the true nature and leading indicators of our within-DC outcome gaps between historically disadvantaged student groups and their more advantaged peers.

We created a DC Equity Comparison Tool to visualize and synthesize the vast amount of released data. We thank OSSE for their leadership in making this level of disaggregated data available; very few (honestly, if any) states make public this level of detail! Continue reading below for an overview of the tool and some of our initial high-level findings from the data.

Equity Tool Overview

We packed a lot of possible analyses into our tool to convey state-level differences among student groups and let the public dig-in at the metric and particular student group levels. We compared the performance of each student group to the percent of points they would earn on the STAR framework using the "All Students" floors and targets. Recall that on STAR each measured student group has its own floors and targets. However, to more acutely hone-in on outcomes gaps, we decided to re-score each group based on the "All Students" scoring range.

To start using the tool, begin by making selections in the blue filter ribbon. You can select a particular school level, group of STAR metrics, and choose a particular metric or several for a specific focus. Directly below the blue ribbon are the "state-level" results for each student group based on blue bar selections as well as score gaps between particular student groups most often discussed using an educational equity lens.

The rest of the dashboard is setup so users can compare two particular student groups. We defaulted the selections to "Not At-Risk" and "At-Risk" students. Any filter selections made on the blue bar carry through to all other data visualizations as well.

From here down, you can see how both groups performed on the STAR Report Card metrics, how the performance changes across wards and school levels, and how specific schools' student groups did in SY2017-18. We included the average n-size across all selected metrics to give you a sense of how many students counted in the score calculation.

What should this Equity Tool be used for?

We foresee many different possible uses for this tool depending on audience. Here are some of our ideas by group:

• City Leadership: We hope the data analyses offer our education sector leaders baseline information as they outline citywide goals for particular student groups, specifically at-risk and students with disabilities which receive additional funding in our per-student formula. Analyzing this level of data on an annual basis will help us track progress on the leading indicators for the outcome we all care about the most, academic achievement. Closing school environment and growth gaps in the short-term will go a long way towards ensuring further closure in achievement gaps.

• Accountability Agencies: This category includes the Public Charter School Board, OSSE School Report Card, and DCPS Central Office. And us, EmpowerK12, to a certain extent, too. We are using this data to help identify schools with historically under-performing student groups that are actually performing better than their more affluent peers as part of our 2019 Bold Improvement school awards. More on that later this summer! For the other agencies who may be exploring various equity-related provisions in their accountability systems, we hope this catalyzes conversation about appropriate business rules to ensure equity throughout our school accountability mechanisms.

• Parents: For our parents, we hope this tool can be useful in conversations with school leaders at PTA meetings, and for the new-to-DC parents that they find it useful in their school choice research.

• School Leaders and Boards: Finally, for our on-the-ground school leaders, we hope this tool is useful in helping you identify schools with great outcomes for student groups you may struggle to effectively reach. We hope this dashboard will bring more cross-collaboration that moves the needle forward for all student groups across the city.

What did we find in the data as we built the tool?

One of the first analyses our team examined as we built the tool was the difference in outcomes gaps between the academic achievement and growth metrics on the STAR frameworks. The next two charts show gaps between student group performance when we filter for just Achievement and just Growth.

A couple of important things jump out at us from these two charts:

• There are achievement gaps between races, even after you control for at-risk status. Some of the socioeconomic race gaps shrink or disappear altogether when you look at growth. At-Risk Black and At-Risk Latinos have better growth data than their At-Risk White peers, but not achievement. However, among "Not At-Risk" students there are growth gaps between races. (Note: there are only 28 At-Risk White students with growth data across the entire city, compared to more than 9,500 At-Risk Black/African-American students)

• While many of our students come to school from adverse environments, there are many schools attaining levels of academic growth that far exceed students from less adverse environments attending schools in more affluent neighborhoods.

We also noticed that many of the outcome gaps grow from elementary-to-middle-to-high school. For example, the chart below shows the outcome gaps between At-Risk and Not At-Risk students by school level. The elementary school gap is 31 points which grows to 39 points in middle school and 45 points in high school. 

If we assume that the average student arrives at PK3 with the same foundation ready to learn (note: research broadly concludes this is not true, but hang with us for this particular exercise), then the STAR socioeconomic outcome gap grows at an average rate of 3.9 points per year in elementary school, 2.7 points per year in middle school, and 1.5 points per year in high school. 

How does this "within-DC" gap data compare to gaps between our students and their national peers?

Some of the outcome gaps may seem stark, and they are. However, our local context is dramatically different than how our students are competing on the national level. The District has one of the highest within-jurisdiction income inequalities of an urban city across the country, according to the US Census Bureau GINI coefficient calculation. From a local perspective, we think it is appropriate to expect improved outcomes for all of our student groups and faster improvement for at-risk students. 

In the national context, we analyze the NAEP data and find mixed-to-positive results. Between 2007 and 2017, 4th grade DC black students improved proficiency rates by 14 points to 22%, while white 4th graders across the US gained only 4 points to 47%. In 8th grade, DC black students gained 4 points to 12% proficient, while white students nationally gained 5 points to 45%. In DC, our white 8th graders most recently posted a 78% proficiency rate, nearly 31 points better than their national peers. Ten years ago, our black students' average NAEP scores ranked #50 out of 51 states. By 2017, DC moved up to #30. The city has made statistically significant progress, but we still have a long way to go to ensuring every student is ready for college and career. 

By monitoring progress at the level of data made available by OSSE, the city is better positioned to identify best practices that will bend the learning curve to new heights for our most at risk students.