At-Risk Students and PARCC Achievement: A Robust Discussion
Photo credit: KIPP DC Valor Academy and Ashley Lemley
In this blog post, we examine the following key questions related to the academic achievement and improvement of DC's at-risk students:
How has the correlation between school level proficiency and percent of at-risk students tested changed over time?
Is the pace of at-risk proficiency gains improving and when will DC at-risk students close the achievement gap with their national higher income peers?
What school level operational and instructional policy decisions are helping to improve at-risk outcomes?
The link between socioeconomic status and academic achievement outcomes has existed for decades and, likely, centuries. To offset some of the inequity inherent in the District's demographics, the City Council and Mayor approved an additional at-risk weight in the student funding formula in 2015. This landmark legislation was designed to help provide additional resources to students in most need of them, helping to reduce the achievement gap between low-income students in the District and their higher-income peers nationwide. This post examines how PARCC achievement has changed for at-risk students since then.
For each of the questions, we provide a quick summary bullet point or two about what the data suggests is the answer. Then, we dig into the heavier statistics that help answer the question.
How has the correlation between school level proficiency and percent of at-risk students tested changed over time?
In 2018-19, at the school level, the correlation between serving higher proportions of at-risk students and lower school level proficiency rates decreased, a positive development for the city.
The effect size of serving larger populations of at-risk students on overall achievement remained at levels lower than 2015-16, specifically dropping to the lowest level for PARCC 3+ Proficiency of any year. This is another positive sign for the District.
More schools demonstrated significant positive improvement in at-risk student gains this year than any other prior PARCC year.
The scatter plot below demonstrates the link between at-risk served and school-wide proficiency rates. Yet, we notice many schools across all levels of income status well above the trend line. At one school serving majority at-risk, KIPP Promise elementary school, at-risk students are actually performing higher than their wealthier peers nationwide in both math and reading! Annually, we honor schools far exceeding expectations with our Bold Performance school award as they demonstrate the higher levels of success that are possible for all students across the city. We will do so again in the near future.
There a couple of ways to mathematically analyze whether District schools are creating a more equitable environment for low-income students as a whole: correlation coefficients and standardized coefficients. The correlation coefficient (a measure that ranges from -1, a perfect negative relationship, to +1, a perfect positive relationship) shows the relative strength of the relationship between at-risk and proficiency. The table below shows how the correlation between percent at-risk tested and overall proficiency at the school level has changed since 2015-16.
Last year, the correlation coefficient decreased for both proficiency metrics from the prior year and is lower than the previous 3 years combined.
One additional way to contextualize the power of the correlation is via the standardized coefficient. Standardized coefficients refer to how many standard deviations a dependent variable will change per standard deviation increase in the predictor variable. In our case, this translates to "when a school increases the percent of students who are at-risk by 22% (the standard deviation of percent at-risk served at the school level), the expected percent of students who are proficient decreases by ___." The chart below fills in the blank by year with lower numbers representing a more positive outcome.
Both the correlation and standardized coefficients between at-risk and proficiency decreased this year, a positive sign for a more equitable school system. Yet, the within-DC math and ELA combined Level 4+ proficiency gap between At-Risk and Not At-Risk students increased by 0.5 percentage points in 2018-19. So what gives? Why did the gap increase but the other indicators of at-risk impact decrease?
The quick answer is that while the PARCC 4+ gap marginally increased at the state level, there were a record number of schools serving higher percentages of at-risk with double-digit proficiency gains this year. Also, while the PARCC 4+ gap may have increased, the PARCC 3+ gap decreased in both math and reading. The chart below shows a list of schools with the 20 largest single-year proficiency gains for at-risk students since 2015-16. We would expect about one-third of the list to have been from 2018-19 in a normal year, yet three-quarters (15) of the top 20 are from this past year across a variety of school types. When the at-risk student growth data is released with the updated DC school report cards this November, we expect several of these schools to land on our 2020 Bold Improvement schools list!
More schools are implementing the strategies we know that work for engaging at-risk students, so now we need all schools serving low-income students to implement them.
We caught up with the two schools serving the highest percentage at-risk to make the Top 20, Friendship Southeast and Eagle Congress Heights, and asked them what worked last year. Check what their leaders said in the response to the last question below.
Is the pace of at-risk proficiency gains improving and when will DC at-risk students close the achievement gap with their national higher income peers?
English language arts (ELA) gains year-over-year for at-risk students have trended up each year since 2015-16. Math was going up until this year with at-risk proficiency up a slight amount in 2019.
With the newest data, the District's at-risk proficiency improvement rate since 2015-16 means our at-risk students are on pace to close the current national achievement gap by 2038, far longer than we would like but at least trending in the right direction.
These two bar charts show the change in at-risk student proficiency over time.
If we want to close the national socioeconomic achievement gap for at-risk students in less than half the current projected time, we will need to follow the examples and advice from schools demonstrating the greatest consistent at-risk student gains. Find out what those are below.
What school level operational and instructional policy decisions are helping to improve at-risk outcomes?
In order to cut the achievement gap faster, schools serving the highest proportions of these students will need to improve quicker. We asked the two schools with the largest at-risk populations to make the Top 20 greatest single-year proficiency improvement for at-risk students list what did they do last year.
At Friendship Southeast elementary school, Principal David Lawery talked about having high expectations and strategically using all sorts of student work and other data to drive improvement. According to Principal Lawery: "We strive to become not only the top-notch early-learning center, but the premier PS-8th learning environment in Ward 8, providing high-quality instruction to develop lifelong learners. In our goal of college and career readiness, we owe much of our success to fidelity to Friendship PCS instructional programming, intentional small-group enrichment and intervention as well as purposeful professional development."
Eagle Academy with two campuses, one in Congress Heights and one residing in Navy Yard last year, posted double-digit proficiency gains at both schools. Their Executive Director Joe Smith attributed gains at both campuses to the following strategies they implemented in the last two years:
New math program, Eureka, with coaching for instructional staff
Intense year long professional development with focus on educating the whole child, social emotional as well as academic, utilizing the latest brain science research
During the summer of 2019, Eagle Academy provided a 6-week intense literacy program for rising 3rd graders with weekly parent workshops included
Teachers attended weekly-to-monthly Professional Learning Community meetings where they examined data and planned for student success
After school tutoring and Saturday Academy was provided for all students
Another place to look for answers is the lessons we learned from our 2019 Bold Improvement schools, most of which continued their pace towards closing the achievement gap by 2024! The Bold school teachers, school leaders, and other staff identified several practices that they feel are most responsible for the rising levels of achievement at their schools:
They use data and information in frequent and intentional ways. Sometimes student performance on morning tasks informs what extra help they get that same afternoon. At Langdon Elementary, for example, student groupings change weekly to match children to the teacher known to be best at teaching the skill they are having trouble mastering. Children move between classrooms and even grade levels.
They collaborate across classrooms as well as across schools. The “bridge” program that links Tubman and CHEC is an example of this practice. Another example: School leaders at three charter elementary schools, KIPP Heights, KIPP Promise and KIPP Valor, meet frequently, visit each other’s campuses, and share data to get better.
They ensure students are ready to learn by creating a warm social-emotional environment. Bold Improvement schools prioritize social-emotional skills and beliefs, integrating them with academic development so that students have the right tools for engagement and a language for learning. At Walker-Jones Education Campus, a DC Public School in Ward 6, teachers track student progress on co-academic skills like self-management on a weekly basis to ensure all students have the right mindset to demonstrate exceptional academic growth.
They hold the highest of expectations for all students. Every student, regardless of the skill deficits they have upon arrival, is exposed to grade-level content every day. It’s not “dumbed down” for them. For students with disabilities and those learning English, these schools maximize their time in general education classrooms, with specialists teaching alongside general educators. At Perry Street Prep, a charter school in Ward 5, students with disabilities are expected to demonstrate academic growth exceeding their non-disabled, higher-income peers, and they have been outpacing higher-income students nationwide.