Disaggregating the Data: A Critical Survival Strategy for Dual Language Programs
By: Kris Nicholls, Ph.D., CEO, Nicholls Educational Consulting
All programs outside of traditional, English-only programs lay in peril of being shut down each year, with their value constantly needing to be proven to school board members, district and site administrators, teachers, families, and the local community. But what does “value” mean to each of these stakeholder groups? To some, it may be embedded in language politics that situate English as the dominant language and the only one that is important to learn if you live in America (Crawford, 1992; González, 2000; Lippi-Green, 1997; Macedo, Dendrinos, & Gounari, 2003). To others, it may be focused on the academic achievement of students in the program. Indeed, grade-level academic achievement in English as well as the partner language is one of the three pillars of dual language education as cited in the “Guiding Principles for Dual Language Education, 3rd Edition” (Howard, Lindholm-Leary, Rogers, Olague, Medina, Kennedy, Sugarman, & Christian, 2018). However, given the federal accountability requirements[1], the most intense focus is on student achievement measured in English.
In standardized achievement assessments, there is typically a 4-point scale used to report student academic proficiency levels.
The data that is most commonly reported is the percentage of all students who are meeting or exceeding standards. The data that is typically reported include all students who took the assessment. However, the reporting of data for all students masks the academic achievement of subgroups for whom there has long been a concern regarding their chronic underachievement, as well as for subgroups who are achieving higher than the reported percentage for all students.
For instance, if the percentage of students meeting or exceeding standards was reported as 70%, then the percentage of students in some subgroups may be significantly less. For other subgroups, the percentage may actually be higher. Disaggregating the data is critical to identifying how each subgroup of students is performing on a particular assessment. In this particular example, shown in the table below, subgroups are performing at the following levels.
As seen in the table above, a greater percentage of the English-only (EO) and Reclassified Fluent English Proficient (RFEP) students are meeting or exceeding standards than the percentage reported for all students. However, a lesser percentage of the English Learner (EL) subgroup is meeting or exceeding standards and thus performing lower than all students. However, these types of comparisons are not tenable, as the data from each of these subgroups is included in the calculation of the percentage reported for all students. Therefore, to obtain a more accurate picture of how each subgroup of students is performing, comparing subgroup-to-subgroup percentages will be more accurate than comparing the subgroup’s percentage to the reported percentage for all students.
Here is an example of how the above data can be used to gain a more accurate perspective on student performance in each subgroup.[1]
As this example demonstrates, comparing a particular subgroup to the aggregated “all students” data can often lead to underestimating the gap, whether positive or negative. Instead, disaggregating the data to allow subgroup-to-subgroup comparison yields more accurate data to reflect upon with regard to the opportunity gap for groups of students, such as English Learners.
Although disaggregating data to derive a more accurate comparison between subgroups is important, for dual language programs, being able to disaggregate the data even further will be a critical tool to support the sustainability of the program. Given that the research on dual language programs[1] shows high academic achievement for all students enrolled in the program, that their achievement outpaces that of their peers not enrolled in the dual language program is often not captured or reported.
Thus, it is recommended that the data system that collects student demographic data also include two fields: one to identify if the student is enrolled in the dual language program and one to identify the year that the student started in the program. Having these fields will allow queries to be run to identify the students enrolled in the dual language program and to then compare their performance with those not enrolled in the program. The data for students in your dual language program can be further disaggregated by subgroup. For instance, the percentage of students meeting or exceeding the standards that are English Learners and enrolled in the program can be compared to those who are not enrolled in the program.
Disaggregating the data in this manner can be helpful in dispelling concern about the English Learners enrolled in the dual language program. Often, when there is a gap such as the one shown in the example, above, there is a push for English Learners to receive more instruction in English, which might trigger a call for any English Learners to be moved from the dual language program to the English-only program. However, by disaggregating the data by English Learners in the dual language program, the percentage of them meeting or exceeding the standards on the assessments can be compared to English Learners who are not in the dual language program. It is highly likely that the percentage of English Learners in the dual language program meeting or exceeding the standards will be greater than those who are not in the program, especially in the upper elementary grades and beyond, as shown in the graph below.
To illustrate the potential benefits from disaggregating the data by students enrolled in the dual language program and by subgroup, consider the data in the table, below.
This data can be further disaggregated by grade level and by length of time in the dual language program. By grade level, the data can compare the achievement of students at each grade level who are in the dual language program and those who are not to further demonstrate the benefits of the DL program. As the research5 shows, most typically the greater positive gaps will be in the upper elementary and secondary grade levels. By disaggregating by length of time in the dual language program (especially important for programs that allow students to enroll after kindergarten and/or first grade), the data can be disaggregated to compare the achievement of students who have been in the dual language program based on the number of years enrolled (i.e., those enrolled for 6 year, for 5 years, for 4 years, etc.) or among students that have been enrolled in the program the same number of years.
Data has an important role in education, and it is critical that dual language programs be critical connoisseurs of their data, understanding the importance of disaggregating to more clearly identify the level of achievement of their students in English and the partner language to guide instruction and to reflect on the implementation and quality of the dual language program. Equally important, there should be transparency in dual language programs with regard to their students’ achievement, to demonstrate the program’s strength and success to all stakeholders and to secure its sustainability.
nichollseducationalconsulting@gmail.com
nichollseducationalconsulting.com
References:
[1] The Every Student Succeeds Act (ESSA) is available at https://www.ed.gov/essa?src=rn[2] Definitions for each of the English Learner subgroups (EL, RFEP, etc.) can be found at: https://caaspp-elpac.cde.ca.gov/caaspp/docs/Understanding_English_Learner_Achievement.pdf
[3] For the purposes of this example, only the data from the EO, EL, and RFEP subgroups will be considered. A broader comparison will be possible when including all the significant subgroups in your district.
[4] For the purposes of this illustration, equal number of students in each subgroup are assumed.
[5] Collier, V.P., & Thomas, W.P. (2004). The astounding effectiveness of dual language education for all. NABE Journal of Research and Practice, 2 (1), 1-20.
Lindholm-Leary, K. (2016). Bilingualism and academic achievement in children in dual language programs. In E. Nicoladis & S. Montanari (Eds.), Lifespan perspectives on bilingualism. Washington DC: APA Books.
Thomas, W.P., & Collier, V.P. (2002). A national study of school effectiveness for language minority students’ long-term academic achievement. Santa Cruz, CA: Center for Research on Education, Diversity and Excellence, University of California-Santa Cruz (351 pp.)
Thomas, W. P., & Collier, V. P. (2012). Dual language education for a transformed world. Dual Language Education of New Mexico/Fuente Press.
Crawford, J. (Ed.). (1992). Language loyalties: A source book on the official English controversy. University of Chicago.
Gonzalez, R. D., & Melis, I. (Eds.). (2014). Language ideologies: Critical perspectives on the official English movement, volume II: History, theory, and policy. Routledge.
Howard, E. R., Lindholm-Leary, K. J., Rogers, D., Olague, N., Medina, J., Kennedy, D., Sugarman, J., & Christian, D. (2018). Guiding Principles for Dual Language Education (3rd ed.). Washington, DC: Center for Applied Linguistics.
Macedo, D., Dendrinos, B., & Gounari, P. (2015). Hegemony of English. Routledge.
Lippi-Green, R. (2012). English with an accent: Language, ideology and discrimination in the United States. Routledge.
