Combating Learning Loss and Promoting Early Childhood Care and Education in Nuevo León, Mexico

We developed several policy alternatives that aligned Morales’ strategic objectives with the provisions of the Futures of Education report. Such work, coinciding with her first 100 days in office, embedded the Harvard Team both in various other projects at the SEP as well as within the broader education landscape, connecting with other secretariat personnel, non-profit directors, advocates and educators across the state. After having findings of the policy analysis exercise both presented at the Futures of Education Global Education Conference and published in Advancing A New Social Contract For Education: Collaborations to Reimagine our Futures Together, the Harvard and Nuevo León SEP teams were eager to continue collaborating.

The education system in the state of Nuevo León, Mexico (enrolling more than 1.6 million students across 7,796 schools) witnessed growing barriers to education and inequitable learning gaps during the Covid-19 pandemic. With the reopening of schools, the state sought to combat poor access and quality of Early Childhood Care & Education (ECCE) services. It also aimed to develop better understanding of learning loss and develop ways to overcome it. Although pre-school is now federally mandated, only 70% of Nuevo León families enroll their children, which many think is because of cultural norms around home-based childcare, limited information and financial barriers. Additionally, school closures are estimated to be associated with statistically significant learning loss in Mexico in reading and mathematics for primary school students.

Morales’ administration implemented an assessment to fill local information gaps: the Medición Independiente de Aprendizaje (MIA). Data collected from the MIA assessment points to significant gaps in basic skills for Nuevo León, with two-thirds of evaluated second to fourth graders unable to read and understand a simple text.

One of our team members was able to travel to Monterrey during the winter break to meet with Morales and the SEP staff to establish a foundation and plan for continued collaboration after the Harvard Team’s coursework ended, forging critical connections with different SEP departments and stakeholders in the state. This included participating in meetings surrounding the development of the revised Strategic Plan for the Education Sector, meeting with assessment and evaluation consulting team, engaging in dialogue around communications strategies surrounding recently collected learning loss data and continuing to support the work of the ECCE sub-sector.

Combating Learning Loss through Data Analysis and Program Evaluation

We also worked on two additional projects this semester to supplement our policy recommendations regarding assessment and data-responsive pedagogies. The first project was an analysis of MIA data collected from second to fourth grade students in November and December of 2021. Although officials at the Secretariat had already produced some analyses of this data, our report used the same raw data to call attention to additional key issues and inequities. Our analysis highlighted large achievement gaps between students enrolled in public schools compared to those enrolled in private schools. Leveraging principles of effective data visualization learned from other HGSE coursework, the Harvard Team was able to display important equity gaps that showed school-level discrepancies across subjects and student sub-groups. The accompanying explanation pointed toward potential root causes and policy implications of assessment results.

The Harvard Team was able to highlight limitations of MIA data, including the fact that the 17% of children not represented in the data are likely among the most vulnerable and socioeconomically disadvantaged, to promote more equitable and robust evaluation practices in the future. Finally, our report used learning poverty, defined by the World Bank as the percentage of 10-year-olds unable to read and comprehend a text, to provide a comparative benchmark for Nuevo León’s current learning levels and show the urgency of combating learning loss in the immediate future.

Additionally, in collaboration with one group member’s colleagues in HGSE’s Program Evaluation and Education Policy course, we designed a potential evaluation for our recommended pilot summer learning intervention that would implement Teaching at the Right Level (TaRL) pedagogy. The proposed evaluation would use a cluster randomized trial design to identify the effects of summer learning on MIA assessment outcomes. The proposal included implementation and feasibility considerations in addition to technical estimates and calculations regarding sample size, statistical power and intraclass correlation. While it was not logistically feasible for SEP officials to implement a pilot program in 2021, this potential evaluation plan will prove useful for planning and evaluating future summer learning interventions.

Bolstering Access and Quality to ECCE through Problem-Driven Iterative Adaptation

The state of ECCE in Nuevo León is not unlike that of most Latin American countries: characterized by mostly unregistered service providers of varying quality, far from universal coverage and lacking efficient coordination. Attributing the state’s increasing educational lag to poor socio-emotional and cognitive development in early childhood, Nuevo León has made ECCE a priority through the campaign “Nuevo León is the best place to be born, raised, and educated.” The campaign’s immediate focus is boosting ECCE coverage, specifically for three-year-olds since the age group saw a 21.5% drop in enrollment during the pandemic. To this aim, Nuevo León aspires to expand and rehabilitate the infrastructure of its preschools, bolster teach quality through LEGO’s play-based learning trainings and to standardize the quality of daycare services by credentialing daycare teachers and registering an estimated 700 unlicensed centers (operating without the Secretariat’s Recognition of Official Validity of Studies (RVOE), either informally or affiliated with one of thirteen federal service providers).

During our team member’s visit to Monterrey in January, ECCE policy recommendations emerging from the original policy analysis exercise received significant attention from the SEP staff. Wanting to pressure-test the promise of such recommendations before moving forward, the Harvard Team chose to pursue the problem of “poor quality and low access to ECCE” through the Problem-Driven Iterative Adaptation (PDIA) approach, developed by Harvard University’s Center for International Development. With support from Nuevo León’s Early Childhood Director, Homero Trujillo, the Harvard Team pursued a 10-week process of defining the problem facing ECCE in Nuevo León, deconstructing the problem into its many root causes, and analyzing the landscape for entry points for change.

The Harvard Team and Trujillo identified three major root causes of the poor performance of the state’s ECCE sector that also presented promising entry points for change: (1) a lack of standardized evaluation and assessments mechanisms, (2) poor in-service training delivery systems for day-care providers and (3) insufficient and un-specific teaching and learning materials for registered ECCE centers. In response to the identification of such entry points, the Harvard Team helped Trujillo to consider the advantages and costs of different ECCE evaluation systems, such as MELQO and MELE tools. Other important considerations involved how hybrid modalities for in-service teacher trainings systems such as TPD@Scale could work in Nuevo León’s contexts, and the identification of best practices surrounding daycare regularization and service provider training.

Future Plans

As Natalie and Ryan reach the close of their time in HGSE’s Master’s program and their fellowship with DRCLAS, we are reflecting on how they wish to continue their relationship with Nuevo León and how such experiences are informing their professional journeys moving forward. Ryan hopes to visit Monterrey again in the near future to continue learning how education policymakers navigate dense and complex political and social landscapes and is bringing his new-found curiosities and passion for ECCE into his consultancy work with the World Bank’s Education Global Practice. Natalie hopes to carry lessons learned through this experience in education policy and practice, working with diverse stakeholders, analyzing and writing about assessment data and using evidence to inform policy recommendations into her future work in education policy.