Ten years ago, the Packard Foundation launched Starting Smart and Strong, an initiative designed to work alongside three California communities – Oakland, Fresno, and the Franklin-McKinley School District in East San Jose – to ensure that every young child grows up healthy and ready for kindergarten.
Recognizing the complexity of the initiative’s goal, the Foundation opted to partner with Engage R+D – a consulting firm dedicated to helping nonprofits, foundations, and public agencies measure their impact, bring together stakeholders, and foster learning and innovation – to conduct the evaluation of Starting Smart and Strong.
Now, as the initiative ends, having made significant impacts in these three communities and yielded valuable lessons for place-based investments in early education, Engage R+D has released a new case study that explores the nuances of evaluating this important effort.
We joined Katie Harkin, Associate Program Officer at the Packard Foundation, along with Clare Nolan, Co-Executive Director, and Pilar Mendoza, Managing Director, from Engage R+D to hear about what they learned, how their approach evolved, and what other funders and evaluators might learn from Starting Smart and Strong.
Katie Harkin
Associate Program Officer
The David and Lucile Packard Foundation
Clare Nolan
Co-Executive Director
Engage R+D
Pilar Mendoza
Managing Director
Engage R+D
What was the Foundation’s goal for evaluation at the beginning of Starting Smart and Strong?
Katie: We knew from the beginning that Starting Smart and Strong was an ambitious and complex initiative to improve early learning systems in three distinct communities. We planned to partner with these three communities for ten years, and we knew that we’d learn valuable lessons in that time, so we wanted to bring in an evaluator who could help us track and process what we experienced and learned to help inform future social change work.
We also wanted to know whether children in Starting Smart and Strong communities would be better off because of the initiative – to track whether what our grantee partners were doing was working.
How did your view of measuring outcomes evolve?
Pilar: In the initial phase of the evaluation, we concentrated on assessing kindergarten readiness, employing traditional methods to gauge the efficacy of programs across communities. This approach aligned with the initiative’s clear priority on improving child outcomes.
However, as we and community leaders delved deeper into their work, it became apparent that effectively advancing progress in kindergarten readiness required tackling deeper systemic barriers and inequities that prevented early learning from getting the resources and institutional support needed to ensure success. For example, when district leadership structures include an early learning representative, who can voice concerns and advocate directly within the chain of command, early learning systems are more likely to get the time, attention, and resources they need – but many districts are not set up this way.
Together with community leaders, we recognized the need to expand our view beyond immediate program results to the broader systems affecting early learning.
Consequently, we broadened our evaluation scope to focus on the systemic conditions that affect these barriers, using our insights to help communities drive the systems change needed to scale effective practices.
For example, in Fresno, as an initial pilot program to support dual-language learners began to show promise, the challenge shifted to scaling these successes across the system. This required developing new partnerships and building consensus across diverse agencies and early learning settings on how to strengthen professional development and elevate the importance of this work more broadly. These efforts have been crucial to achieving sustainable changes and increases in local and state resources available to support this work.
Starting Smart and Strong worked in three unique communities. How did you approach measuring success in communities with distinct needs and opportunities?
Katie: From the start, the Foundation and Engage R+D agreed on the importance of continuous feedback and input from the three communities, so communication and collaboration were built into our approach.
Because of the critical role of systems, each community was starting from a different place and had different priorities. For instance, Fresno Unified School District is the third largest district in California, so its challenges and opportunities can vary a great deal compared with those of a smaller district like East San Jose’s Franklin-McKinley. We embraced this concept and came to rely on our community partners as strategists and leaders.
How did equity become a more central focus over the course of the evaluation?
Clare: Two key developments prompted us to make equity a more central focus of the evaluation. First, there was growing dialogue about the importance of equity in social change and the need for foundations to integrate this focus into their funding efforts. While our team engaged with the Foundation to consider how equity could be incorporated into our work, we didn’t make immediate changes, given that equity was not an explicit focus of the initiative initially. However, everything shifted in 2017, three years into Starting Smart and Strong, when the Equitable Evaluation Initiative released its framework. This provided us with new language and resources to more strongly embed equity into evaluation processes and products.
At the same time, we learned from the communities we were partnering with about how they were actively addressing equity in their work. For example, our partners in Oakland heard clearly from families in their district that they needed to incorporate trauma-informed care and practices into Starting Smart and Strong – even taking the additional step of making equity a formal pillar of Starting Smart and Strong in their district. Inspired by Oakland’s leadership, our team, together with the Foundation, committed to threading equity even more deeply in Starting Smart and Strong, exploring ways to shift power and facilitate community ownership.
You mentioned that at the beginning of this process that you hoped what the Foundation learned through Starting Smart and Strong would inform others pursuing social change. What lessons came out of this work?
Katie: Systems change initiatives require adaptive, responsive evaluation approaches that match the complexity and long-term nature of the work. These evaluations should prioritize community leadership, equity, and continuous learning – and recognize the role community leaders play in developing strategy and guiding practices.
We also learned – and this shouldn’t be surprising – that transparent, collaborative relationships between funders, evaluators, and communities are essential for overcoming traditional power imbalances and fostering the trust that’s so important for social change work.
The case study that Engage R+D just published goes in-depth on the lessons we learned through the evaluation of Starting Smart and Strong. That’s a great place to go if you want to learn more.
Perhaps most importantly, we were validated in our belief that audacious, complex systems change is possible. Our journey across ten years of Starting Smart and Strong wasn’t always easy – we had to learn new ways of partnering with communities, unlearn some aspects of traditional funder/grantee relationships, and embrace flexibility in a whole new way. But it mattered. The communities we worked alongside are better positioned to support early learning, and the lessons they learned are taking root in communities across the country. I hope other funders will see this work as a proof of concept for bold systems change work. It’s worth it.
