Preparing districts for scaling best practices

Are you a district administrator thinking about scaling a set of practices across your schools? If you answered yes, odds are you are probably asking a similar question – what exactly do I need to do to spread the best practices at one school to other schools?

When you are thinking of this question, I would recommend thinking of two buckets of research about scaling.  First is Richard Elmore’s piece, “Getting to Scale with Good Educational Practice.”[1] Elmore suggests four practices for administrators to keep in mind when scaling best practices:

1. Develop normative structures for practice: Sets of criteria, frameworks, standards can guide practice across different contexts and specifically different schools.
2. Develop structures that support educators intrinsic motivation to challenge themselves: This includes structures like face-to-face relationships, groups of people struggling together on common problems, etc. that center around the results of students.
3. Create an intentional process for reproduction of successes: Allow schools to reproduce practices that are not exact replicas, but rather good teaching practices that resemble each other.
4. Create structures to promote learning new practices and incentive systems that support them: If you expect teachers to be learning, you must have organizational structures that promote that learning like time for new required tasks, time for observing others, and access to special knowledge.

The other bucket of research is from a RAND publication by Glennan and colleagues[2] that reviews a large body of research on scaling. The key take-aways for me from this report were:

1. No matter the target of reform or the design construct, the process is iterative, complex, and requires support by multiple actors. You can’t just to expect schools to take up new practices on their own, but must provide them with time, support, room for learning/mistakes, and helpful structures and processes to learn new behaviors and ways of being.
2. The actors must jointly address a set of known, interconnected tasks if scale-up is to succeed; especially, the actors must align policies and infrastructure to sustain practice: This concept suggests that the schools and district must create a policy and infrastructure environment that is conducive to scaling.  For example, if you want all teachers to be implementing a new strategy in reading, you better provide them with the professional development and release time from teachers to learn that new strategy.
3. How each group addresses new practices varies according to design, context, and resources: Once again, context matters, and the “scaling” of a certain practice across a set of schools is not going to look exactly the same across all schools. Also, some schools are going to need different amounts of resources like time and money to put a new practice into place.

I also thought the RAND report put together a nice visual of what they called the “interactive process of scale-up”. As seen in Figure 1 below, the developer of the practice being scaled, the school, and the district/state are all interacting with the teacher in each classroom, and providing differing levels of support, money, and establishment of policies and infrastructure. This figure highlights the key actors taking place in a scale-up that district administrators need to keep in mind.

According to Elmore, “Problems of scaling are deeply rooted in the incentives and cultural norms of the institutions, and can not be fixed with simple policy shifts or exhortations from people with money.” (p. 25) I would agree with this statement, and I think the research above suggests some more complex steps that need to be taken to make ripe contexts across schools for adopting new practices.

---

[1] Elmore. R.F. (Spring 1996). Getting to Scale with Good Educational Practice. Harvard Educational Review. Vol. 66, No. 1.

[2] Glennan, T. K., Bodilly, S.J., Galegher, J.R., Kerr, K. A. (2004). Expanding the Reach of Educational Reforms: Perspectives from Leaders in the Scale-up of Educational Initiatives. RAND Corporation. Prepared for the Ford Foundation.

Review of Research on Effective Uses of an Early Warning Indicator

I thought it might be useful to post this information about Early Warning Indicators considering there is a lot of discussion about the development and use of indicators across many districts.  In general, it seems like early warning indicators are an important tool that researchers can develop for districts, but creating the indicator is only half the work. Schools then need to create a coordinated, coherent system of supports for these students with these indicators.

After reviewing some research, the Early Warning Indicator Working Group (comprised on personnel from SFUSD, City College, the City of San Francisco, and Stanford University and part of the Bridge to Success Program) developed this set of common findings related to effective uses of an Early Warning Indicator by schools and district. The common findings are highlighted below. Figure 1 also shows the research corresponding to the different findings. 

1)    Cross-functional teams analyzing data and coordinating services: Schools need a data system that is updated on an ongoing basis. This finding was reiterated in a number of reports.[i] Cross-functional teams need to meet to discuss data and the strategies being used with each student. These teams would also allow CBOs and other partner organizations working with the students to help coordinate their services.[ii] School staff needs professional development on how to examine the data, develop strategies based on the data, and how to work in cross-functional teams to monitor students’ progress and the implementation of these strategies.[iii]

2)    Strategies for supporting students that are shared across the school. One study described the use of Freshman Academies as a way of providing a network of relationships for EWI students. This district also instituted progressively intensive interventions like mandatory tutoring, mentors, and increased levels of communication with families.[iv] Also, the Institute for Education Science’s Practice Guide on Dropout Prevention from the What Works Clearinghouse suggests the strategy of supporting students across the schools by creating a personalized learning environment and making sure you provide rigorous and relevant instruction.[v]

3)    Strategies focused on developing positive relationships between adults and students.  A Chicago study suggests that teacher coordination of instruction and relationships with students leads to students having above expected performance in their coursework.[vi] Also, the Institute for Education Science’s Practice Guide on Dropout Prevention from the What Works Clearinghouse suggests the strategy of assigning an adult advocate to students at risk of dropping out. [vii] Also, another research-backed strategy was supporting a positive school climate as a means of increasing student engagement and providing a positive academic experience for students.[viii] San Francisco high schools like Mission and Lincoln that have a high level of implementation for the EWI strategies seem to focus on strategies like student engagement.

4)    Strategies centered on academic experiences. Students’ academic experiences, as points 3 and 4 suggest, is the one strategy that strung throughout all of the research. The research from Chicago emphasizes, “Teachers working together in a coordinated way – taking responsibility for the whole school; providing relevant, coherent instruction; and developing strong relationships with students – most strongly distinguishes schools with above expected students performance in courses.”[ix]

Figure 1: Common recommended strategies across the research reviewed

Research	Data systems & coordination	School-wide strategies	Strategies focused on relationships	Strategies centered on academic experiences
Herzong, et al. 2012	x	x	x
Dynarski, et al. 2008	x	x	x	x
Pinkus, 2008		x
Allensworth and Easton, 2007	x		x	x
Kennelly & Monrad, 2007	x			x

---

[i] Herzog, L., Davis, M.D., Legters, N. (2012). Learning What It Takes: An initial look at how schools are using early warning indicator data and collaborative response teams to keep all students on track to success. Johns Hopkins University School of Education Center for Social Organization of Schools.

[ii] Dynarski, M., Clarke, L., Cobb, B., Finn, J. Rumberger, R. and Smink, J. (2008). Drop Out Prevention: A Practice Guide. (NCEE 2008-4025). Washington, D.C: National Center for Education Evaluation and Regional Assistance, Institute for Education Sciences. Retrieved from http://ies.ed.gov/ncee/wwc.

[iii] Herzog, et al.

[iv] Pinkus, L. (August 2008). Using Early-Warning Data to Improve Graduation Rates: Closing Cracks in the Education System. Policy Brief.  Alliance for Excellent Education.

[v]Dynarski, et. al. 2008.

[vi] Allensworth, E.M. and Easton, J.Q. (July 2007). What Matters for Staying ON-Track and Graduating in Chicago Public High Schools. Research Report. Consortium on Chicago School Research at the University of Chicago.

[vii]Dynarski, et. al. 2008.

[viii] Kennelly, L. and Monrad, M. (October 2007). Approaches to Dropout Prevention: Heeding early warning signs with appropriate interventions. National High School Center at the American Institute for Research.

[ix] Allensworth and Easton, 2007, p. 33-34.

Context Matters for School Improvement

Many reformers in education talk about scaling effective practices as a means for school improvement.  If only they could replicate the essential elements of effective schools as a means to solve our problems in education.

Yet, as many who have tried know, it is challenging to take a successful school and replicate that set of effective practices in another school. If this is the case, then what can educators learn from effective schools studies? How should this research help educators support school improvement?

I would argue that educators examine the findings from effective school studies through the lens of their own school or schools’ context.  While this statement may seem straight forward, it is actually a multi-layered process that can take time and effort. What contextual factors are most at play in their unique school or set of schools that interact with the characteristics of those effective schools? While most educators assume they know their school community, do they really know all the variables at play?

I think we need to go all the way back to Elliot Eisner’s[1] description of schools and districts as “living systems,” where Eisner encourages reformers to focus on the quality of processes within schools as well as the outcomes of schooling. Eisner encourages researchers (and I would argue practitioners, too) to use an approach to education reform that pays attention to the processes in schools as well as the context in which those processes happen.

Some districts embark on ambitious reforms with specific frameworks outlining the key elements of effective schools.  Whether it is the characteristics of effective schools in the famous 90/90/90 study[2] or Bryk et al.’s[3] five essential supports framework, these characteristics must be explored through the lens of the context of that specific school, district, and community.

To give an example, my dissertation was an effective schools study, examining a set of elementary schools in San Francisco through the lens of the five essential supports for school improvement stemming from Bryk et al.’s work in Chicago.  Yet, even while studying these schools, the San Francisco context weighed so heavily on the case study findings, that in fact the most frequent themes across the schools looked a bit different than Bryk, et. al’s five essential supports.  San Francisco’s strong history of court ordered consent decrees to support desegregation as well as services for English learners based on their Lau Plan were especially influential as well as the current strategic plan initiatives at the time of the study and the district’s wide use of weighted student formula.  These characteristics are in fact unique to the context of San Francisco, and influences how the five essential supports interact in a school setting towards improvement.

As we describe in our report, Elementary Schools for Equity,[4] the common characteristics across these effective elementary schools in San Francisco were slightly different than the common characteristics across the Chicago schools.  I would argue these difference stemmed from the contextual factors at work in San Francisco. For example, the framework stemming from Bryk, et. al’s work has leadership acting as a catalyst for change, when in fact I would argue that leadership in San Francisco’s effective schools acted as a foundation for not just change, but the whole school ecology. Unique to San Francisco, principals in fact had much leeway over their budgeting process and had additional support for personnel issues through SFUSD’s programs like their Peer Assistance Review (PAR), which helped support struggling teachers.

To study a school’s context, I might suggest studying the ecology across Eisner’s five major dimensions of school reform:

1. The intentional - What really counts in schools? What does this school value and who does this school serve?
2. The structural – How do you organize subjects, time and roles?
3. The curricular – What is being taught? What content, activities do we use to deliver curriculum and how is the curriculum organized?
4. The pedagogical – How is the curriculum delivered? What instructional practices do teachers use? How does the school help teachers build their instructional capacity?
5. The evaluative – What is the quality of the content being taught, the teaching, etc.?

The culture and histories of schools, communities, and districts have shaped these five dimensions that Eisner describes, and the characteristics of these dimensions will interact with those the findings and frameworks from effective schools studies being implemented during a reform process. Therefore, while embarking on any new school improvement or scaling effort, the context of schools and districts needs be examined so that an attempt to implement the characteristics of “effective schools” can be tailored to each unique school context.

---

[1] Eisner, E. W. (1992). Education Reform and the Ecology of Schooling. Teachers College Record. Volume 93 Number 4, p. 610-627.

[2] Reeves, D. B. (2000). Accountability in Action: A Blue Print for Learning Organizations. Denver, CO: Advanced Learning Press.

[3] Bryk, A.S., Sebring, P.B., Allensworth, E., Luppescu, S., Easton, J.Q. (2010) Organizing Schools for Improvement: Lessons from Chicago. University of ChicagoPress.

[4] Wentworth, L., Kessler, J., and Darling-Hammond, L. Elementary Schools for Equity: Policies and Practices that Help Close the Opportunity Gap. Stanford Center for Opportunity Policy in Education. Retrieved on May 21, 2014 from https://edpolicy.stanford.edu/sites/default/files/publications/elementary-schools-equity-policies-and-practices-help-close-opportunity-gap.pdf

Organizing schools to optimize the use of student supports - Co-written by Hayin Kim, SFUSD Director of Community Schools

You know that toolbox you have at home?   Have you ever gone to look for a hammer, and for some reason you find three screwdrivers instead?  Do you improvise to make the screwdriver work like a hammer?  This metaphor can extend to the myriad of supports and strategies that schools often look to as they search for the best approach to “fix problems” that get in the way of student success. Just like in the home toolbox, while schools might have a random compilation of supports, there is seldom an organized way to make sure that they have what they need at the right time, that tools are easy to access, easy to use, and all work towards a common purpose. 

Stanford University’s Challenge Success describes one way to help schools make sense of their “toolkit," and identify the additional supports they need. The process illustrates a coherent approach to implementing a Community Schools strategy; that is, the process by which schools can assemble school and community resources that suit the needs and assets of students and staff.   Challenge Success emphasizes the importance of developing a clear vision of success before assembling the strategies to realize that vision. Such an approach is especially important given the multiple variables that impact effective teaching and learning – i.e. teacher preparation and planning time, curriculum, classroom climate and culture, course scheduling, family engagement, student health and well-being, social-emotional development.  Given these complexities, developing a coherent and effective toolbox for schools is more than just gathering appropriate hammers and screwdrivers.  It also means attending to the structures, protocols, systems, and practices that help support all members of the school community.  Thus, coherence does not mean everyone doing the same thing at the same time, but using each tool in an appropriate way – the right tool at the right time for the right student in the right context.

Many schools may be expected to benefit from instructional and non-instructional supports like Response to Intervention training, Restorative Practices, Grade Level Teams, Instructional Leadership Teams, School Site Councils, Positive Behavior Intervention Systems, Wellness Center, Peer Resources, Beacons, after school, nurse, social worker, family liaison, counselors, and the list goes on.  However, none of these student or school resources alone offers a comprehensive solution to academic success. What can get lost in all of this assembly is why these supports are needed, or how all the elements fit together. 

Challenge Success posits that schools must first build a clear vision across the school community – including faculty, students, administrators, and families – and use student outcomes data to validate progress towards that vision, and to determine the efficacy and complementarity of various student and school supports. Challenge Success works with school communities to identify the root causes of specific problems and then leads teams through an action planning process to change or move one part of their school efforts – a well-organized toolbox – towards the vision.

To learn more about Challenge Success at Stanford University, go to: http://www.challengesuccess.org