Creating a Coherent System to Support - RAND Corporation

Loading...

C O R P O R AT I O N

Creating a Coherent System to Support Instruction Aligned with State Standards Promising Practices of the Louisiana Department of Education Julia H. Kaufman, Lindsey E. Thompson, and V. Darleen Opfer

S U M M A RY ■ Teachers are often bombarded with contradictory and changing messages about how • New findings from two American Teacher Panel surveys they should be helping students meet state standards and point to some large and intriguing differences between raising student achievement. The push across the United surveyed teachers from states that have adopted ComStates for uniform and rigorous standards—as well as mon Core State Standards and those in one particular high-quality, aligned assessments—represents a chance state: Louisiana. for states to reimagine ways to provide teachers with • Compared with other teachers nationally, Louisiana clearer and more-coherent messages about what they can teachers use some Common Core–aligned instrucbe doing every day in their classrooms to support student tional resources at a higher rate than other teachers, learning. Yet, to date, we have little clear evidence that demonstrate a better understanding of their Common any state-level work is making a difference for teachers’ Core–aligned standards, and report undertaking more implementation of standards; nor do we have evidence of instructional activities that align with their standards. state strategies that could be supporting strong implemen• Louisiana Department of Education strategies that could tation of standards. be contributing to these results include their work to The impetus for this report is new evidence that state create a coherent environment for instruction, transpardepartment of education work to align instruction with ent and regular communication about academics, and standards may make a difference for teachers’ practices support for local decisionmaking. and understanding about their state standards. Using data from the RAND American Teacher Panel, we found that Louisiana teachers were more likely than other teachers to consult resources that address their state standards, and they reported teaching—and thinking about teaching—in ways that differ from U.S. norms and that are more aligned with Common Core State Standards. We examine Louisiana Department of Education strategies that could be contributing to these results, including a coherent academic strategy focused on alignment and quality across systems supporting standards, transparent and regular communication about academics across layers of the education system, and support for local decisionmaking and ownership of change by districts and teachers. This report is intended to provide guidance to states about sensible state systems that give educators coherent messages and concrete tools to help students meet high academic standards. Ke y findin gs

2

I NTRO DUCTI O N For state standards to help improve student achievement, they must change the minds and practices of teachers. Yet one of the biggest takeaways from decades of standards-based reform efforts is that teachers have uneven knowledge and implementation of state standards: Some teachers engage deeply in high-quality, standards-aligned practices; others engage in those practices at a superficial level; and still others are unable or unwilling to engage in them at all (Coburn, 2001, 2004; Cohen and Ball, 1990; Spillane and Zeuli, 1999; Weiss et al., 2003; Coburn, Pearson, and Woulfin, 2011; Mayer, 1999). The wide variation in how teachers address standards is not surprising. Teachers are often bombarded with contradictory messages about how they should help students meet state standards. Such messages are implicitly embedded in the standardized assessments that teachers are required to give (and through which teachers in some states are evaluated), the instructional materials they are expected to use regularly, and their professional development, all of which may be detached from one another and not aligned with their standards. Every day, in their classrooms, teachers must make sense of these messages alongside their own experiences to make decisions about what and how to teach. As early as the 1990s, O’Day and Smith (1993) encouraged states to embrace a “systemic” approach to guiding instruction through coherent and aligned curriculum frameworks, curriculum materials, professional development, and accountability. O’Day and Smith suggested that state departments of education are uniquely able to influence all parts of the kindergarten through 12th grade (K–12) system. By using a systemic reform approach, states could ensure that all aspects of the system are aligned—from standards and curricula to professional development and assessments—toward the singular goal of standardsbased instruction, thereby sending a focused and consistent message to schools and teachers about what they should be doing. Yet, historically, this alignment is not easily undertaken by state administrators, who are often under pressure from the public, commercial vendors, and legislators to revise standards, assessments, professional development, and curricula frequently. The result is that state departments of education might be sending conflicting messages to teachers about instruction from one year to the next, and at the same time, district administrators might be sending additional and differing messages to teachers about what they should be doing in the classroom (Cohen, 1995; Fuhrman and Elmore, 1990).

The adoption in many states of Common Core State Standards (CCSS)—or state-developed standards that are similar to CCSS—represents a new opportunity for states to get it right. It is a chance to provide teachers with more-coherent and clear messages about what they can be doing every day in their classrooms to help students meet standards. Yet, in the words of David Cohen (1995, p. 12), systemic reforms (like those potentially represented by CCSS) are “not being grafted onto the same tree in each state.” Even states that have adopted CCSS are quite varied in what they are doing to support teachers’ work to address standards. For example, a dwindling subset of states that have adopted CCSS have also adopted one of the two common assessments intended to gauge progress toward CCSS: Smarter Balanced and Partnership for Assessment of Readiness for College and Careers (PARCC) (Gewertz, 2016). And states vary a great deal in the professional development and curricula that they recommend or support teachers to use (Coburn, Hill, and Spillane, 2016). Thus far, studies have provided some limited evidence on educators’ perceptions and capacity to implement CCSS (Kane et al., 2016; Opfer, Kaufman, and Thompson, 2016). But studies have not provided clear evidence of state-by-state differences in teachers’ implementation of CCSS or highlighted state strategies that may be supporting thoughtful and strong implementation of CCSS. State policymakers and educators thus have little guidance on best practices by which teachers can be supported to help students meet state standards.

The Purpose of This Report The catalyst for this report is new evidence that state department of education work to align instruction with standards may make a difference for teachers’ practices and understanding of their state’s standards. Several times over the past few years, the RAND Corporation’s American Teacher Panel (ATP) has surveyed a standing sample of approximately 2,700 K–12 classroom teachers across the United States about their perceptions and instruction. In addition to providing nationally representative data from teachers across the United States, the ATP provides data at the state level for a select number of states. New findings from two ATP surveys—one fielded in June 2015 and another fielded in October 2015—point to some large and intriguing differences between surveyed teachers from states that have adopted CCSS and those in one particular

3

state: Louisiana. Specifically, Louisiana teachers are using more instructional materials aligned with Common Core, and they report thinking and teaching in ways that are more in line with the tenets of Common Core. Over the past several years, Louisiana students have made some significant strides in being more career and college ready, despite Louisiana being one of the poorest states in the country.1 In 2013, Louisiana became one of only 12 states that required all 11th graders to take the ACT collegereadiness assessment. Two years later, in 2015, Louisiana students gained more points in their composite ACT scores, on ­average, than students in any of the other 12 states. The number of ­L ouisiana students taking Advanced Placement (AP) courses has more than doubled between 2012 and 2016.2 Furthermore, Louisiana’s high school graduation rate and the number of students enrolled in college are both at all-time highs.3 In addition, compared with their performance in 2013, Louisiana’s fourth-grade students achieved the highest growth among all states on the 2015 National Assessment of Educational Progress (NAEP) reading test.4 This remarkable progress is likely due in part to changes teachers are making to their instruction. In this report, we examine teachers’ survey responses about their implementation of state standards in Louisiana compared with the responses of teachers in other states that have adopted CCSS. We then identify state policies and practices in Louisiana that could be supporting teachers’ work. This report is intended to provide guidance to states about sensible state systems that give educators coherent messages and concrete tools to help students meet high academic standards.

Data and Methods The ATP is a randomly selected, nationally representative panel of American K–12 public school teachers. The ATP also includes state-representative samples in four states: California, Louisiana, New Mexico, and New York. For the June and October 2015 web-based surveys on which these findings are based, we focused on three main aspects of teachers’ implementation of state standards: their use of instructional materials, their understanding of approaches and practices aligned with their state’s standards and CCSS, and the standards-aligned practices in which their students are engaged in the classroom. The June 2015 ATP survey explored teachers’ instructional practices and their understanding of their state’s standards for mathematics and English language arts (ELA), while the October 2015 survey focused only on teachers’ use of instructional materials. For simplicity’s sake, throughout this report, we refer to states that have adopted CCSS or modified versions of those standards as SACC (for standards adapted from Common Core) states. In the findings for this report, we focus on survey responses by mathematics and ELA teachers to both surveys, and we compare responses by Louisiana teachers with those in other SACC states.5 We consider differences in teachers’ perceptions to be significant when they are unlikely to have occurred by chance (i.e., a p-value of 0.05 or smaller in statistical tests comparing Louisiana teachers and those from other states).6 Details on sample size and respondents are included in Table 1. To ensure representativeness, panel members were originally sampled randomly from across the nation. The teacher sample includes all full-time public school teachers in grades K–12 in all subjects, including teachers of special

Table 1. ATP Response Rates Across the United States and in Louisiana Respondents

Response Rate (%)

2,745

1,705

62

3.7

381

249

65

8.5

2,018

1,168

58

4.5

327

170

52

8.5

Total Panelists

Maximum Margin of Error (%)

June 2015 survey (to all teachers) All U.S. teachers Louisiana teachers

October 2015 survey (only to math and ELA teachers) All U.S. teachers Louisiana teachers

4

education students and English-language learners. However, for the October 2015 survey, only math and ELA teachers were surveyed. For the October and June 2015 surveys, respondents were paid a small monetary incentive to complete the surveys. ATP response rates are similar to those of other national surveys,7 but nonresponse could lead to some bias in our estimates. To address this potential bias, the weighted estimates provided in this report are based on a model for nonresponse that gives more weight to teachers in subgroups that were less likely to respond to our survey.8 Based on the differences we observed between survey selfreports of Louisiana teachers and those in other states, we opted to gather more information about Louisiana state policies and processes by interviewing Louisiana Department of Education (LDOE) officials and reviewing documents and resources that the state provides to school administrators and teachers. In late Spring 2016, we interviewed five high-level LDOE officials who oversee major aspects of the state’s education work. Interviews were semi-structured and intended to help us understand LDOE policies and practices related to teachers and teacher learning, as well as general LDOE policy priorities and strategies for implementing policies and goals. We reviewed public documents and resources available to school administrators and teachers on the LDOE website, as well as other summary presentation documents and other educator resources that LDOE interviewees provided to us. Through iterative qualitative coding of the interview data, we identified major themes and key state strategies for supporting teachers’ knowledge and practice. Through our documentation analysis, we identified the key tools and resources supporting LDOE strategies.

Limitations Our survey methods do not allow us to provide causal evidence of connections between state policies and teacher practices across the state. We also did not examine change in teachers’ perceptions and practices over time for this analysis, although we plan to do so in future work. The survey data shared in this report thus only capture perceptions and practices at a single time point. It is possible that teachers in Louisiana already engaged in these perceptions and practices before LDOE put its current strategies in motion. In addition, our analyses of LDOE policies and practices are based only on interviews with state officials and a review of public documents that LDOE provides for Louisiana educators. We did not collect information from school administrators and teachers to confirm LDOE policies

and practices, aside from what we collected regarding teachers’ knowledge of standards and their instructional practices. However, at the least, our work can point to concrete department of education strategies that could be making a difference in a state where student achievement and teacher work in schools looks quantitatively different from national averages.

Organization of the Report The next section summarizes significant differences in the responses of teachers in Louisiana and those in other SACC states on three key markers for implementation of state standards. We then discuss LDOE strategies that we hypothesize could be supporting teachers’ implementation of standards. We conclude the report with a final summary and consider implications of our findings for states and districts.

K E Y D I FFE R E N C ES BET W E E N LO U ISIANA M ATH AN D E N G LISH L AN GUAG E ARTS TE AC H E RS AN D TH E I R CO U NTE R PARTS I N OTH E R SACC STATES In this section, we compare differences in the responses of teachers in Louisiana and those in other SACC states on the following three markers of teachers’ implementation of standards: • using instructional materials aligned with standards • understanding state standards • implementing standards-aligned classroom practices.9 Louisiana teachers diverged from other states’ teachers in all three areas, with particular differences in teachers’ mathematics instructional materials, the practices teachers think are aligned with their ELA standards, and teachers’ use of grade-­ appropriate texts.

Marker 1: Using Instructional Materials Aligned with State Standards Compared with teachers in other states, more Louisiana teachers regularly used or consulted standards-aligned instructional resources. To help students meet state standards, teachers need access to high-quality, standards-aligned instructional materials. In

5

As seen in Figure 1, 72 percent of Louisiana math teachers reported using EngageNY for their instruction in October 2015, compared with 36 percent of math teachers in other SACC states. We also asked teachers about their use of Eureka Math apart from EngageNY as a separate item in our survey. Similarly, 70 percent of Louisiana math teachers reported using Eureka Math, compared with only 10 percent of teachers in

Figure 1. Use of EngageNY Materials by Math and ELA Teachers Percentage of surveyed teachers reportedly using EngageNY materials

SACC states, including Louisiana, that means alignment with the Common Core State Standards. Although many instructional resources claim alignment with CCSS, recent reviews of published mathematics materials suggest that few actually are (Polikoff, 2015; EdReports, undated-b). In its reviews of 26 K–8 instructional materials for mathematics, EdReports found that only Bridges in Mathematics’ and Eureka Math’s curricula were aligned with CCSS across all grades and in the areas of (1) focus and coherence and (2) rigor and mathematical practices.10 A version of Eureka Math is available for free online on the EngageNY.org website, but it is also available as a textbook series published by Great Minds. EdReports also recently reviewed ELA materials for grades 3–8 for their alignment with CCSS, including text quality and complexity; alignment with standards components; and building of ­k nowledge with texts, vocabulary, and tasks (EdReports, undated-a). Of the seven textbook series that EdReports reviewed, Amplify ELA, Expeditionary Learning, and ReadyGEN met expectations in all domains and at all grade levels for which they were reviewed.11 In the October 2015 ATP survey, we asked math and ELA teachers across the United States to tell us how frequently they drew upon a large list of online and published materials for their instruction. The results suggest that Louisiana teachers were more likely than teachers from other SACC states to draw upon materials connected with CCSS. Three large and unexpected differences emerged in this regard: • Compared with math teachers in other SACC states, Louisiana math teachers were far more likely—by more than 30 percentage points—to draw upon EngageNY or Eureka Math for their instruction. • Louisiana ELA teachers were significantly more likely than ELA teachers in other SACC states to draw upon ­EngageNY materials for their instruction. • Compared with teachers in other SACC states, 10–20 percentage points more math and ELA teachers in Louisiana reported consulting some online resources aligned with or related to state standards.

Teachers in other SACC states Teachers in Louisiana 72

48 36

Math teachers*

30

ELA teachers*

SOURCE: October 2015 ATP survey. NOTE: An asterisk denotes that the difference between Louisiana teachers and those in other SACC states was significant. RAND RR1613-1

other SACC states. Most of the teachers who reported using EngageNY also reported using Eureka Math. Specifically, 67 percent of teachers reported use of both EngageNY and Eureka Math. Many of those teachers may have been using only the version of Eureka Math available within ­EngageNY. Altogether, nearly 90 percent of Louisiana math teachers reported using either EngageNY or Eureka Math for their instruction, compared with 44 percent of math teachers in other SACC states. Beyond Eureka Math and EngageNY, math teachers in Louisiana also reported using some other materials significantly more than their counterparts in SACC states, including Go Math!, Glencoe Math, and Pearson Prentice Hall Algebra I. We did not find evidence that other resources used by Louisiana math teachers are as aligned with Common Core as Eureka Math is. EdReports (undated-b), for example, suggested that, for the criterion of focus and coherence, Go Math! and Glencoe Math aligned with Common Core at some grade levels, but they did not exemplify rigor and mathematical practices aligned with Common Core as frequently as Eureka Math did. Thus, Louisiana math teachers may not always be using resources aligned with Common Core, although they seem to be using some standards-aligned resources more than teachers in other SACC states. As with their counterparts who teach math, Louisiana ELA teachers also reported using EngageNY more than ELA teachers in other SACC states (48 percent compared with a little more than 30 percent, respectively). Louisiana ELA teachers also reported using a range of other ELA instructional resources

6

more than ELA teachers in other SACC states, including Accelerated Reader, Core Knowledge Language Arts, Open Court Reading, and Reading Wonders. In the October 2015 survey, 88 percent of Louisiana math and ELA teachers reported consulting their state department of education’s website for ideas and materials to integrate into their instruction, compared with 68 percent of teachers in other SACC states (see Figure 2). In fact, 27 percent of Louisiana teachers reported going to their state website for instructional materials or ideas at least once a week. By comparison, only 12 percent of teachers in other SACC states reported

doing so. Similarly, more Louisiana teachers appeared to seek out some other online resources that explicitly address Common Core, such as Corestandards.org. On the other hand, Louisiana teachers were also more likely to seek out some resources that are not as clearly aligned with Common Core, such as Teacherspayteachers.com and Teachingchannel.org, which offer a wide variety of example lessons and activities. The higher percentages of Louisiana teachers consulting these resources may suggest that Louisiana teachers feel the need to consult online resources more, in general, compared with their peers in other states.

Figure 2. Use of Online Materials by Math and ELA Teachers Math and ELA teachers in other SACC states

Math and ELA teachers in Louisiana

95

Google.com

93 68

State department of education website*

88 73

Teacherspayteachers.com*

85 77

Pinterest.com

83 48

Readworks.org

56 30

Teachingchannel.org*

47 33

Corestandards.org*

Learnzillion.com*

43 25 45 47

Readwritethink.org

Khanacademy.org

44 45 40 Percentage of surveyed teachers reportedly using the resource

SOURCE: October 2015 ATP survey. NOTE: An asterisk denotes that the difference between Louisiana teachers and those in other SACC states was significant. RAND RR1613-2

7

Marker 2: Understanding State Standards Louisiana teachers demonstrated a more accurate understanding of approaches and practices aligned with CCSS for ELA compared with teachers in other SACC states. Using standards-aligned instructional materials is an important first step for teachers to address state standards. Yet if teachers do not deeply understand their standards—or the instructional practices that are aligned with them—their instruction may fall short of helping students meet those standards. In particular, according to studies of CCSS, there are several major “instructional shifts” that teachers must make in order to help students meet CCSS or similar state standards, which would apply for those in SACC states (Brown and Kappes, 2012; Shanahan, 2013; Student Achievement Partners, 2016). Those shifts for ELA or literacy include the following: • Students should regularly be exposed to challenging, grade-appropriate texts. • Texts must be at the center of instruction (e.g., teachers should consistently require students to cite evidence from texts to justify their claims and ask text-specific questions that require students to deeply read texts). Although there is no explicit guidance in CCSS on the extent to which teachers should select texts at students’ individual reading levels, the standards do make clear that teachers should place an emphasis on using grade-level texts with the whole class, which could conceivably reduce the time that teachers spend using leveled readers and other texts written at students’ individual reading levels. A majority of teachers in SACC states do not demonstrate a clear understanding of these shifts, even though standards in SACC states are very similar to CCSS. For example, Opfer, Kaufman, and Thompson (2016) noted that two-thirds of ELA teachers in SACC states believed that their state standards advocate teachers providing texts for students based on individual reading levels, which contradicts the CCSS emphasis on providing complex texts for all students. In addition, more than three-fourths of U.S. ELA teachers believed that their standards encouraged them to teach reading skills “first” and apart from texts so that students would be able to apply these skills to any text, which also contradicts CCSS. In contrast, only about 20 percent of teachers indicated that their standards encouraged them to focus, first, on teaching texts and embedding

instruction on reading skills as tools to understand any text (Opfer, Kaufman, and Thompson, 2016). Compared with ELA teachers in other SACC states, higher percentages of Louisiana ELA teachers identified practices and approaches aligned with Common Core. First, as indicated in Figure 3, lower percentages of Louisiana ELA teachers than teachers in other SACC states thought that “selecting texts for individual students based on their reading levels” was an approach aligned with their state standards. This finding suggests that Louisiana teachers are embracing the CCSS focus on grade-appropriate texts more than teachers in other SACC states. In addition, most Louisiana ELA teachers perceived that their standards encourage them to teach particular texts and organize reading-skills instruction around the texts, rather than teach reading skills first so that students can apply them to any text (see Figure 4). Thus, compared with ELA teachers in other SACC states, higher percentages of Louisiana teachers were thinking about their ELA instruction in ways that were aligned with CCSS. On the other hand, we observed no differences between teachers in Louisiana and those in SACC states regarding several other approaches that are not clearly aligned with CCSS, including using abridged or adapted versions of complex texts for struggling readers and assigning complex novels for all students to read.

If teachers do not deeply understand their standards—or the instructional practices that are aligned with them— their instruction may fall short of helping students meet those standards.

8

Figure 3. ELA Teachers’ Understanding of Approaches for Selecting Texts Aligned with Common Core ELA teachers in other SACC states

ELA teachers in Louisiana

70

Selecting texts for individual students based on their reading levels*

47 48

Using abridged or adapted versions of complex texts for struggling readers

53

Selecting texts for a class based on qualitative factors (e.g., knowledge demands) and quantitative factors (e.g., word length)

47 40 18

Assigning complex novels that all students in a class are required to read

Other approach*

24 6 1

I don’t know

4 10 Percentage of surveyed teachers reporting that the approach aligns with standards

SOURCE: June 2015 ATP survey. NOTE: An asterisk denotes that the difference between Louisiana teachers and those in other SACC states was significant. RAND RR1613-3

Percentage of surveyed teachers reporting that the approach aligns with standards

Figure 4. ELA Teachers’ Understanding of Reading Instructional Approaches Aligned with Common Core ELA teachers in other SACC states ELA teachers in Louisiana

75

49

49

21

4 Focus on reading skills first and then organize teaching around them so that students will apply these skills to any text*

Teach particular texts that students should read and then organize instruction around them, teaching reading skills as tools to understand text*

Other approach or “I don’t know”

SOURCE: June 2015 ATP survey. NOTE: An asterisk denotes that the difference between Louisiana teachers and those in other SACC states was significant. RAND RR1613-4

2

9

One of the key instructional shifts emphasized by CCSS is that math teachers should narrow the focus of their instructional time to only the content that is specific to standards for the grade levels in which they teach (Porter et al., 2011; Student Achievement Partners, 2016). This shift is intended to curb the “inch deep, mile wide” approach to math instruction documented by Schmidt, McKnight, and Raizen (1997), who found that U.S. mathematics curricula and texts address far more topics at many grade levels compared with curricula in other countries. In one recent nationally representative survey, Bay-Williams (2016) found that teachers in states that adopted CCSS reported addressing the appropriate major topics noted in CCSS for their grade levels. We also found via the ATP survey that high percentages of math teachers in SACC states knew which topics were aligned with CCSS at their grade levels. However, our ATP survey findings also indicated that K–8 math teachers incorrectly identified additional topics as being aligned with CCSS at their grade levels (Opfer, Kaufman, and Thompson, 2016). These findings suggest that teachers are still likely teaching too many topics per grade level, including some that are not aligned with CCSS. Interestingly, compared with the national trends, Louisiana K–5 math teachers were more likely to choose the topics aligned with CCSS at their grade levels and—equally importantly—did not identify topics at their grade levels as being aligned with CCSS when they were not. Specifically, more than half of Louisiana K–5 math teachers were able to identify all the CCSS topics aligned with their grade levels that were highlighted in the survey and did not choose any additional topics as part of what their standards expect them to teach. In contrast, only 36 percent of K–5 teachers in other SACC states were able to choose all the topics aligned with CCSS at their grade levels and no other topics. We did not observe differences between math teachers of grades 6–8 in Louisiana and those in other SACC states regarding identification of math topics aligned with state standards at their grade levels (see Figure 5).

Figure 5. Math Teachers’ Understanding of Major Math Topics Aligned with Common Core at Their Grade Levels Percentage of surveyed teachers choosing both correct standards and nothing else

Compared with math teachers in other SACC states, more math teachers in Louisiana identified the CCSSaligned math topics for their grade levels.

Teachers in other SACC states Teachers in Louisiana

57 36 22

Grades K−5*

22

Grades 6−8

SOURCE: June 2015 ATP survey. NOTE: An asterisk denotes that the difference between Louisiana teachers and those in other SACC states was significant. For this set of questions, we excluded high school teachers because those teachers do not always teach the same content at the same grade level (e.g., a ninth-grade high school teacher could be teaching algebra, geometry, or general mathematics). RAND RR1613-5

Marker 3: Implementing Standards-Aligned Classroom Practices Compared with teachers in other SACC states, Louisiana ELA teachers were more likely to report that their students engaged in some practices aligned with CCSS, including more text-centered practices. Lastly, we examined differences between teachers in Louisiana and those in other SACC states in regard to the frequency of CCSS-aligned practices in their classrooms, including their reports of students’ engagement in work outlined in the CCSS Anchor Standards for ELA/Literacy (Common Core State Standards Initiative, 2016a, 2016b) and Common Core Standards for Mathematical Practice (Common Core State Standards Initiative, 2016c). Of the 32 ELA Anchor Standards in CCSS focused on reading, writing, and speaking and listening, we asked about 12 in the ATP survey, including five explicitly focused on reading and text-centered practices.

10

Based on our results, Louisiana ELA teachers diverged considerably from our national teacher sample in regard to student engagement in several classroom practices (see Figure 6). Specifically, compared with their counterparts in other SACC states, 15–16 percentage points more Louisiana ELA teachers reported that their students engaged in the following CCSSaligned practices: • Students use evidence from a text to make inferences or support conclusions drawn from a text. (CCSS ELA Anchor Standard 1 for Reading) • Students analyze the structure of texts, including how specific sentences, paragraphs, and larger portions of text relate to each other and the whole. (CCSS ELA Anchor Standard 5 for Reading)

• Students demonstrate a command of conventions of standard English when writing or speaking. (CCSS ELA Anchor Standard 1 for Language) Additionally, compared with national averages, much higher percentages of Louisiana teachers reported that students spend their reading time on the same grade-level texts, and lower percentages of teachers reported that students read different texts depending on their reading levels (see Figure 7). These findings are perhaps logical given that Louisiana teachers are less likely than other U.S. teachers to think that their standards encourage them to select different texts for students based on their reading levels. But the findings offer important evidence that teachers report doing what they think their standards advocate (i.e., focusing on grade-appropriate texts).

Figure 6. ELA Teachers’ Implementation of Standards-Aligned Practices ELA teachers in other SACC states

ELA teachers in Louisiana

57

Use evidence from a text to make inferences or support conclusions drawn from the text*

73 50

Demonstrate a command of conventions of standard English when writing or speaking*

66 41

Participate in a range of conversations and collaborations with diverse partners

38

Use a range of general academic and domain-specific words and phrases sufficient for college and career readiness

35 42 33

Connect literacy instruction to other content (e.g., science, social studies)

33 26

Read a nonfiction text in the classroom

27 20

Adapt speech to a variety of contexts and communicative tasks

25 18

Analyze the structure of texts, including how portions of text relate to each other and the whole* Strengthen writing by planning, revising, editing, rewriting, or trying a new approach

33 14 13 Percentage of surveyed teachers indicating that their students engaged in the practice “daily or almost daily”

SOURCE: June 2015 ATP survey. NOTE: An asterisk denotes that the difference between Louisiana teachers and those in other SACC states was significant. RAND RR1613-6

11

Percentage of time, according to teacher estimates

Figure 7. ELA Teachers’ Estimates of the Percentage of Reading Time That Students Spend on Grade-Level Versus Reading-Level Texts ELA Elementary teachers in other SACC states ELA Elementary teachers in Louisiana

73 60 54

56 46

44

39 27

In class* Outside of class* Students read the same text written at grade level

In class* Outside of class* Students read different texts depending on their reading levels

SOURCE: October 2015 ATP survey. NOTE: An asterisk denotes that the difference between Louisiana teachers and those in other SACC states was significant. RAND RR1613-7

Secondary math teachers in Louisiana were more likely than those in other SACC states to engage students in some Common Core Standards for Mathematical Practice. We observed a small number of substantial differences between Louisiana math teacher practices at the secondary level (grades 6–12) and those of secondary math teachers in other SACC states. We asked teachers about their engagement in practices parallel to six of the eight Common Core Standards for Mathematical Practice (Common Core State Standards Initiative, 2016c). Roughly 90 percent of secondary math teachers in Louisiana reported that they ask students to explain and justify their work and ask students to use math symbols and language appropriately on a daily basis compared with only about two-thirds of teachers in other SACC states. But we observed no additional significant differences between secondary teachers in Louisiana and those in other SACC states on any other standards-aligned practices, and we did not observe significant differences between elementary teachers. Taken together, our ATP survey findings offer compelling new evidence that teachers in Louisiana are both teaching and thinking about teaching in ways that are more aligned with CCSS than teachers in other CSSS-aligned states. The

next section of our report explores reasons for these differences by examining LDOE strategies that could be connected with teachers’ work in schools.

STATE SYSTEMS AN D STR ATEG I ES SU PP O RTI N G LO U ISIANA TE AC H E RS’ IM PLEM E NTATI O N O F STAN DAR DS Our ATP findings show that Louisiana teachers report doing work that is more aligned with CCSS than their counterparts’ reports from other SACC states. What state work could be supporting these considerable differences in Louisiana teachers’ understanding and instructional practices? In this section, we explore the key strategies in Louisiana intended to support implementation of state standards. Our findings are based on analysis of interviews, documents on the LDOE website, and documents and presentations provided to us by LDOE. All of our work focuses on strategies that have been put in place since the new state superintendent for education in Louisiana took office in January 2012. Our coding of themes from our interviews and document analysis suggests that LDOE has relied on the following three

12

main strategies to support teachers’ implementation of state standards: 1. a coherent academic strategy focused on integration, ­a lignment, and quality among systems supporting standards 2. transparent and regular communication about ­academics within the state department and across layers of the ­education system 3. strong support for local decisionmaking and ownership of change by districts and teachers. We describe these strategies in more depth and then consider how they work together to create a sensible system in which teachers can understand and implement their state standards.

Strategy 1: A Coherent Academic Strategy Focused on Integration, Alignment, and Quality Among Systems Supporting Standards LDOE has examined and revised three key “systems” that all point teachers down the same pathway toward helping students meet Louisiana state standards: curricula, professional development, and student assessments. LDOE staff have worked to ensure strong alignment and coherence among these systems and, at the same time, designed them to be high quality.

Curricula LDOE considers curricula to be a linchpin in much of the department’s alignment work. One state official noted that curricula provide “an anchor” for teachers, offering crucial support for how to implement standards. Many states recommend or require various published and online resources that teachers can consult for their instruction, and these recommendations are often posted on the states’ websites. But LDOE does more than recommend textbooks and resources. First, Louisiana provides free online, annotated reviews of published K–12 textbooks for mathematics and ELA. Reviews rate materials as “Tier 1 – Exemplifies Quality,” “Tier 2 – Approaching Quality,” and “Tier 3 – Not Representing Quality.” For instructional materials, review criteria are closely related to CCSS and Publishers’ Criteria for CCSS (Coleman and Pimentel, 2012a, 2012b; Common Core State Standards Initiative, 2013a, 2013b), and LDOE hires teachers from its Teacher Leader

program to conduct the reviews (we examine Teacher Leaders in more depth later in this report). Notably, just a handful of curricula are rated as Tier 1. For mathematics, Tier 1 curricula include Eureka Math for grades K–11 (part of EngageNY); College Board Springboard Math for grades 9–11; the Math Learning Center for grades K–5; and Zearn for grades 1–4. For ELA, Tier 1 curricula include Core Knowledge ELA for grades K–3 (part of EngageNY) and Houghton Mifflin Harcourt Collections for ELA for grades 9–12.12 On its website, LDOE stresses that these reviews are intended to help districts make informed decisions about purchasing materials “that are best for their local communities.” At the same time, all Tier 1 programs receive a state contract, which makes it easier for districts to procure them. As noted earlier, it appears that Louisiana has been largely successful in “scaling” Eureka Math through this strategy: Almost 90 percent of teachers in Louisiana who responded to our survey noted using Eureka Math or the version of Eureka Math available through EngageNY for their instruction, although they also reported using some texts that have been rated below Tier 1 by LDOE. One state official told us that LDOE observed “a hole in the marketplace” regarding materials well-aligned with CCSS for ELA, particular for grades 4–8. For that reason, LDOE developed free, online ELA curricula for grades 3–12 aligned with state standards; the curricula include unit overviews all the way down to individual lessons, student handouts, teaching notes, and student assessments.13 If a district adopts the statedeveloped ELA curricula, it is free to use any of the materials that LDOE provides online, although the district has to purchase the texts (e.g., novels or books) that are part of those curricula (which are not available online). LDOE provides clear information about the costs of those texts to help districts’ decision­making.14 At the same time, the department stresses that districts are not required by the state to adopt any particular curricula or resources, including those for ELA or math. LDOE officials and personnel with whom we spoke emphasized that the provision of vetted resources has been challenging and time-consuming. Prior to 2012, districts were required to purchase and use a limited set of state-required materials. In response to recommendations from a state committee, legislators changed the laws requiring use of this narrow set of materials, and LDOE encouraged districts to work with vendors to submit materials for review. In the words of a state official, “It’s not pleasant when you give big publishers Tier 3 ratings and they have big lobbies and bring in millions of ­dollars . . . . It would have been easy to let it go.” But LDOE

13

has focused on curricula as an essential element of its strategy to support teaching, with the consistent message that districts must have clear and useful information about high-quality curricula in order to make the best textbook-adoption decisions.

Professional Development Recommending and developing curricula do not—in ­themselves—ensure high-quality, standards-aligned instruction. LDOE’s professional development strategy is explicitly intended to support curriculum implementation by targeting and supporting development that is aligned with standards and curricula. On its website, LDOE provides a “Vendor PD Course Catalogue” that lists professional development vendors and information about whether those vendors meet an array of criteria, including whether they help teachers implement a high-quality curriculum, build content knowledge, analyze the quality of student work, manage a classroom environment, provide teachers practice and feedback, and offer ongoing support through mentoring, coaching, or Professional Learning Communities.15 In particular, LDOE recommends only vendors that have made explicit linkages between their professional development and curricula designated as Tier 1 by LDOE. The department also works with vendors to offer different packages at varying price points so that districts can choose the package that works for them. While most states do provide professional development opportunities to K–12 teachers, we have not found other clear examples of state department of education work to make connections between specific professional development vendors and curricula or to investigate the quality of these vendors in such a variety of ways. LDOE also provides frequent professional development directly to Teacher Leaders, a group of 5,000 educators across the state and in each district who are not only involved in creating some of the resources available on the LDOE website but also are expected to attend state professional development trainings and then return to their districts and schools to provide professional development to their fellow educators. We discuss LDOE’s Teacher Leader program in more depth as part of the second strategy outlined later.

Student Assessments Summative and formative assessments are the last puzzle piece of aligned systems to support state standards. Louisiana was one of nine states in which students took the PARCC test in

LDOE’s professional development strategy is explicitly intended to support curriculum implementation by targeting and supporting development that is aligned with standards and curricula. 2015. However, in spring 2015, Louisiana passed legislation revising state standards and requiring that fewer questions on the annual standardized state test come from PARCC assessments. In response to that change, LDOE staff worked to ensure that the state standards and assessments remained high quality. One state official commented, “[Our] team of item writers and reviewers—I would daresay—are some of the best in the country.” Several LDOE staff with whom we spoke highlighted the role of assessment in supporting accountability. LDOE gives letter grades to schools based on student achievement data. One LDOE official noted that issuance of letter grades “gets people focused on student learning,” and another LDOE staff m ­ ember indicated that the department’s work to support instructional improvement would be less influential without accountability mechanisms. At the same time, one official commented that “accountability is meaningless absent it producing a coherent reaction that is aligned to the substance of the work for the teacher.” LDOE has recently begun work on the alignment and quality of assessments that districts use to gain formative information about students’ progress toward standards. Districts regularly spend large sums of money on district-created assessments, and LDOE’s preliminary review of these assessments suggested that they are often poorly aligned with standards, curricula, and states’ summative statewide tests. In the words of one LDOE state official, “I have concerns about the amount of

14

classroom time that [district assessments] take up and particularly because of the quality of questions being used . . . when it doesn’t match with the summative [state test].” That official continued by noting that when district assessments are not aligned with standards and summative assessments, teachers get unclear messages about what they should be doing in the classroom. In an effort to better align formative assessments with state standards and the state summative assessment, LDOE reviews available interim assessments and provides recommendations to districts on particular assessments for some grade levels, as well as guidance to districts for selecting and developing their own assessments. To that end, LDOE has also built an online tool that teachers can use to find and use formative assessment items.16 One state official described the tool as “thousands of items that are high quality and aligned with state assessments.” She noted that teachers can search by module for two of the most highly used curriculum programs in Louisiana—Eureka Math and the LDOE-developed ELA units—and can search by text, standard, or topic. The official remarked that the tool “allows teachers to build high-quality assessments that connect to curriculum and give them better data so they can make better decisions about adjusting their instruction.” LDOE hopes

that its assessment tools will not only provide clear guidance to districts and teachers but also help districts save money and create more coherence across state and district systems.

Strategy 2: Transparent and Regular Communication About Academics Within the State Department and Across Layers of the Education System As we illustrate in Figure 8, LDOE regularly disseminates information and communicates about standards, curricula, professional development, and assessment—and the alignment of these systems—with multiple layers of school and district staff in Louisiana. LDOE staff and team leads also communicate with one another regularly.

Communication Within the State Department of Education Within LDOE, there are separate teams of people who address content, assessment, and implementation goals. Yet LDOE staff keep academics and subject-area content central to their con-

Figure 8. Aligned Systems Supporting Standards Implementation in Louisiana

Louisiana Department of Education

Communication between regional network leaders/coaches and district superintendents and staff Louisiana state districts

Communication between LDOE and educators at each layer of district systems Standards

Curricula

Assessments

Professional development

District superintendents

District assistant superintendents

School leaders

Teacher leaders

RAND RR1613-8

15

versation across personnel teams. One state official remarked that Louisiana’s state superintendent, in particular, “sees the role of academics as a part of policy and accountability.” Furthermore, LDOE teams talk together frequently in order to communicate consistent messages. One member of the LDOE staff commented, “Nothing goes out the door unless it’s connected to the structures and the other initiatives. . . . [It’s] hard, and it requires a lot of time and collaboration . . . but that’s our orientation and belief.” LDOE currently includes about 300 full-time employees, a decrease from roughly 600 employees at the start of its new administration in 2012. Such a large reduction in staff may have diminished capacity in some ways. However, state officials have noted that these reductions have created some focus and efficiency within the department. One state official commented that the reductions were “helpful to coherence because they created an environment where there just weren’t as many people out there saying whatever they wanted to say.”

Communication Across Each Layer of the Education System LDOE staff described their communication with district and school leaders and teachers as “disciplined,” “routine,” and “very predictable.” The department plans regular meetings, phone calls, and webinars with superintendents, as well as separate meetings with assistant superintendents or central office supervisors (e.g., heads of academics or curricula). Staff also have regular meetings with school principals. A state official noted that they provide “real resources” at every meeting to help district and school stakeholders with decisions they are making at a particular time of year, and district and school staff attend the meetings to find out about resources. Such resources include an annual calendar of state events and planning guides intended to help districts set priorities and make funding decisions for the coming year.17 District leaders also talk within their networks. Louisiana is divided into three regions, and each is overseen by a network leader and team. Network leaders are described as spending the majority of their week in districts, helping districts outline goals and identify areas where they need more support. Those network leaders communicate back to the state about district needs and areas of concern. Thus, networks act as another level of state communication with districts. Lastly, Louisiana has engaged more than 5,000 Teacher Leaders—approximately two teachers per school across

LDOE staff keep academics and subjectarea content central to their conversation across personnel teams. Louisiana­—who receive monthly newsletters, participate in monthly webinars, and attend quarterly collaborations to receive information and training on curricula and tools provided by the state, as well as further their professional development. Districts appoint the teachers, and the state then communicates directly with them. Some subsets of these Teacher Leaders also are engaged in reviewing instructional materials, writing newsletters, and leading trainings. State staff view Teacher Leaders as vessels to quickly communicate information to schools. One state official commented, “We realized early on that we would never be able to communicate coherence if the superintendent of each school system was going to [be] the vehicle through which we needed to filter every bit of information [to teachers]. . . . We had to go right for the teachers.” According to our brief scan of state department of education websites, some other states have teacher leader or “master teacher” programs. Idaho, Maryland, New York, and Tennessee, for example, train hundreds or even thousands of master or exemplary teachers across their states who then provide mentorship and training to their peers. Other states provide frameworks and standards for districts to support teacher leadership. However, we have not identified other state programs that recruit such a large number of teacher leaders and regularly use them in as many ways as LDOE does. Teacher Leaders are not required by LDOE to attend any trainings, and they are not paid by the state for their work, although some districts choose to provide some payment in the form of stipends or release time. Yet nearly all Teacher Leaders across the state attended the most recent annual summit, which may speak to the quality of the training they receive at the summits. The Teacher Leaders are charged with sharing what they learn at state webinars and in-person meetings and trainings with other teachers at their schools.

16

Strategy 3: Strong Support for Local Decisionmaking and Ownership of Change by Districts and Teachers LDOE emphasized to us that the tools and supports it provides are intended to help districts, schools, and teachers make informed decisions about curricula, assessments, professional development, and instruction rather than mandate the use of particular materials. This stance is also communicated in various places on its website. LDOE has thus positioned itself as the purveyor of a high-quality, aligned marketplace to guide local decisions about curricula, professional development, and assessments. Specifically, the website provides districts and teachers with a clearinghouse of information about many vendors, including reviews on the alignment among curricula, professional development, and Louisiana standards. While the department does not make decisions for local school districts, it is very clear about the materials, training, and assessments that are most aligned and faithful to state standards, and it strongly supports more-aligned curricula and professional development. One LDOE official said, “We don’t force anybody to purchase Tier I [professional development], but we only fund and endorse professional development providers that work with Tier I instruments. We will only do statewide contracts for bulk purchasing for Tier I contracts.” Another official made clear that the department is more interested in ensuring that schools are helping students learn and supporting districts and teachers than in monitoring which curricula, professional development, and formative assessments districts are using: “What I’m interested in—Are your students’ results improving [on standards-aligned summative assessments] and do your teachers feel supported in that work? If the answers are yes and yes on [assessments and] surveys we use to figure it out, then what you are doing seems to be working.” In addition to providing this clearinghouse of tools for districts, LDOE closely involves educators and “exemplar” districts in piloting and reviewing those tools. Districts and teachers are therefore able to “own” the work along with the state and claim some expertise in using these resources, and this strategy draws educators into substantive conversations with those in other districts about state tools and how they can be used. The department’s work with Teacher Leaders is a good case in point. As mentioned earlier, some Teacher Leaders review curricula, and some provide professional development to one another at the Teacher Leader annual summit and other gatherings. A state official noted, “A lot of our work has been

about ‘How do you create governance structures over reforms that are not owned by reformers and bureaucrats but are owned by real people in communities?’”

CO N C LUSI O N AN D IM PLI C ATI O NS More than 20 years ago, David Cohen (1995) asked, “What is the system in systemic reform?” He wrote that, while systemic reformers had made impressive strides in their push for new standards and aligned assessments to guide instruction, the conflicting and numerous directions of these reforms at the state and district levels have turned into “a gathering babel of reform ideas and practices.” Cohen argued that this incoherence in the system is exacerbated by weak teacher knowledge about academic subjects, professional values, and what he called “the social resources of practice,” or community resources and institutions that send signals about what teachers should know and do. Cohen concluded that systemic reforms implemented through the policy instruments available to a state—such as standards and assessments—have a poor chance of leading to large-scale change in the nature of instruction. Our findings in Louisiana counter Cohen’s statements about the inability of states to influence school policy. Results from ATP surveys indicate that, compared with other teachers nationally, Louisiana teachers use some CCSS-aligned curricula at a higher rate than other teachers, demonstrate a better understanding of their CCSS-aligned standards, and report undertaking more instructional activities that align with their standards. Our interviews with Louisiana state officials suggest that LDOE’s work to create a coherent environment for instruction is one likely reason for the differences we observed between Louisiana teachers and their peers in other states. In the words of one state official, LDOE’s goal is to create the conditions in which a teacher is able to say, “I’m motivated to achieve with my kids, and every incentive I have in my professional life leads me there. I know what it looks like, I can quantify it, and I understand how all the tools I have at my disposal play a role in getting me there.” At a time when many states have added instructional mandates on top of conflicting layers of legislation and requirements, Louisiana has worked to “clean out the closet” and point school districts and teachers all down the same pathway to instructional improvement. The state has not required teachers, schools, or districts to choose particular instructional materials, professional development, or assessments. Instead, state

17

staff have tried to “set the table” so that both administrators and educators receive regular, consistent, aligned messages and tools to support instruction. We have highlighted the following three main strategies by which Louisiana has re-envisioned this coherent environment for instruction: 1. a coherent academic strategy focused on integration, alignment, and quality among the three systems supporting standards: curricula, professional development, and assessment 2. regular and consistent communication across layers of the education system—from superintendents to teachers—to share information about tools and support collaboration 3. support for local decisionmaking and ownership of change. These three strategies may not be equally important to produce change in what teachers do. One or two of these strategies may be more effective than the others. There also could be other statewide strategies supporting teachers to think and act in ways that are different from those in other SACC states. For example, this report did not closely explore differences in what LDOE is doing to help teachers meet mathematics standards versus ELA standards. The tools that LDOE provides are somewhat different for those subjects, and the differences could be more or less helpful to teachers. Our ATP data suggest that Louisiana ELA teachers, in particular, embrace CCSS-aligned perceptions and practices more than their ELA counterparts in other SACC states, whereas we did not note as many significant differences in mathematics in regard to the CCSS-aligned approaches and practices we measured. However, our national ATP data on the implementation of state standards also suggest that math teachers, particularly at the elementary level, may be able to access more instructional resources aligned with CCSS than ELA teachers. Thus, U.S. math teachers, in general, may be doing work that is better aligned with CCSS. We will explore these and other differences among teachers in more depth through future ATP surveys.

How Can States and Districts Move Forward? Each state has its own legacy of political differences, legislation, and organizational structures. Strategies that may have worked in Louisiana to support implementation of state standards may not be effective in every state. However, LDOE strategies that we have highlighted in this report—coherence across systems, communication across layers of the educa-

If states could vet and recommend curricular and professional development tools that are high quality and aligned with their standards, districts and teachers could focus more of their energy on supporting student learning. tion system, and local ownership of educational change—are also reflected in recommendations from decades of education research. In particular, systemic reform research posits that teachers will be better able to engage in instructional reforms (such as state standards) if they are working in an environment in which all the systems supporting their instruction are giving them common and clear messages about what they should be doing in their classrooms. Alignment between standards and student assessments is a basic starting point, and—by adopting or developing new assessments­—many states are working to achieve that alignment. Even if states cannot clean out all the layers of their systems that send conflicting messages to teachers, state departments of education should consider prioritizing two additional complementary areas to standards and assessments that research suggests are key to systemic reforms: curricula and professional development. If states could vet and recommend curricular and professional development tools that are high quality and aligned with their standards, districts and teachers could focus more of their energy on supporting student learning by using those tools rather than spending time and resources searching for such tools. Other difficult work for states is redefining how they communicate with educators and within the state department of education. Years of bureaucratic practices, particular organizational structures, and cultural norms may be keeping state teams and

18

In an ideal education system, states, districts, teachers, and families are on the same page about what students should learn, and all educators have strong and aligned resources at their disposal to help students get there. departments from frequent communication with one another. This report noted that all the state teams within LDOE are in regular communication, collaborating frequently. In particular, academic and subject-area content expertise is consistently intertwined with the work of all department teams, including their accountability work, the work they do in pre-K and high school, and work for pre-service teacher education. Such a flat communication structure is likely important to accomplish authentic alignment work and may require that states do some hard work to create structures—meetings, routines, networks—that provide regular opportunities for communication about academics within and across their departments of education. In addition, states may lack routines and modes of communication across all the layers of their education systems and may rely on their regional offices to communicate down to superintendents, who, in turn, are expected to communicate down to principals, who communicate down to teachers. Our findings in LDOE suggest that states should consider communicating directly and regularly to both administrators and teachers. In particular, LDOE’s extensive communication with and supports to thousands of teacher leaders across the state has the best chance of directly supporting improvements to instruction. It makes logical sense. And, yet, state departments of education may not prioritize direct communication with the educators on the front lines with students in classrooms. The forms of communication used by LDOE also serve to create ownership of efforts by local educators. Teachers and local school leaders participate in the vetting of materials and professional development opportunities, and they have an important role in disseminating these materials, opportuni-

ties, and messages about practice to their colleagues. Such involvement of educators allows them to feel ownership and responsibility for the success of these efforts. Engaging these local educators also expands LDOE’s limited staff capacity by drawing on expertise and talent from across the state. These findings also have implications for districts. States and districts, together, play a dual role in creating a coherent environment for teaching. In states where departments of education are doing good work to define and align systems of standards, curricula, professional development, and assessments, districts can support state work by ensuring that administrators and teachers understand state expectations for teaching and learning, communicating those expectations to their teachers and school leaders, and working to align district systems with what is available at the state level. When states do not have the capacity or will to create coherent environments for instruction, districts have a more difficult job. But they must also seek to build that coherent environment in their own districts by aligning state standards and assessments closely with curricula and professional development. Additionally, when states are not providing enough clarity, support, and public resources, district and school leaders, educators, families, and education organizations must push for systemic reform and change. Stakeholders could advocate state work to integrate and align curricula, professional development, and assessments and consider how to support and extend such work. In an ideal education system, states, districts, teachers, and families are on the same page about what students should learn, and all educators have strong and aligned tools and resources at their disposal to help students get there.

19

Notes

For recent information on Louisiana student performance on AP tests, see Louisiana Department of Education, 2016b.

vey, teachers of core subjects (math, ELA, science, and social ­studies) responded at higher rates than teachers of other subjects. For the October 2015 survey, teachers from the Midwest region of the United States responded at higher rates than teachers from other regions, and teachers from the Northeast region of the United States responded at lower rates than teachers from other regions; teachers from larger schools responded at lower rates than teachers from medium-sized schools; and elementary teachers responded at higher rates than secondary teachers. No other major subgroup differences were observed or accounted for through the weighting.

For more details on these results, see Louisiana Department of Education, undated-f.

These three key markers are justified and discussed in additional depth in Opfer, Kaufman, and Thompson, 2016.

In addition, Louisiana fourth-grade students achieved the secondhighest growth on the NAEP mathematics test from 2013 to 2015. However, the difference was not significant for math, although it was for reading. For more information about Louisiana NAEP scores, see Nation’s Report Card, 2015.

EdReports used a “gateway system” for its reviews of 26 commonly used, published textbook series for K–8 students. If materials did not meet expectations for the first gateway (focus and coherence), they were not reviewed for rigor and mathematical practices. Thus, some materials were not assessed in all areas related to CCSS and may be aligned in some areas that were not examined. For more information on the EdReports methodology, see EdReports, undated-c.

According to 2010–2014 American Community Survey five-year profiles from the U.S. Census Bureau, 19.6 percent of people in Louisiana are below the poverty level, which is the third-highest percentage in the country (only Mississippi’s and New Mexico’s percentages are higher). For data tables and tools for the American Community Survey, see U.S. Census Bureau, undated. 1

2

3

4

For June 2015 survey percentages in this report, we consider teachers in other SACC states to be any teachers in other states beyond ­L ouisiana with the exception of Alaska, Indiana, Minnesota, Nebraska, Oklahoma, Texas, and Virginia. For the October 2015 survey percentages, we consider teachers in other SACC states to be any teachers in other states beyond Louisiana with the exception of those same states and South Carolina. Given that Minnesota has adopted CCSS for ELA but not mathematics, we include Minnesota as an SACC state in any analysis throughout this report referencing ELA or ELA standards. These states were excluded based on documented state adoption of CCSS (Common Core State Standards Initiative, 2016d). 5

For any items where we compared Louisiana teachers and other SACC teachers on a range of related variables, we used the BenjaminiHochberg procedure to adjust for multiple comparisons, applying a false discovery rate of 0.10. For more information, see Benjamini and Hochberg, 1995.

6

Response rates for large, national surveys have been in decline, and this tendency accelerated after the emergence of web questionnaires. A meta-study of 68 surveys in 49 studies by Cook, Heath, and Thompson (2000) found an average 40-percent response rate among national survey studies. Similarly, Nulty (2008) found that responses to web-based surveys ranged between 20 and 47 percent.

7

Weights were based on a model for nonresponse that incorporates such characteristics as teacher subject, school level, region size, and rate of free or reduced-price lunch eligibility. For the June 2015 sur-

8

9

10

Similar to its approach for mathematics, EdReports used a gateway system for its reviews of seven commonly used, published textbook series for students in grades 3–8. If materials did not meet expectations for the first gateway (text quality and complexity, plus alignment to standards components), they were not reviewed for the second gateway (building knowledge with texts, vocabulary, and tasks). 11

For curriculum reviews, see Louisiana Department of Education, undated-a.

12

For all curricular materials, see Louisiana Department of ­Education, undated-e.

13

For information on pricing, see Louisiana Department of ­Education, undated-d. 14

For the professional development vendor catalogue, see Louisiana Department of Education, 2016a.

15

At the time of publication, this formative assessment tool was unavailable because it was being transitioned to a new delivery platform, but it was scheduled to be available in September 2016. For more information, see Louisiana Department of Education, undated-c. 16

For the district support resources, see Louisiana Department of Education, undated-b.

17

20

References Bay-Williams, J., Common Core Math in the K-8 Classroom: Results from a National Teacher Survey, Washington, D.C.: Thomas B. Fordham Institute, June 2016. As of September 14, 2016: http://edexcellence.net/publications/common-core-math-in-the-k-8-classroom-results-from-a-national-teacher-survey Benjamini, Y., and Y. Hochberg, “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing,” Journal of the Royal Statistical Society, Series B, No. 57, 1995, pp. 289–300. Brown, S., and L. Kappes, Implementing the Common Core State Standards: A Primer on “Close Reading of Text,” Washington, D.C.: Aspen Institute, October 2012. As of September 3, 2016: http://www.aspendrl.org/portal/browse/DocumentDetail?documentId=1396&download Coburn, C. E., “Collective Sensemaking About Reading: How Teachers Mediate Reading Policy in Their Professional Communities,” Educational Evaluation and Policy Analysis, Vol. 23, No. 2, 2001, pp. 145–170. ———, “Beyond Decoupling: Rethinking the Relationship Between the Institutional Environment and the Classroom,” Sociology of Education, Vol. 77, No. 3, 2004, pp. 211–244. Coburn, C. E., H. C. Hill, and J. P. Spillane, “Alignment and Accountability in Policy Design and Implementation: The Common Core State Standards and Implementation Research,” Educational Researcher, Vol. 45, No. 4, 2016, pp. 243–251. Coburn, C. E., P. D. Pearson, and S. L. Woulfin, “Reading Policy in the Era of Accountability,” in M. L. Kamil, P. D. Pearson, E. B. Moje, and P. P. Afflerbach, eds., Handbook of Reading Research, Vol. 4, New York: Routledge, 2011, pp. 629–653. Cohen, D. K., “What Is the System in Systemic Reform?” Educational Researcher, Vol. 24, No. 9, 1995, pp. 11–31. Cohen, D. K., and D. L. Ball, “Policy and Practice: An Overview,” Educational Evaluation and Policy Analysis, Vol. 12, No. 3, Fall 1990, pp. 233–239. Coleman, D., and S. Pimentel, Revised Publishers’ Criteria for the Common Core State Standards in English Language Arts and Literacy, Grades 3–12, Washington, D.C.: Common Core State Standards Initiative, April 12, 2012a. As of September 14, 2016: http://www.corestandards.org/wp-content/uploads/Publishers_Criteria_for_Literacy_for_Grades_3-12.pdf ———, Revised Publishers’ Criteria for the Common Core State Standards in English Language Arts and Literacy, Grades K–2, Washington, D.C.: Common Core State Standards Initiative, April 12, 2012b. As of September 14, 2016: http://www.corestandards.org/wp-content/uploads/Publishers_Criteria_for_Literacy_for_Grades_K-2.pdf Common Core State Standards Initiative, High School Publishers’ Criteria for the Common Core State Standards for Mathematics, Washington, D.C., April 9, 2013a. As of September 14, 2016: http://www.corestandards.org/wp-content/uploads/Math_Publishers_Criteria_HS_Spring_2013_FINAL1.pdf ———, K–8 Publishers’ Criteria for the Common Core State Standards for Mathematics, Washington, D.C., April 9, 2013b. As of September 14, 2016: http://www.corestandards.org/wp-content/uploads/Math_Publishers_Criteria_K-8_Spring_2013_FINAL1.pdf ———, “English Language Arts Standards, Anchor Standards, College and Career Readiness Anchor Standards for Language,” web page, 2016a. As of June 26, 2016: http://www.corestandards.org/ELA-Literacy/CCRA/L/ ———, “English Language Arts Standards, Anchor Standards, College and Career Readiness Anchor Standards for Reading,” web page, 2016b. As of June 26, 2016: http://www.corestandards.org/ELA-Literacy/CCRA/R/ ———, “Standards for Mathematical Practice,” web page, 2016c. As of September 13, 2016: http://www.corestandards.org/Math/Practice/ ———, “Standards in Your State,” web page, 2016d. As of June 26, 2016: http://www.corestandards.org/standards-in-your-state/ Cook, C., F. Heath, and R. L. Thompson, “A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys,” Educational and Psychological Measurement, Vol. 60, No. 6, 2000, pp. 821–836.

21

EdReports, “ELA Reports,” web page, undated-a. As of September 3, 2016: http://www.edreports.org/ela/reports/index.html#?f=&o=0 ———, “Math Reports,” web page, undated-b. As of August 29, 2016: http://www.edreports.org/math/reports/index.html#?f=&o=0 ———, “Our Approach,” web page, undated-c. As of September 13, 2016: http://www.edreports.org/about/our-approach/index.html Fuhrman, S. H., and R. F. Elmore, “Understanding Local Control in the Wake of State Education Reform,” Educational Evaluation and Policy Analysis, Vol. 12, No. 1, 1990, pp. 82–96. Gewertz, C. “State Solidarity Erodes on Common-Core Tests,” Education Week, March 22, 2016. As of June 27, 2016: http://www.edweek.org/ew/articles/2016/03/23/state-solidarity-erodes-on-common-core-tests.html?r=1591708071&preview=1 Kane, T. J., A. M. Owens, W. H. Marinell, D. R. C. Thal, and D. O. Staiger, Teaching Higher: Educators’ Perspectives on Common Core Implementation, Cambridge, Mass.: Harvard University Center for Education Policy Research, 2015. As of September 3, 2016: http://cepr.harvard.edu/teaching-higher Louisiana Department of Education, “Curricular Resources Annotated Reviews,” web page, Baton Rouge, La., undated-a. As of September 9, 2016: http://www.louisianabelieves.com/academics/online-instructional-materials-reviews/curricular-resources-annotated-reviews ———, “District Support Toolbox,” web page, Baton Rouge, La., undated-b. As of September 9, 2016: http://www.louisianabelieves.com/resources/classroom-support/district-support-toolbox ———, “EAGLE 2.0,” online assessment tool, Baton Rouge, La., undated-c. As of September 9, 2016: http://www.louisianabelieves.com/assessment/eagle ———, “Online Instructional Materials,” web page, Baton Rouge, La., undated-d. As of September 9, 2016: http://www.louisianabelieves.com/resources/library/curricular-resources ———, “Planning Resources,” web page, Baton Rouge, La., undated-e. As of September 9, 2016: http://www.louisianabelieves.com/resources/library/Planning-Resources ———, “Statewide Results,” news release, Baton Rouge, La., undated-f. As of September 9, 2016: http://www.louisianabelieves.com/resources/library/Statewide-Results ———, 2016 Vendor PD Course Catalog, Baton Rouge, La., 2016a. As of September 9, 2016: http://www.louisianabelieves.com/docs/default-source/teacher-toolbox-resources/2016-vendor-pd---course-catalog.pdf ———, “Louisiana Students Earn Record Number of Advanced Placement College Credits,” Baton Rouge, La., August 8, 2016b. As of September 9, 2016: http://www.louisianabelieves.com/newsroom/news-releases/2016/08/08/ louisiana-students-earn-record-number-of-advanced-placement-college-credits Mayer, D. P., “Measuring Instructional Practice: Can Policymakers Trust Survey Data?” Educational Evaluation and Policy Analysis, Vol. 21, No. 1, 1999, pp. 29–45. Nation’s Report Card, “NAEP: 2015 Mathematics & Reading,” web page, 2015. As of September 9, 2016: http://www.nationsreportcard.gov/reading_math_2015/#?grade=4 Nulty, D. D., “The Adequacy of Response Rates to Online and Paper Surveys: What Can Be Done?” Assessment and Evaluation in Higher Education, Vol. 33, No. 3, 2008, pp. 301–314. O’Day, J. A., and M. S. Smith, “Systemic Reform and Educational Opportunity,” in S. Fuhrman, ed., Designing Coherent Education Policy: Improving the System, San Francisco, Calif.: Jossey-Bass Publishers, 1993, pp. 250–312. Opfer, V. D., J. H. Kaufman, and L. E. Thompson, Implementation of K–12 State Standards for Mathematics and English Language Arts and Literacy, Santa Monica, Calif.: RAND Corporation, RR-1529-HCT, 2016. As of September 3, 2016: http://www.rand.org/pubs/research_reports/RR1529.html

22

Polikoff, M.S., “How Well Aligned Are Textbooks to the Common Core Standards in Mathematics?” American Educational Research Journal, 2015, pp. 1–27. Porter, A., J. McMaken, J. Hwang, and R. Yang, “Common Core Standards: The New U.S. Intended Curriculum,” Educational Researcher, Vol. 40, No. 3, 2011, pp. 103–116. Schmidt, W. H., C. C. McKnight, and S. A. Raizen, A Splintered Vision: An Investigation of U.S. Science and Mathematics Education, Boston: Kluwer Academic Press, 1997. Shanahan, T., “Letting the Text Take Center Stage: How the Common Core State Standards Will Transform English Language Arts Instruction,” American Educator, Vol. 37, No. 3, 2013. pp. 4–11. Spillane, J. P., and J. S. Zeuli, “Reform and Teaching: Exploring Patterns of Practice in the Context of National and State Mathematics Reforms,” Educational Evaluation and Policy Analysis, Vol. 21, No. 1, 1999, pp. 1–27. Student Achievement Partners, “The Common Core Shifts at a Glance,” web page, February 2016. As of June 26, 2016: http://achievethecore.org/page/277/the-common-core-shifts-at-a-glance U.S. Census Bureau, “American Community Survey: Data Tables & Tools,” web page, undated. As of September 9, 2016: http://www.census.gov/acs/www/data/data-tables-and-tools/index.php Weiss, I. R., J. D. Pasley, P. S. Smith, E. R. Banilower, and D. J. Heck, Looking Inside the Classroom: A Study of K–12 Mathematics and Science Education in the United States, Chapel Hill, N.C.: Horizon Research, 2003.

23

About the Authors Julia H. Kaufman’s primary areas of interest are the measurement of teacher instruction and how policies and programs can best ­support high-quality instruction and students’ academic and socioemotional growth. Prior to coming to RAND, Kaufman’s research focused on the main factors that support teachers’ high-quality implementation of inquiry-based mathematics curricula and the extent to which survey measures can accurately capture teachers’ instruction. She has taught in secondary schools, as well as adult basic education and higher education settings. She holds a Ph.D. in International Education from New York University and a ­Master’s in Teaching from the University of Pittsburgh. Lindsey E. Thompson is a research assistant who works in the field of K–12 education policy. She is particularly interested in school reform, community partnerships, equity, curriculum, teacher preparation, and education technology. Before coming to RAND, Thompson taught math in Pittsburgh, Pennsylvania, and Mombasa, Kenya, and she holds a B.A. in mathematics and education studies from Bowdoin College. V. Darleen Opfer is director of RAND Education and holds the Distinguished Chair in Education Policy at the RAND Corporation. Before joining RAND in April 2011, Opfer served on the faculty of education at the University of Cambridge, England, where she was director of research and senior lecturer in research methods and school improvement. Opfer holds a B.A. in education from Stetson University, as well as an M.Ed. in behavior disorders and a Ph.D. in education policy studies, both from the University of Virginia.

About This Report In this report, we use data from the American Teacher Panel to provide evidence that teachers in Louisiana are thinking about and implementing their state standards in ways that are significantly different from teachers in other states and examine Louisiana Department of Education strategies that might be contributing to these results. We would like to thank the Louisiana State Department of Education staff who generously took time out of their busy schedules to talk with us about their work. We would also like to thank Jessica Hwang, Scott Naftel, and Joseph Pane for their assistance with data analyses, as well as the American Teacher Panel research team, including Brian Stecher, Michael Robbins, and Susannah Faxon-Mills. Thanks also to Scott Norton and Heather Schwartz, who reviewed this report and provided ideas and guidance that have helped us articulate the key findings and their implications. In addition, we would like to thank the Helmsley Charitable Trust for funding the study that made this work possible. This research was conducted by RAND Education, a unit of the RAND Corporation, and funded by the Helmsley ­Charitable Trust. For more about RAND Education, visit www.rand.org/education. For more about the RAND American Teacher Panel and how you could take advantage of this resource, see www.rand.org/education/projects/atp-aslp.

Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of our research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html. For more information on this publication, visit www.rand.org/t/rr1613.

© Copyright 2016 RAND Corporation

www.rand.org The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. C O R P O R AT I O N

RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. registered trademark.

R® is a RR-1613-HCT

Loading...

Creating a Coherent System to Support - RAND Corporation

C O R P O R AT I O N Creating a Coherent System to Support Instruction Aligned with State Standards Promising Practices of the Louisiana Department o...

368KB Sizes 3 Downloads 11 Views

Recommend Documents

Banking System Reform in China - RAND Corporation
This PDF document was made available from www.rand.org as a public service of the RAND .... Abolishment of the Credit Pl

Support RAND
Aug 3, 2009 - of reform that had not been raised in discussions in Massachusetts . Using these materials, we developed a

Support RAND
This PDF document was made available from www.rand.org as a public service of the RAND ..... to filling the prescription

Predictive Policing - RAND Corporation
mote software packages as end-to-end solutions for predictive policing, humans ...... The test used. 39 The details are

Case Studies - RAND Corporation
which have a strong interest in achieving better health at lower cost, the answer may be “no.” .... 9 Lloyd-Jones DM

nicholas burger - RAND Corporation
Associate Director, Department of Economics, Sociology, and Statistics, RAND Corporation, Arlington, VA,. 2012–2016 ..

A Patrol Car Allocation Model - RAND Corporation
general-purpose program that we call PCAM (Patrol Car Allocation Model). This model incorporates, by user option, ... of

To How Many Customers Should a Drug Dealer - RAND Corporation
May 5, 1995 - To How Many Customers Should a Drug. Dealer Sell? Ionathan P. Caulkins. DRll-1079-FF. May 1995. Prepared f

The Evolving Terrorist Threat to Southeast Asia: A - RAND Corporation
This PDF document was made available from www.rand.org as ...... Front Pembela Islam. GAM. Gerakan Aceh Merdeka. GMIP. G

administration and management - RAND Corporation
the main areas is the administration and management of depart- ments and their deployment of their human and technical a