Teaching the Movement 2014
Appendix B: Methodology
The methods we employed to grade the states in 2014 differ in two major ways from the 2011 report. First, we moved away from a “checklist” approach to the content standards and considered additional aspects of states’ major documents. Second, we also considered states’ supporting resources. As Table 2 shows, major documents counted for 60 percent of a state’s score, with supporting resources counting for 40 percent.
The Major Documents
Our examination of the major documents rated states in four areas: content, sequence, depth, and connections. The score in these areas made up 60 percent of a state’s final grade.
Content
Our content analysis proceeded in two stages. First, we assigned scores to each state by comparing its standards and frameworks to the standardized rubric of content expectations developed for the 2011 report. Next, we looked at aggregate levels of detail required overall in major documents across all states. The rubric allowed us to compare states to each other fairly, while the secondary analysis gave us a sense of both the breadth and the “middle” of state expectations regarding the civil rights movement.
Events and major figures in the civil rights movement were predominantly Southern, so it was important for this study to create a fair rubric that would not benefit states based solely on their geography. Most states, along with the District of Columbia, require a class or unit on state history. Students in Southern states might, in theory, be required to learn more about the civil rights movement than students in Western states. This study controls for that imbalance as much as possible.
The rubric in Table 18 was developed through closely reading a dozen of the most widely used American history textbooks over a variety of grade levels and in consultation with historians in the field. It evaluates states based on their required coverage of essential content as well as their integration of the civil rights movement into a larger instructional approach. We tried, with this rubric, to set out an approachable span of core knowledge that a competent citizen needs to gain a reasonably full understanding of the civil rights movement. It is neither complete nor exhaustive; rather, it represents an attempt to synthesize essential information while appreciating the time constraints faced by teachers.*
Sequence
We looked at sequencing at three levels. First, we wanted to see if states talked about the movement as something that expanded beyond the common “Memphis to Montgomery” timeline. The civil rights movement is both wider and longer than textbooks and standards often lead us to believe. Many state documents reflected this reach. Some states reached backward, linking the movement to the Great Migration and pioneers like W. E. B. Du Bois. Other states looked forward, discussing the “long hot summers” of the 1970s.
Second, we examined sequencing in the context of the arc of American history. We were interested to see if, and how, states made an effort to make sense of the civil rights movement in its broader historical context. States did this differently. Some states depicted the movement as part of a larger struggle for minority enfranchisement. Others framed the movement in terms of the tension between state and federal rights. Some made no effort here, listing the movement as one of many events occurring after World War II.
Finally, we looked at sequencing across grade levels to see when states introduced the civil rights movement and how the ideas progressed as students matured. States were assigned a score of 0 to 4 in this area. A score of 4 meant that a state was exceptional in one or more areas in addition to covering the others. Sequencing accounted for 15 percent of a state’s final grade.
Depth
For depth, the report assessed the way states’ major documents discussed three aspects of the civil rights movement. First, we wanted to know if the causes of the civil rights movement were clearly presented. Second, we were curious if the major documents were clear about the nature of resistance to the movement. This allowed us to include specific state reactions as well as other concepts like the Dixiecrats and Massive Resistance not captured in the content score. Finally, we looked at whether documents encouraged students to learn about conflicts within the civil rights movement. This meant going beyond including figures like Stokely Carmichael or Malcolm X in a list of “important figures.” States were assigned a score of 0 to 4 in this area. A score of 4 meant that a state was exceptional in one or more areas in addition to covering the others. Depth accounted for 15 percent of a state’s final grade.
Connections
In the 2011 report, 15 percent of a state’s grade was based on the way its standards connected the civil rights movement to the broader context of students’ lived experience. This report has increased the emphasis. We graded connections made in the major documents in three ways. First, we looked at whether states connected the civil rights movement to present-day events. Second, we evaluated the connections made to civic education. Some states explicitly discuss the lessons of active citizenship that can be drawn from the civil rights movement; others embed discussion of key movement events and ideas in their civics and government standards. Others made no connections, leaving it up to teachers to connect the dots for their students. Finally, we looked at whether states made connections between the civil rights movement and other movements in the 21st century. For this area, it was not enough to simply mention the civil rights movement in a list of other struggles. Outstanding states made connections explicit or encouraged students to compare tactics, ideas and strategies. States were assigned a score of 0 to 4 in this area. A score of 4 meant that a state was exceptional in one or more areas in addition to covering the others. Connections accounted for 15 percent of a state’s final grade.
Supporting Resources
Our examination of supporting resources rated states in three areas. We looked at whether these resources were comprehensive, if they promoted historical thinking and how they were presented. The score in these areas accounted for 40 percent of a state’s final grade.
Comprehensive
State resources that were comprehensive crossed grade levels to reach students at different ages. They covered many aspects of the movement, reaching beyond the aspects covered in the major documents. Finally, comprehensive resources included several lesson and unit plans. States were assigned a score of 0 to 4 in this area. States that received a score of 4 had resources that reached beyond the norm, making the civil rights movement relevant to the state’s population. This item accounted for 13 percent of a state’s final grade.
Promotion of Historical Thinking
As in all study of history, the use of original historical documents is essential to develop a rich understanding of an era. When we examined state resources with this lens, we were looking at three dimensions. First, at the most basic level, we considered whether state resources promoted the use of original historical documents. Second, looking beyond promotion, we considered whether the resources were constructed to help teachers use original historical documents. Finally, we wanted to see whether the resources went beyond the traditional narrative about the civil rights movement. States were assigned a score of 0 to 4 in this area. States that scored a 4 had resources that promoted the thoughtful use of historical documents beyond the basics (i.e., “I Have a Dream” and “Letter From Birmingham Jail”). This item accounted for 13 percent of a state’s final grade.
Access and Presentation
It was not enough for a state to have outstanding resources. Teachers are pressed for time to find materials to add value to their instruction, so resources must be easy to browse and use. We assessed state resources in three ways here. First, materials should be easy to access online. Second, they should be clearly organized by grade and topic. Third, we wanted to see if materials were presented in a way that makes them easy to use. States were assigned a score of 0 to 4 in this area. States that scored a 4 had materials that were consistent in quality, employing best practices in lesson planning and unit design. This item accounted for 13 percent of a state’s final grade.
About the Content Rubric
The greatest focus of this rubric is on specific content that students should know. We divided this content into six categories: leaders, groups, events, history (causes), opposition and tactics. States were assigned a score of 0 to 4 based on their content score, where 80 percent earned a score of 4, 60 percent a score of 3, 40 percent a score of 2, 20 percent a score of 1 and less than 20 percent a zero. Considered as part of our weighted rubric, essential content contributed to 15 percent of a state’s overall score.
Of necessity, the rubric is incomplete. In particular, it reflects a regional version of the civil rights movement which, while consistent with textbook and state versions of events, is increasingly at odds with more nuanced portrayals of the movement in modern scholarship. There is no dispute among historians that key activists and events happened outside the South. The rubric attempts to capture some of this by allowing states the freedom to name their own influential leaders, but the events category remains a hybrid of national and Southern tipping points.
Using the rubric, state standards and frameworks were read and assigned a value of 1 for each specific component included. For example, in the category dealing with movement leaders, states were given a 1 if they mentioned that students should learn about Martin Luther King Jr. and another 1 if they mentioned Rosa Parks. Thus a full list of movement leaders required by the states, collectively, was developed. States were coded a 0 or 1 for each leader depending on whether they required study of that leader. States received a score in each of the rubric’s categories. For example, in the leaders category, states were assigned a score based on the percentage of eight leaders they identified to be studied. If states identified six of the recommended eight, the raw score for leaders was 75 percent. The other categories (groups, events, history, opposition, tactics) were scored similarly, according to the items and accompanying weights in the rubric. Full coding for each state is found in Appendix A.
