Analysis of a PARCC ELA Assessment Task Through an ELL Lens (Part 1)

As you may have heard, the Common Core assessment consortium PARCC just released new sample items last week in English language arts (ELA) in grades 3-11 and in mathematics in grades 3-8 and high school. Since teachers will need to plan their CCSS-based instruction around the CCSS standards as well as the CCSS-based assessments, I thought it would be helpful in part 1 of this post to dissect a sample test task from an ELL point of view in order to take a closer look at what the item might mean for ELLs and those who teach them. Next week, I'll be back with part 2 and some suggestions for the classroom based on my analysis of this task as well as additional sample tasks from other grade levels.

What Students Are Asked To Do

For this exercise, I chose a third-grade ELA research simulation task. (Note: This task is no longer available on the PARCC website but reflects the reading level and type of questions for third grade.) In this sample task, there are seven items for students – four “evidence-based selected response” items, two “technology-enhanced constructed response” items, and one “prose constructed response” item.

Passage #1

In order to take part in the computer based assessment task, students first read a “purpose setting statement” for their task and then read the first informational passage. In this case, the first text they read is an excerpt from the children's book Eliza’s Cherry Trees: Japan’s Gift to America by Andrea Zimmerman.

Passage #2

After they answer the four selected response items and two constructed response items, students then read a second passage, which is a text about George Washington Carver called The Peanut Man. They are then asked to use the computer to write an article that synthesizes aspects of both texts. I’ll walk you through the sample items below, tell you how they are scored, and then provide you with some food for thought on what an assessment task such as this might mean for ELLs.

Sample Item 1

Part A Question:  The article includes these details about Eliza's life:

• She wrote newspaper articles to tell others about what she saw in Alaska to inform those who had not been there. (paragraph  1)
• She wrote the first guidebook about Alaska. (paragraph 1)
• She was the first woman to work at the National Geographic Society, where she wrote many articles and books. (paragraph 11)

What do these details help show about Eliza?

1. They show that she shared the benefits of her experiences with others.*
2. They show she had many important jobs during her lifetime, but becoming a photographer was one of her proudest moments.
3. They show that her earlier travels were more exciting than the work she did later in her life.
4. They show that she had a careful plan for everything she did in her life.

*Indicates the correct answer

Part B Question: Ideas from paragraphs 1 and 11 were used to help you learn about Eliza. Click on two other paragraphs that include additional support for the answer in Part A. There are more than two paragraphs that include additional support, but you need to only choose two.

Scoring for Item 1

2 points are awarded when the student correctly chooses the answer to Part A (A) and two correct paragraphs in Part B (any two of the following paragraphs: 5, 7, 8, 9, 12, 13,14).
1 point is awarded when the student correctly chooses the answer to Part A and 1 correct paragraph in Part B.

Sample Item 2

Part A Question: Which statement best describes how the events in paragraphs 13 through 15 are related to each other?

1. They explain how Washington, D.C., would change if cherry trees were planted around the city.
2. They show that Eliza found a new way to get cherry trees planted in Washington, D.C.*
3. They compare the ways Eliza and Mrs. Taft tried to add beauty to Washington, D.C.
4. They describe how Mr. Takamine gave Eliza the idea to bring cherry trees to Washington, D.C.

Part B Question: Which sentence from the article best supports the answer in Part A?

1. "When they bloomed, the trees became clouds of pink blossoms."
2. "She kept trying for more than twenty years!"
3. "She wrote a letter to the president's wife, Mrs. Taft."*
4. quot;With the help of Mr. Takamine, a generous Japanese scientist, they had the trees sent from Japan."

Scoring for Item 2

2 points are awarded when the student correctly chooses the answer to Part A (B) and the answer to Part B (C).
1 point is awarded when the student correctly chooses the answer to Part A (B).

Sample Item 3

Question:

You have read two texts about famous people in American history who solved a problem by working to make a change.
Write an article for your school newspaper describing how Eliza and Carver faced challenges to change something in America.

• In your article, be sure to describe in detail why some solutions they tried worked and others did not work.
• Tell how the challenges each one faced were the same and how they were different.

Scoring for Item 3

Grade 3 scoring rubric for prose constructed response items

My ELL Impressions of the Texts

The shifts in the CCSS for ELA/Literacy call for students to closely read informational text and cite evidence from those texts. This sample item made those demands very clear to me in terms of how students will be expected to demonstrate their skills with these aspects of the standards. Even though the PARCC Accommodations Manual allows for such accommodations as English/native language word-to-word dictionaries for ELLs, accommodations alone can’t address the full scope of ELLs’ challenges with assessments of content given in English.

The two assessment texts’ academic language (including complex sentence structures, precise vocabulary, cultural references, and deliberate use of different types of punctuation) present multiple challenges to ELLs in accessing these informational texts. For example, within one paragraph in the first text I noted some metaphors (“When they blossomed, they became clouds of pink blossoms”) and similes (“As the petals drifted down, it was like pink snowfall.”)

I also found the Japanese word sakura ("cherry blossom") used and italicized in the same paragraph. English language learners will have the extra challenge of figuring out the meaning of academic English and also need to make meaning from a Japanese word – if they recognize it’s a Japanese word. I also found some polysemous words, or words having more than one meaning, in the texts. For example, the meaning of the word “society” in National Geographic Society could be confusing for ELLs (as well as some native English speakers, I imagine).

In addition to the academic language, there is also a significant amount of assumed background knowledge and familiarity with cultural references necessary for students to be able to make enough meaning from these texts to be successful on the assessment. For example, to fully comprehend the cherry trees text, readers would benefit from knowing something about cherry trees; Alaska; Washington, DC’s landscape; and Japan. Some cultural references made in the text set include the National Geographic Society, the Smithsonian Institution, the Tuskegee Institute, and the United Peanut Association of America. Knowledge of the figures highlighted in the texts themselves, such as George Washington Carver and President Taft, would also help students comprehend the text.

The research simulation sample task is just one type of task among other tasks students must take part in on the sample ELA assessment. (The Grade 3 sample also contains literary analysis tasks and narrative tasks.) I didn't see how much time was suggested for students to complete the research simulation task. Even though ELLs are allowed extra time to complete the assessment, I wonder about them having the stamina to tackle all parts of this third grade test.

My ELL Impressions of the Writing Rubric

In the final part of this task, students are asked to synthesize information from both texts to write an article for their school newspaper. For this test item, students have to describe and compare solutions found across both texts and compare the challenges between Eliza and Carver. Students’ writing is graded on three criteria: Reading (0-3 points on comprehension of key ideas and details), Writing (0-3 points on written expression), and Writing (0-4 points on knowledge of language and conventions). Any student who does not respond in English is automatically given a score of 0. I didn't see any information about how the three domains are weighted (if at all) and how a final score is obtained.

To score a 3 in Reading, a student response must demonstrate accurate and full comprehension of the central ideas expressed in the text(s) and reference the text explicitly. For the reasons I cited above, it will be challenging for many ELLs to fully comprehend the informational texts used for this task.

To score a 3 in Writing (written expression), students’ writing must be largely appropriate to the task and purpose, among other criteria such as using reasoning and text-based evidence. In this case, I’m left wondering how familiar ELLs at different levels of English language proficiency will be with the concept/genre of a school newspaper in general and the type of discourse required for them to write an article for a newspaper.

To score a 4 in Writing (knowledge of language and conventions), the response must demonstrate a command of the conventions of standard English consistent with edited writing. There may be a few distracting errors in grammar and usage, but the meaning of the writing must be clear. I am concerned about how ELLs will be scored on this dimension of the rubric. By the very nature of language acquisition, ELLs at lower levels of English language proficiency will produce writing that contains errors in grammar and usage.

Final Thoughts

The language used in the task’s annotations indicates that the current rubric is general. After standard setting has occurred and authentic student writing samples have been collected through tryouts or field tests, “anchor papers” will be chosen to anchor each score point on the rubric. At that point, the generic rubric will be tailored to create a rubric that is specific to the prose constructed response item. Educators of ELLs will need to advocate that ELLs at different levels of English language proficiency be included in the field tests and that their writing samples be used as anchor papers. If ELLs are included in the process, educators are more likely to be aware of what ELLs’ writing looks like and how it may differ from that of native English speakers.

Your Ideas

What questions do you have after reading through this sample items?  How does this information impact how you'll approach your instruction under the Common Core?  We'll continue the conversation next week!