Computer-Based Common Core Testing: Considerations and Supports for ELLs

As many states begin to fully implement computer-based tests to measure students’ progress toward reaching the Common Core State Standards, it is an opportune time to take a closer look at how computer-based assessments might affect ELLs in particular. You may recall some previous blog posts that explored the content of CCSS assessments for ELLs. Two of these blog posts included Assessment of ELLs on the CCSS and the more in-depth Analysis of a PARCC item two-part blog post.

This blog post focuses on a different angle of the assessment debate, which does not directly involve the content of ELL tests but rather considerations that are important for schools and districts to address in planning their assessments. It first examines various initiatives taking place across the nation in terms of CCSS assessments for all students and takes a peek at field test results for ELLs. It then provides an overview of accommodations on CCSS content assessments and of English language proficiency assessments for ELLs. Next, the post explores which aspects of computer-based assessment might prove to be especially challenging for ELLs and ends with some resources to support ELLs’ success in computer-based testing.

National K-12 Testing Landscape

All but seven states have adopted the Common Core State Standards in English Language Arts and Mathematics, but those states have approached their assessments of students’ mastery of these standards in different ways. At last count, 18 states across the U.S. are members of the Smarter Balanced Assessment Consortium, and ten states plus DC are members of Partnership for Assessment of Readiness for College and Careers (PARCC). State membership in these two assessment consortia has shrunk since the consortia were originally formed. There are currently 21 states that are not members of these consortia (including those states that did not adopt the CCSS). Massachusetts, a Common Core adopter, remained undecided, as districts can either use PARCC or the current MCAS this year. (Take a look at a current map of the states’ assessment affiliations here.)

Field Test Results for ELLs

As educators of ELLs, we already know that ELLs experience unique challenges demonstrating what they know and can do on large-scale summative assessments that have not been created with their needs in mind. The field test results from CCSS consortia exams quantify the difficulties ELLs face on such assessments. For example, Smarter Balanced has released its field test results, disaggregated by demographic group. Smarter Balanced parses out its results from a low of Level 1 to a high of Level 4, with a Level 3 being a passing score. On Smarter Balanced Grade 6 English language arts, 4.6% of ELLs scored a level 3 or above as compared with 40.5% of all students (pg. 9).

In sixth grade mathematics, 5.2% of ELLs scored at a level 3 or higher, compared to 32.9% of all students (pg. 16).

The gap between ELLs and non-ELLs remains to be seen when the PARCC and Smarter Balanced tests are fully operational this year. While it’s not unexpected that new tests will produce a drop in student scores, some state legislatures such as New Jersey’s are reacting to strong parental and teacher pushback on the use of test results for students as well as teachers. The New Jersey Assembly just approved a bill that would prevent the results of the PARCC test from being used to measure student achievement or as part of teacher evaluation for three years.

Beyond the major issue of ELLs not being able to fully demonstrate their knowledge of content on the assessments due to their developing English proficiency, there are additional challenges to assessing ELLs’ grasp of the CCSS as assessments become fully operational this spring, which follow below.

ELL Assessment Considerations

Timing and Length of CCSS Tests in General

Two important areas to be considered for all students with the introduction of the new assessments are timing and length of the new tests. In terms of timing, the content testing “window” also seems to be opening early this year. For example, the PARCC test window opened beginning February 16, and Smarter Balanced assessments can be administered starting March 10. States decide their own testing window for each these assessments. With the snow days that schools nationwide have already taken this winter (and are still facing), the amount of material covered prior to the assessment window has become a concern for many districts that affects all students.

The length of CCSS tests is also a consideration for districts. Each consortium has included performance tasks that require more time spent on complex, multi-step problems in which students often need to justify their answers or draw from several sources to build an argument. These items come in addition to typical multiple choice and shorter-answer questions. PARCC estimates that most students can complete their tests in 6.5 – 7.5 hours, but they advise schools to schedule nearly 10 hours for elementary students and more than 11 hours for high school students. Smarter Balanced projects 7.5 to 8 hours per test per student.

Timing and Length of CCSS Tests for ELLs

The time it takes ELLs to complete CCSS content assessments may be considerably longer than for non-ELLs. In addition to ELLs taking content assessments when they are in certain grade levels, they also need to take part in English language proficiency assessments no matter which grade they are in. These assessments are designed to measure ELLs’ acquisition of the language necessary to support their access to content.

As a result, districts need to coordinate the double load of assessments that many ELLs need to take that fall between two testing windows. The general WIDA ACCESS for ELLs testing window is currently from early December to early June. To ensure that ELLs are given their English language proficiency assessments, school testing coordinators must find building space in which to hold the test as well as train test administrators to individually administer and score the speaking portion of the WIDA ACCESS for ELLs assessments. The computer-based ELPA21 assessment will become operational in 2015-16, so their testing window is not yet available to the public.

Accommodations for ELLs

An additional consideration that teachers and administrators need to be aware of is the range of different types of accommodations available to ELLs on the content assessments. The two testing consortia differ in their approach to accommodations for ELLs.

Our blog post on PARCC accommodations provides an overall sense of what PARCC allows for ELLs. The accommodations allowed for ELLs on PARCC include extended time, general administration directions clarified in student’s home language, general administration directions read aloud and repeated as needed in student’s home language, scribe or speech-to-text responses dictated for mathematics assessment in English, and word-to-word dictionaries (that students bring). ELLs who are dually identified as students with disabilities are also offered accommodations for students with disabilities such as assistive technology or braille.

Our blog post on SBAC accommodations details their different approach. ELLs are allowed to use a set of universal accessibility tools such as scratch paper or a digital notepad and designated supports such as a translated pop-up glossary that’s built in to the computer-based assessment. In addition, ELLs who are dually identified are allowed accommodations such as braille or closed captioning.

Computer-Based Testing & ELLs

The computer-based testing environment will be new for many students, including ELLs. Here are some additional technology considerations to take into account, as well as some ideas for supporting ELLs' access to and familiarity with the technology necessary to take the tests.

PARCC’s Computer-Based Approach

PARCC provides several types of tutorials for students to navigate this new space, ranging from computer-based practice items at different grade bands to a tutorial on the features of the computer-based environment and tutorials on using accommodations available for special needs students. Since ELL accommodations are not built in to the computer-based PARCC tests, there are no student tutorials on their use. A screen shot of one of the PARCC tutorials is below.

While PARCC utilizes technology-enhanced items, the assessment is not computer adaptive. Computer adaptive testing means that each student receives a unique set of assessment items based on previous responses. With computer adaptive testing, if a student gets a response correct, he/she will then receive a more-difficult question designed to test a higher-order thinking task (and vice versa). For example, the Graduate Record Examinations (or GRE) test that many graduate schools require is computer adaptive.

Smarter Balanced’s Computer-Based Approach

In contrast to PARCC, Smarter Balanced utilizes computer adaptive testing to adjust the difficulty of questions throughout the assessment for each student. Smarter Balanced claims that computer adaptive testing provides better information for teachers, is more efficient and more secure, and is a more accurate way to evaluate student achievement.

Practice and training tests are available to students by grade band so they can experience all of the grade-specific item types as well as universal tools, designated supports, and accommodations on the exam. While ELLs are allowed a translated, pop-up glossary, I was not able to see what this looks like without having access to the secure site.

Computer-Based Testing: Challenges for ELLs

One of the biggest challenges for students in adjusting to computer-based assessment that we cannot overlook is the “digital divide” that exists between students from low-income homes (currently two-thirds of ELLs nationwide¹) and students whose families can afford access to technology in the home. These divides also tend to exist in terms of students’ more limited access to technology in schools which serve low-income neighborhoods.

In addition to less access to technology in general, ELLs also might not be fully versed or “literate” in the use of technology that’s necessary for them to navigate the test platform in particular. For example, students may be very familiar with mobile devices or cell phones to which they have access, but not with certain kinds of tasks on a computer like clicking and dragging numbers to a number line or typing an extended answer. In addition, ELLs arrive in schools at all points of the school year, and in many cases they will still need to take summative content tests even though they may not be privy to the same training on the test platform that other students are given. Regardless of their arrival dates, ELLs will need to understand all the online test features – especially accommodations when applicable – and know how to use them. The consortia assessments’ student tutorials are in English and seem to operate under the assumption that students using them are native English speakers, so there are no built-in scaffolds for ELLs.

Guidelines for ELL Computer-Based Content Testing

These guidelines provide a starting point from which to advocate for ELLs’ access to support for computer-based content assessments. There are certainly many more considerations, but these can be used as a checklist to begin to decide where to take action. While some may say that these suggestions constitute “teaching to the test,” I would argue that they are a way of positioning ELLs to have a better chance at focusing on the content of the assessments instead of getting distracted by the platform. I’d also argue that the first three supports would benefit non-ELLs as well.

• Teach the terms ELLs will need to know to navigate the platform. For example, from the PARCC tutorial screenshot above, some terms I’d work on with ELLs include “item,” “toolbar,” “select,” “deselect,” and “icon.” I’d also focus on some specific uses of terms for computer-based testing, such as to “flag an item.” One way to reinforce the terms is by posting a word wall of the terms, their meaning, an image, and a home language translation if possible.
• Devote time to practicing test items and navigating the test platform with ELLs. After pre-teaching necessary terms for ELLs to navigate the platform, have students apply these terms by working through the test platform together as a class, trying out some test items, and making sure that they know how to use the accommodations that are available to them. The teacher could model how he or she uses the technology while also doing a think-aloud of how he or she answers the content using ELL accommodations where available. Once the teacher has modeled practice items, ELLs could work on them together in pairs, noting their questions on the platform as well as content for the teacher.
• Make sure ELLs have access to keyboarding skills. These computer-based assessments assume that students can type their responses to questions. It is important to ensure that ELLs gain experience using keyboards so that they are not slowed down by typing skills. On that note, typing classes are gaining popularity for students in general with CCSS testing.
• Collaborate with content teachers and test specialists on use of accommodations. It is essential that ESL teachers work with grade level or content teachers as well as their school’s testing coordinator to ensure that ELLs receive the appropriate accommodations and are also given access to the accommodations that they are allowed on the assessments. ELLs should only be given those accommodations that are appropriate for their background and level of proficiency. (For example, a student not literate in her home language should not use a bilingual dictionary or translated online glossary.) ELLs should be given these same appropriate accommodations during classroom instruction.
• Make sure ELL parents and caregivers are aware of the test requirements and platform. ELL parents and caregivers (as well as the ELLs themselves) should see the platform their children will use and should also be aware of how the assessment results will be used. Testing could be the topic of an ELL parent meeting either in school or in a place convenient for parents to meet such as a community center.

Resources to help ELLs from Colorín Colorado

• Preparing ELLs to be 21st-Century Learners
• Technology and ELLs Resource Section

Let’s Hear from You!

What are you doing to help ELLs navigate the computer-based testing environment? What has been of particular help to you? What tips do you have? Where do challenges remain?

Footnotes

¹National Education Association Policy Brief. English Language Learner Face Unique Challenges http://educationvotes.nea.org/wp-content/uploads/2010/05/ELL.pdf