Why do Kids Fail Tests?

- Article by Heather Chirtea

When a crash came from across the classroom teacher Bob Keddell suppressed the instinctive reaction to discipline Doug, a student who was quite famous for his suspensions and general lack of involvement in education.  Doug was working on Test Factory while balancing precariously on the two back legs of his chair, the first time he scored above 340, a passing grade on the Maryland Functional Math Test.  Doug was so excited to pass he inadvertently flipped himself over backwards, crashed to the floor and screamed "Look!". Prone and pointing wildly at the computer screen he exclaimed, "Oh my God!" several times. "I’ve never passed anything before!"

It’s true, Doug was projected never to pass the Maryland Functional Math Test.  But he, like many of his lower-performing classmates at Wilde Lake Middle School, did beat the odds and pass the test in May of 2003. These students like many others, took part in a pilot project that spawned from a partnership between Tool Factory and Howard County Public Schools in Maryland. The project focused on raising test scores across the low performing student population at Wilde Lake Middle School. As part of the project, Tool Factory constructed a test-taking website where students could practice working out the kinds of questions they might encounter on their standardized tests.  And even before it was finished, the Wilde Lake students were using it in math class. 

The Maryland Functional Math Test was selected to be the basis for the pilot website because the test had been administered for over 10 years.  There was a great deal of baseline data available from past testing years, and schools could reliably predict student pass rates.  There were also sample tests available, and writers could mimic the often-tricky phrasing on various test questions.  Armed with a few example tests, Tool Factory wrote 20 new Maryland Functional Math Tests and posted them online at www.testfactory.net.  The site was then posted freely to all schools in the state of Maryland. There was immediate reaction with 10% of Maryland schools requesting testing accounts within the first 60 days. At the peak of the pilot, Test Factory was administering over 7,000 tests per day to students across the state.

At the same time the free statewide pilot was occurring, Wilde Lake Middle School conducted research to track acceleration for its low performing student population. Acceleration was measured against scores of peers who were not taking part in the project. Wilde Lake’s student body is approximately half white, half African American, with over 25% of the students coming from low-income families, and over 10% classified as special ed. With just 8 weeks of intervention, Wilde Lake registered a 15% increase in test scores amongst the study participants, a statistic which surprised everyone working on the project.  The success wasn’t just confined to Wilde Lake though.

During the same pilot period, Kettering Middle school reported school wide score increases of 10%.  The Coordinator of Assessment and Staff Development for Talbot County Public Schools explained in an email, “…their pass rate increased from 27% to 66% during the pilot.”  Melissa Robbins, a math teacher in Carroll County reported a 100% pass rate, up from 75%. The teachers, researchers, and developers who worked on the project learned a lot about why kids fail tests.  Here is what they found.

A standardized test attempts to measure skills across the entire curriculum at a single moment in time. Conversely, American textbooks are typically structured to teach independent math skills in isolation.  Chapter-by-chapter students tackle one skill at a time: first addition, then subtraction, then multiplication, then division, and so on. Students repeatedly solve batches of like test questions. The classic textbook structure that presents one skill per chapter, in isolation, is not necessarily the best tool to prepare students to look at random groups of questions from across the entire curriculum, and determine which of those independent skills to apply in each instance. 

Consider what this means from a practical perspective.  When students take their standardized tests, this may very well be one of the first and only times they will ever be faced with answering math questions across the entire curriculum in one sitting prior to high school. Doing so requires a higher level of cognitive demand than repetitive calculations.  To pass a standardized test, students need to be able to recognize when and where to apply the independent concepts they have learned in their textbooks. Applying concepts is a skill they need to learn, just like addition is a skill. And with a little practice, students can master it just as readily.    

Another pivotal attribute of a US textbook, is the tendency to put one page of word problems at the end of each chapter.  Word problems covering a single calculation skill can be taught on different days or even in different units.  When students switch from solving batches of straight computational problems, to batches of word problems, they don't always realize they are applying the same skill in a different way.  They don’t always make the connection. Thus a student with perfectly adequate calculation skills may often fail a test question that’s phrased as a word problem, out of context of the textbook chapter.  To solve questions out of context, students need to carefully read every test question, then determine the proper calculative skill to apply.  With a little practice “carefully reading test questions” will also increase test scores.

There was a TIMSS study that tested students in math and science from 38 countries, in order to compare proficiency across all nations. Singapore students scored the highest worldwide in mathematics, while the United States finished closer to the median of all nations tested.  So why did Singapore score so high?  The answer begins in the textbooks.  The Singapore textbooks focus on teaching the relationships between math skills.  They take the same problem and solve it in multiple ways, thus presenting mathematics as a system.  Students may spend an entire 40-minute class period on just one single math problem, converting addition sentences into subtraction, changing numbers into decimals, fractions, percents, exponents, scientific notation, and absolute values.  By solving the same problem in a myriad of different ways, students become practiced at fluidly translating between different math skills. With this in mind, it becomes easier to see why the Singapore students might score higher on a standardized test. They are accustomed to bouncing back and forth between various concepts across the curriculum, thus the standardized test more closely matches their daily classroom activities. And that translates into higher test scores. 

Teacher Bob Keddell, who works mainly with lower performing middle school students said, “Singapore begins this strategy in second grade and earlier.  The one boy in my class who went through the Singapore system in Hong Kong is now one of the top 5 students in Howard County at the secondary level.  Yes, he was stuck in my intervention class because he was foreign and didn't speak English well...but after he brought me the Singapore books that he finished in second grade (the same books I was using in seventh grade) he tested-out of all math available everywhere in Howard County, and now takes math on Saturdays with a professor from Johns Hopkins University.” 

Mr. Keddell has seen firsthand how Singapore’s systematic teaching methods can help close the achievement gap, and allow students to leave behind elementary math very quickly.  He said, “Acceleration requires students to understand that every time they look at problem, it can be solved in more than one way.  Acceleration for students who are labeled underachievers often requires these kids to feel more powerful than the math that they have already failed.”  Practice testing lets them to build that confidence in private, where they are allowed to make mistakes in a non-threatening environment.  It also fosters the development of skills in applying knowledge across the curriculum, which is central to every student’s ability to passing a standardized test.

Unexpected usage patterns cropped up during the Test Factory pilot study that surprised the entire team. We originally thought students would take test #1, then move on to test #2, and continue until they achieved a passing score.  But when we reviewed the actual student data we discovered that kids were taking the same tests over-and-over again.  An active user with low test scores might take the same test as many as 8 times in a row before moving on to the next sequential test.  Out of curiosity, we consulted an assessment researcher for insight.  He told us students could be trying to memorize the answers in order to cheat a better score.  But the pattern was so pervasive we couldn’t ignore it so we asked some classroom teachers for their thoughts.

John Bridgeo, math chair at Kettering Middle School had a different view.  He said, “…because students can go back to a test and complete it a second or third time they can gain confidence by beating back old dragons.  Once they have gotten a 100% on a test the third time out, they seem to be able to take an entirely new test and only require two tries to perfect the second test, and soon they are doing tests getting perfect or near perfect scores and self diagnosing their own problems.”

The system was administering over 7,000 tests per day during the pilot period. Researchers were shocked that over 20% of the tests were being administered between the hours of 7:00 PM and 11:00 PM at night. Due to equity issues, teachers couldn’t assign Test Factory as homework, because not all students have computers at home. That meant one in five tests were taken voluntarily in the evening by students simply wanting to pass the test.

Numerous teachers reported that the immediate feedback was pivotal to student improvement. It was even changing their teacher-student classroom dynamic. The immediate access to scoring data proved to give students the power to understand where their skills were weak, thus they began to take personal responsibility for their own progress, often for the first time. We found students would challenge themselves and ask for help in areas where they were weak, rather than the teacher always having to assign the next activity. This changed the classroom dynamics in a very positive way.

Interpretation Matters

In order to figure out how to increase test scores, it's also useful to dissect the process by which a test is written. A test writer begins with a list of objectives that could appear on the state’s standardized test.  A standard such as add three digit numbers might be transformed into a problem-solving test question as follows:

Samuel picked 122 peaches and Jose picked 272 peaches.  How many peaches did they pick in total?

Alternatively, it may also be interpreted as a computational test question such as:

122 + 272 = ?

Both methods shown above are perfectly valid ways to test addition of three digit numbers.   However depending on which state you are standing in, the standardized test will likely be biased towards either a problem solving approach or a computational approach. The way in which individual standards are tested, will be dependent on both the interpretation of the state’s test writer, as well as the mandates of the state testing committee. 

Our educators are faced with the daunting task of interpreting these state standards and converting them into meaningful classroom activities.  It's not always clear-cut.  No matter how carefully state standards are worded, there will still be room for interpretation on the part of the teacher. 

In the classroom, the textbook will also favor one approach or one style of phrasing, and teachers will be most likely to teach the method presented by their textbook. If the textbook interpretation matches the state's interpretation of a standard, then those students will invariably score higher.  However, it is statistically impossible for any math textbook to match every state’s interpretation of every objective across 50 states.  Financially it's infeasible for the textbook publishers to print a different textbook tailored for each state, so textbook writers frequently take the middle ground.

Phrasing is Tricky

We’ve seen students who can achieve passing scores on their unit-based classroom tests throughout the year, and still fail their state standardized test.  Consider this test question:

You and two friends go to a restaurant and decide to split the bill evenly. If the total bill is $60, then how much will each of you have to pay?

If you answered $30, then you've made a very common standardized test error.  It's easy to overlook the phrase, "You and two friends...", thus dividing by 2 rather than 3.  The phrasing on standardized tests typically employs these devices, and it's easy to see how a student with perfectly adequate skills in division, might get this test question wrong.  Once students get caught by tricky phrasing, they will remember the experience and be less likely to make the same mistake again. However if the first time they encounter these types of questions is on the state’s standardized test, they may just fail!

No matter how simple a concept this may seem, if we want to raise test scores we have to allow our students to practice solving the types of problems they will encounter on their standardized tests.  Otherwise, what the state is testing is not what we’re learning in the classroom. There is too often a disconnect between how the state is testing standards, and how they are being taught in the classroom.  Tools are needed to bridge that gap so that our students are given the power to succeed.

Testing by Computer

Testing by computer is “Manifest Destiny”.  The days of standardized test results arriving 8-12 months after tests are taken, and when it’s too late to intervene…will soon be ending. As the internet pervades the classroom, more and more states are actually giving their standardized tests on computers.  For many students, particularly lower income students without computer access, the first test they ever take on a computer may very well be their state proficiency exams.  If this is the gauge by which we determine whether our schools are classified as "Failing", then why not allow the kids to prepare? 

http://www.toolfactory.com/Research/ResearchSummary.htm

Based on the success of the pilot project, Tool Factory has developed the Test Factory system for the MSA tests at grades 3-5. Get a free 30-day trial at http://www.testfactory.net Test Factory is sold as an annual subscription for a single school building site license. With a Test Factory site license, every member of the student body and staff can use their accounts at school, at home, or anywhere there is an internet connection.

Unlimited Usage for a Single School:
Elementary School $999/year
Middle School $1,999/year
High School $2,999/year

Tool Factory accepts purchase orders, check, money order, Visa, or Master Card.
Call: (800) 220-8386, Fax: (802) 375-6860, Email: orders@toolfactory.com
Mail purchase orders to:

Tool Factory, Inc.

3336 Sunderland Hill Road

Sunderland, VT 05250