Why do Kids
Fail Tests?

Article by Heather Chirtea
When a crash came from across the classroom teacher Bob Keddell suppressed
the instinctive reaction to discipline Doug, a student who was quite famous
for his suspensions and general lack of involvement in education. Doug was working on Test Factory while balancing precariously on the two back legs of
his chair, the first time he scored above 340, a passing grade on the
Maryland Functional Math Test. Doug was so excited to pass he inadvertently
flipped himself over backwards, crashed to the floor and screamed "Look!".
Prone and pointing wildly at the computer screen he exclaimed, "Oh
my God!" several times. "I’ve never passed anything before!"
It’s true, Doug was projected never to pass the Maryland Functional Math Test. But
he, like many of his lowerperforming classmates at Wilde Lake Middle
School, did beat the odds and pass the test in May of 2003. These students
like many others, took part in a pilot project that spawned from a partnership
between Tool Factory and Howard County Public Schools in Maryland. The
project focused on raising test scores across the low performing student
population at Wilde Lake Middle School. As part of the project, Tool Factory
constructed a testtaking website where students could practice working
out the kinds of questions they might encounter on their standardized
tests. And even before it was finished, the Wilde
Lake students were using it in math class.
The Maryland Functional Math Test
was selected to be the basis for the pilot website because the test
had been administered for over 10 years.
There was a great deal of baseline data available from past testing
years, and schools could reliably predict student pass rates. There were also sample tests available, and writers could mimic
the oftentricky phrasing on various test questions. Armed with a few example tests, Tool Factory wrote 20 new Maryland Functional Math Tests and posted
them online at www.testfactory.net. The site was then posted freely to all schools
in the state of Maryland. There was immediate reaction with 10% of Maryland
schools requesting testing accounts within the first 60 days. At the peak
of the pilot, Test Factory was administering over 7,000 tests per
day to students across the state.
At
the same time the free statewide pilot was occurring, Wilde Lake Middle
School conducted research to track acceleration for its low performing
student population. Acceleration was measured against scores of peers
who were not taking part in the project. Wilde Lake’s student body is
approximately half white, half African American, with over 25% of the
students coming from lowincome families, and over 10% classified as special
ed. With just 8 weeks of intervention, Wilde Lake registered a 15% increase
in test scores amongst the study participants, a statistic which surprised
everyone working on the project. The
success wasn’t just confined to Wilde Lake though.
During the
same pilot period, Kettering Middle school reported school wide score
increases of 10%. The Coordinator
of Assessment and Staff Development for Talbot County Public Schools explained
in an email, “…their pass rate increased from 27% to 66% during the pilot.”
Melissa Robbins, a math teacher in Carroll County reported a 100%
pass rate, up from 75%. The teachers, researchers, and developers who
worked on the project learned a lot about why kids fail tests.
Here is what they found.
The Textbooks
A standardized test attempts to measure skills
across the entire curriculum at a single moment in time. Conversely, American textbooks are typically structured
to teach independent math skills in isolation.
Chapterbychapter students tackle one skill at a time: first addition,
then subtraction, then multiplication, then division, and so on. Students
repeatedly solve batches of like test questions.
The classic textbook structure that presents one skill per chapter, in
isolation, is not necessarily the best tool to prepare students to look
at random groups of questions from across the entire curriculum, and determine
which of those independent skills to apply in each instance.
Consider what this means from a practical perspective.
When students take their standardized tests, this may very well
be one of the first and only times they will ever be faced with answering
math questions across the entire curriculum in one sitting prior to high
school. Doing so requires a higher level of cognitive demand than repetitive
calculations. To pass a standardized
test, students need to be able to recognize when and where to apply the
independent concepts they have learned in their textbooks. Applying
concepts is a skill they need to learn, just like addition is a skill. And with a little
practice, students can master it just as readily.
Another pivotal attribute of a US textbook, is the
tendency to put one page of word problems at the end of each chapter. Word problems covering a single calculation
skill can be taught on different days or even in different units. When students switch from solving batches of
straight computational problems, to batches of word problems, they don't
always realize they are applying the same skill in a different way. They
don’t always make the connection. Thus a student with perfectly adequate
calculation skills may often fail a test question that’s phrased as a
word problem, out of context of the textbook chapter.
To solve questions out of context, students need to carefully read
every test question, then determine the proper calculative skill to apply. With a little practice “carefully reading test
questions” will also increase test scores.
In Singapore
There was a TIMSS study that tested students
in math and science from 38 countries, in order to compare proficiency
across all nations. Singapore students scored the highest worldwide in
mathematics, while the United States finished closer to the median of
all nations tested. So why did Singapore score so high? The answer begins in the textbooks. The Singapore textbooks focus on teaching the relationships between math
skills. They take the same
problem and solve it in multiple ways, thus presenting mathematics
as a system. Students may spend an entire 40minute class period
on just one single math problem, converting addition sentences into subtraction,
changing numbers into decimals, fractions, percents, exponents, scientific
notation, and absolute values. By
solving the same problem in a myriad of different ways, students become
practiced at fluidly translating between different math skills. With this in mind, it becomes easier to see
why the Singapore students might score higher on a standardized test.
They are accustomed to bouncing back and forth between various concepts
across the curriculum, thus the standardized test more closely matches
their daily classroom activities. And that translates into higher test
scores.
Teacher Bob Keddell, who works mainly with lower performing middle school
students said, “Singapore begins this strategy in second grade and earlier.
The one boy in my class who went through the Singapore system in
Hong Kong is now one of the top 5 students in Howard County at the secondary
level. Yes, he was stuck in my intervention class because he was foreign
and didn't speak English well...but after he brought me the Singapore
books that he finished in second grade (the same books I was using in
seventh grade) he testedout of all math available everywhere in Howard
County, and now takes math on Saturdays with a professor from Johns Hopkins
University.”
Mr. Keddell has seen firsthand how Singapore’s systematic teaching methods
can help close the achievement gap, and allow students to leave behind
elementary math very quickly. He
said, “Acceleration requires students to understand that every time they
look at problem, it can be solved in more than one way. Acceleration
for students who are labeled underachievers
often requires these kids to feel more powerful than the math that they
have already failed.” Practice
testing lets them to build that confidence in private, where they are
allowed to make mistakes in a nonthreatening environment.
It also fosters the development of skills in applying knowledge
across the curriculum, which is central to every student’s ability to
passing a standardized test.
Research Findings
Unexpected usage patterns cropped up during
the Test Factory pilot study
that surprised the entire team. We originally thought students would take
test #1, then move on to test #2, and continue until they achieved a passing
score. But when we reviewed the actual student data
we discovered that kids were taking the same tests overandover again. An active user with low test scores might take
the same test as many as 8 times in a row before moving on to the next
sequential test. Out of curiosity,
we consulted an assessment researcher for insight. He told us students could be trying to memorize the answers in order
to cheat a better score. But the
pattern was so pervasive we couldn’t ignore it so we asked some classroom
teachers for their thoughts.
John Bridgeo,
math chair at Kettering Middle School had a different view.
He said, “…because students can go back to a test and
complete it a second or third time they can gain confidence by beating
back old dragons. Once they have
gotten a 100% on a test the third time out, they seem to be able to take
an entirely new test and only require two tries to perfect the second
test, and soon they are doing tests getting perfect or near perfect scores
and self diagnosing their own problems.”
The system was administering over 7,000 tests per day during the pilot
period. Researchers were shocked that over 20% of the tests were being
administered between the hours of 7:00 PM and 11:00 PM at night. Due to
equity issues, teachers couldn’t assign Test Factory as homework,
because not all students have computers at home. That meant one in five
tests were taken voluntarily in the evening by students simply wanting
to pass the test.
Immediate Feedback
Numerous teachers reported that the immediate
feedback was pivotal to student improvement. It was even changing their
teacherstudent classroom dynamic. The immediate access to scoring data proved to give students the power
to understand where their skills were weak, thus they began to take personal
responsibility for their own progress, often for the first time. We found
students would challenge themselves and ask for help in areas where they
were weak, rather than the teacher always having to assign the next activity.
This changed the classroom dynamics in a very positive way.
Interpretation Matters
In order to figure out how to increase test scores, it's also useful to
dissect the process by which a test is written. A test writer begins with
a list of objectives that could appear on the state’s standardized test. A standard such as add three digit numbers might be transformed into a problemsolving test question as follows:
Samuel
picked 122 peaches and Jose picked 272 peaches.
How many peaches did they pick in total?
Alternatively, it may also be interpreted as a computational test question
such as:
122
+ 272 = ?
Both methods shown above are perfectly valid ways to test addition of three digit numbers. However depending on which state you are standing
in, the standardized test will likely be biased towards either a problem
solving approach or a computational approach. The way in which individual
standards are tested, will be dependent on both the interpretation of
the state’s test writer, as well as the mandates of the state testing
committee.
Our educators are faced with the daunting task
of interpreting these state standards and converting them into meaningful
classroom activities. It's not
always clearcut. No matter how
carefully state standards are worded, there will still be room for interpretation
on the part of the teacher.
In the classroom, the textbook will also favor one
approach or one style of phrasing, and teachers will be most likely to
teach the method presented by their textbook. If the textbook interpretation
matches the state's interpretation of a standard, then those students
will invariably score higher. However,
it is statistically impossible for any math textbook to match every state’s
interpretation of every objective across 50 states. Financially it's infeasible for the textbook publishers to print
a different textbook tailored for each state, so textbook writers frequently
take the middle ground.
Phrasing
is Tricky
We’ve seen students who can achieve passing scores on their unitbased classroom
tests throughout the year, and still fail their state standardized test.
Consider this test question:
You and two friends go
to a restaurant and decide to split the bill evenly. If the total bill
is $60, then how much will each of you have to pay?
If you answered $30, then you've made a very common standardized test error.
It's easy to overlook the phrase, "You and two friends...",
thus dividing by 2 rather than 3. The
phrasing on standardized tests typically employs these devices, and it's
easy to see how a student with perfectly adequate skills in division,
might get this test question wrong. Once
students get caught by tricky phrasing, they will remember the experience
and be less likely to make the same mistake again. However if the
first time they encounter these types of questions is on the state’s standardized
test, they may just fail!
No matter how simple a concept this may seem, if we want to raise test
scores we have to allow our students to practice solving the types of
problems they will encounter on their standardized tests. Otherwise, what the state is testing is not what we’re learning
in the classroom. There is too often a disconnect between how the state
is testing standards, and how they are being taught in the classroom. Tools are needed to bridge that gap so that
our students are given the power to succeed.
Testing by Computer
Testing by computer is “Manifest Destiny”. The days of standardized test results arriving 812 months after
tests are taken, and when it’s too late to intervene…will soon be ending.
As the internet pervades the classroom, more and more states are actually
giving their standardized tests on computers.
For many students, particularly lower income students without computer
access, the first test they ever take on a computer may very well be their
state proficiency exams. If this
is the gauge by which we determine whether our schools are classified
as "Failing", then why not allow the kids to prepare?
Download the Full Research
Study
http://www.toolfactory.com/Research/ResearchSummary.htm
Test Factory for MSA
Based on the success of the pilot project, Tool Factory has developed the
Test Factory system for the MSA tests at grades 35. Get
a free 30day trial at http://www.testfactory.net Test
Factory is sold as an annual subscription for a single school building
site license. With a Test Factory site license, every member of
the student body and staff can use their accounts at school, at home,
or anywhere there is an internet connection.
Unlimited
Usage for a Single School:
Elementary School $999/year
Middle School $1,999/year
High School $2,999/year
Tool Factory
accepts purchase orders, check, money order, Visa, or Master Card.
Call: (800) 2208386, Fax: (802) 3756860, Email: orders@toolfactory.com
Mail purchase orders to:
Tool Factory,
Inc.
3336 Sunderland
Hill Road
Sunderland,
VT 05250
