During the Spring 2003 semester,
Wilde Lake Middle school conducted a research project to accelerate
learning for its low performing student population. The goal was to increase
test scores on the Maryland Functional Math Test (MFMT), a standardized
test which is given at the middle school level. The student body is approximately
half white, half African American, with over 25% of the student body in
the free lunch program, and over 10% classified as special ed. With 8
weeks of intervention, Wilde Lake registered a 15% increase in test scores
amongst the study participants.
DOC format, 2.1 MB (Right mouse click, "save target as")
Study Design
There were 3 areas of emphasis
which contributed to the score increases.
1) Direct Instruction
 Conventional textbook and teacherdesigned lessons.
2) Computer Assisted Instruction
 Students practiced taking the Maryland Functional Math Test on
www.testfactory.net, with 20
different trial exams available. Students also practiced basic skills
using the Ten
Minute Numeracy Tests CDROM. Wilde Lake's teachers added 10
new question sets to the CD, which were specifically targeted at those
skills covered on the MFMT standardized exam.
3) After School Programs
 The study participants were also enrolled in at least one after school
program where students could study animal husbandry, marine biology,
mountaineering, or even become webmasters using MultiMedia
Lab. You can view a student
produced website.
Total number of students
in treatment group = 43
Total number of students
examined not in treatment group = 23
Treatment period = Total
of 13 teaching days X 50 minutes per day (650 minutes)
Overall Score Increases
Here are the overall score
increases for the full study group (treatment and nontreatment shown
together).
Pass/Fail
Rate Comparison Between Treatment and NonTreatment Groups
 For nontreatments students,
the probability of passing was a 5050 proposition, with passes to
failures at about a 1to1 ratio. Actual numbers 52% fail, 48% pass.
 After treatment with Ten
Minute Numeracy Tests and www.testfactory.net,
which occurred over a 68 week period (13 class sessions), the pass
rate approached 75%, making the passtofail ratio 3to1. Actual
numbers 25.6% fail, 74.4% pass.
Pass Rate
Based on Ten
Minute Numeracy Tests Usage
The chart below
shows the score increases when Ten
Minute Numeracy Test is added to the www.testfactory.net
treatment.
Comparative Point Gain
in MFMT Scores
Below is an examination
of the 22point average gain in MFMT scores from October 2002 through
Spring 2003.
 Nontreatment students
gained 11 points on average whereas treatment students gained 26 points.
 Students of critical need
received additional individualized attention during the study. These
students are normally given a very low probability of passing the
MFMT by the school's guidance department. Despite this, they registered
the highest individual point gains of all students in the school,
and had an overall MFMT pass rate of 79% and average 27 point gain.
 It should be noted that
five students of critical need improved by 30 points or more, with
two students improving by 50 points.
Statements of Fact
 Seventh grade achieved
a 90% pass rate. This is the highest passing percentage at this grade
at this time of the school year in the recent history of Wilde Lake
Middle School.
 The average number of
Test Factory practice tests completed in the treatment group was six.
 Of 15 students who completed
10 or more Test Factory practice tests, 14 passed the MFMT, a rate
of 94%.
 A pre and post attitudinal
survey was administered to students in September and in May. Among
the treatment group of students, all student responses were either
equal to or higher in terms of student beliefs in their ability to
perform well in the following categories. Fractions; Word Problems;
Formulas; Decimals; Geometry; Long Division; Math Vocabulary; Times
Tables; 60 Question Tests; Mathematic Properties; Use of Variables
and Percents.
 Over 20% of all tests
administered on Test Factory, were taken at night by students working
at home.
What We
Learned from the Study
Every technology
product used in the study, provided students with direct and immediate
feedback on their progress. This gave students the power to understand
where their skill were weak, thus they immediately began to take responsibility
for their own progress. We found students would challenge themselves
and ask for help in areas where they were weak, rather than the teacher
always having to assign the next activity. This changed the classroom
dynamics in a positive way.
Family involvement
became possible (using Test Factory at home) and this had a direct positive
impact on student success.
The parent/teacher
support and guidance, combined with tools which give students the ability
to see and track their own progress, leads to accelerating achievement.
Score Increases,
Informal Research
Many schools who did not
participate in the research study, are also reporting test score
increases with Test Factory usage. Here are some examples:
Kettering Middle School
John
Bridgeo, Technology Coordinator/Math Chairperson
...Our experience
in the first six weeks with www.testfactory.net
has been nothing short of a phenomenon.
It has raised our overall scores in the school 10%. From 45%
to 55%. We hope to reproduce
the effect this summer and get scores to the 65% level.
When we begin to use this program in the fall I know we will
take the 7^{th} grade’s typical 25% passing rate and raise it
to 50%. This will cascade through those overall test scores for the school
hopefully raising our scores in the school to the 90% level over the
course of the next couple years. Read
John's full letter.
Talbot
County Public Schools, Maryland
Jan
Greenhawk, Coordinator of Assessment and Staff Development for Talbot
County
When we first
gave the new online version of the functional math test, our pass rate
was a very poor 27%. After a trial subscription to Test Factory, our
pass rate has risen to 66%. This is an amazing increase! Our teachers
and students love the program as it is easy to use and instrumental
in getting our students prepared to take the functional test.
Brian
Wessner, Tool Factory Trainer, Maryland
....a 7th
grade student stopped me in the hallway after seeing my Tool Factory
shirt. He asked "Are you the Test Factory guy?" I said yes. He got up
from the floor, shook my hand, and said "Thank you for Test Factory.
I passed." This student had not been expected to pass the MFMT during
middle school.
Further
Research
During summer
school, Tool Factory will extend the research project to include
four more Maryland schools, each with at least 50 students who have
failed the Maryland Functional Math Test. Data from the summer study
should become available in late September. Contact Brian
Wessner for further information.
