During the Spring 2003 semester, Wilde Lake Middle school conducted a formal research project to accelerate learning for its low performing student population. The goal was to increase test scores on the Maryland Functional Math Test (MFMT), a standardized test which is given at the middle school level. The student body is approximately half white, half African American, with over 25% of the student body in the free lunch program, and over 10% classified as special ed. With 8 weeks of intervention, Wilde Lake registered a 15% increase in test scores amongst the study participants.

DOC format, 2.1 MB (Right mouse click, "save target as")

Study Design

There were 3 areas of emphasis which contributed to the score increases.

1) Direct Instruction - Conventional textbook and teacher-designed lessons.

2) Computer Assisted Instruction - Students practiced taking the Maryland Functional Math Test on www.testfactory.net, with 20 different trial exams available. Students also practiced basic skills using the Ten Minute Numeracy Tests CD-ROM. Wilde Lake's teachers added 10 new question sets to the CD, which were specifically targeted at those skills covered on the MFMT standardized exam.

3) After School Programs - The study participants were also enrolled in at least one after school program where students could study animal husbandry, marine biology, mountaineering, or even become webmasters using MultiMedia Lab. You can view a student produced website.

Total number of students in treatment group = 43

Total number of students examined not in treatment group = 23

Treatment period = Total of 13 teaching days X 50 minutes per day (650 minutes)

Overall Score Increases

Here are the overall score increases for the full study group (treatment and non-treatment shown together).

Pass/Fail Rate Comparison Between Treatment and Non-Treatment Groups

• For non-treatments students, the probability of passing was a 50-50 proposition, with passes to failures at about a 1-to-1 ratio. Actual numbers 52% fail, 48% pass.
• After treatment with Ten Minute Numeracy Tests and www.testfactory.net, which occurred over a 6-8 week period (13 class sessions), the pass rate approached 75%, making the pass-to-fail ratio 3-to-1. Actual numbers 25.6% fail, 74.4% pass.

Pass Rate Based on Ten Minute Numeracy Tests Usage

The chart below shows the score increases when Ten Minute Numeracy Test is added to the www.testfactory.net treatment.

• Of the 43 students in the full treatment group, we collected detailed Ten Minute Numeracy Test achievement data for 18 of those students.
• Overall, students who used Ten Minute Numeracy Tests and www.testfactory.net combination experienced an 88.9% pass rate.
• Of the students who completed less than 250 Ten Minute Numeracy Test questions, 40% passed the MFMT.
• Of the students who completed more than 250 Ten Minute Numeracy Test questions, 100% passed the MFMT.

Comparative Point Gain in MFMT Scores

Below is an examination of the 22-point average gain in MFMT scores from October 2002 through Spring 2003.

• Non-treatment students gained 11 points on average whereas treatment students gained 26 points.
• Students of critical need received additional individualized attention during the study. These students are normally given a very low probability of passing the MFMT by the school's guidance department. Despite this, they registered the highest individual point gains of all students in the school, and had an overall MFMT pass rate of 79% and average 27 point gain.
• It should be noted that five students of critical need improved by 30 points or more, with two students improving by 50 points.

Statements of Fact

• Seventh grade achieved a 90% pass rate. This is the highest passing percentage at this grade at this time of the school year in the recent history of Wilde Lake Middle School.
• The average number of Test Factory practice tests completed in the treatment group was six.
• Of 15 students who completed 10 or more Test Factory practice tests, 14 passed the MFMT, a rate of 94%.
• A pre- and post attitudinal survey was administered to students in September and in May. Among the treatment group of students, all student responses were either equal to or higher in terms of student beliefs in their ability to perform well in the following categories. Fractions; Word Problems; Formulas; Decimals; Geometry; Long Division; Math Vocabulary; Times Tables; 60 Question Tests; Mathematic Properties; Use of Variables and Percents.
• Over 20% of all tests administered on Test Factory, were taken at night by students working at home.

What We Learned from the Study

Every technology product used in the study, provided students with direct and immediate feedback on their progress. This gave students the power to understand where their skill were weak, thus they immediately began to take responsibility for their own progress. We found students would challenge themselves and ask for help in areas where they were weak, rather than the teacher always having to assign the next activity. This changed the classroom dynamics in a positive way.

Family involvement became possible (using Test Factory at home) and this had a direct positive impact on student success.

The parent/teacher support and guidance, combined with tools which give students the ability to see and track their own progress, leads to accelerating achievement.

Score Increases, Informal Research

Many schools who did not participate in the formal research study, are also reporting test score increases with Test Factory usage. Here are some examples:

Kettering Middle School

John Bridgeo, Technology Coordinator/Math Chairperson

...Our experience in the first six weeks with www.testfactory.net has been nothing short of a phenomenon.  It has raised our overall scores in the school 10%. From 45% to 55%.  We hope to reproduce the effect this summer and get scores to the 65% level.  When we begin to use this program in the fall I know we will take the 7^th grade’s typical 25% passing rate and raise it to 50%.  This will cascade through those overall test scores for the school hopefully raising our scores in the school to the 90% level over the course of the next couple years. Read John's full letter.

Talbot County Public Schools, Maryland

Jan Greenhawk, Coordinator of Assessment and Staff Development for Talbot County

When we first gave the new online version of the functional math test, our pass rate was a very poor 27%. After a trial subscription to Test Factory, our pass rate has risen to 66%. This is an amazing increase! Our teachers and students love the program as it is easy to use and instrumental in getting our students prepared to take the functional test.

Brian Wessner, Tool Factory Trainer, Maryland

....a 7th grade student stopped me in the hallway after seeing my Tool Factory shirt. He asked "Are you the Test Factory guy?" I said yes. He got up from the floor, shook my hand, and said "Thank you for Test Factory. I passed." This student had not been expected to pass the MFMT during middle school.

Further Research

During summer school, Tool Factory will extend the formal research project to include four more Maryland schools, each with at least 50 students who have failed the Maryland Functional Math Test. Data from the summer study should become available in late September. Contact Brian Wessner for further information.