Location:
EDUCATION - TESTING;

OLR Research Report


March 31, 2010

 

2010-R-0167

CONNECTICUT MASTERY TEST'S MEASUREMENT
OF GRADE LEVEL SKILLS

By: Jeanne Hayes, Legislative Fellow

You asked whether the Connecticut Mastery Test (CMT) for math accurately measures grade-level skills.

SUMMARY

The math CMT measures how well students perform on subjects they are expected to have mastered by the time they take the test, as opposed to subjects they are currently learning. There are five CMT levels of performance; students scoring in the top two levels are considered to have met state goals. The State Board of Education (SBE) set these levels after considering the levels in other states, opinions of educators and organizations in Connecticut, and curriculum standards of The National Council of Teachers of Mathematics.

It is the position of the SBE that the CMT accurately measures grade-level skills. In addition to the CMT, fourth and eighth grade students also take the National Assessment of Educational Progress (NAEP). This test is administered by the federal government to compare student achievement across state lines and includes a math component. A study by the National Center for Education Statistics found that the math CMT is less rigorous than the math NAEP.

CMT HISTORY

Connecticut's public school students have been required to take the CMT since 1985. The SBE designed the test to (1) help schools and teachers identify curriculum weaknesses for improvement, (2) establish high expectations for Connecticut students, (3) identify students who need extra help, (4) improve the accountability of the education system, and (5) help parents and teachers monitor a child's achievement over the years. (SBE, Connecticut Mastery Test: What Every Parent/Guardian Should Know About the CMT for Grades 3 Through 8).

CMT DEVELOPMENT

The CMT tests math skills for students in grades three through eight. According to the SBE, the math test assesses grade-appropriate skills and concepts expected to be mastered by the time of testing, along with a student's ability to solve realistic problems (SBE, Connecticut Mastery Test: Understanding Test Scores on the Individual Student Report, 2009). The test is updated every five to seven years based on updates or changes to curriculum standards.

The math CMT is aligned with the 2007 Connecticut Prekindergarten-Grade 8 Mathematics Curriculum Standards, which documents state grade-level expectations (GLEs) and defines skills that every child should master by the end of the school year. Educators representing districts across the state, professional organizations, higher education institutions, and the regional educational service centers assisted the SBE in writing and reviewing the GLEs. According to the SBE, it made the GLEs comparable to the standards of The National Council of Teachers of Mathematics, and the “exemplary” state standards in Indiana and California (Connecticut PK-8 Mathematics Curriculum Standards).

CMT PERFORMANCE LEVELS

Student scores on the math CMT fall into one of five performance levels: Advanced, Goal, Proficient, Basic, and Below Basic. Each performance level has a grade-specific description. Table 1, for example, shows the performance descriptions for the fourth grade math test:

Table 1: Grade Four CMT Math Performance Levels

Scaled Score

Performance Level Description

293-400

Advanced

Generally, fourth-grade students who perform at this level demonstrate exceptional knowledge of grade-level content. These students demonstrate well-developed conceptual understanding, computational skills, and problem-solving skills, as well as an advanced ability to solve complex and abstract mathematical problems. Typically, the solutions these students provide to

Scaled Score

Performance Level Description

293-400

Advanced

(continued) math problems are well organized and include clear and concise explanations.

245-292

Goal

Generally, fourth-grade students who perform at this level demonstrate extensive knowledge of grade-level content. These students demonstrate well-developed conceptual understanding, computational skills and problem- solving skills, as well as an ability to solve complex and abstract mathematical problems. Typically, the solutions these students provide to math problems are organized and include clear and concise explanations.

215-244

Proficient

Generally, fourth-grade students who perform at this level demonstrate adequate knowledge of grade-level content. These students demonstrate adequate conceptual understanding, computational skills and problem-solving skills, as well as an ability to solve complex and abstract mathematical problems. Typically, the solutions these students provide to math problems are adequate and include sufficient explanations.

191-214

Basic

Generally, fourth-grade students who perform at this level demonstrate partially developed knowledge of grade-level content. These students demonstrate partially developed conceptual understanding, computational skills, and problem-solving skills, as well as a limited ability to solve complex and abstract mathematical problems. Typically, the solutions these students provide to math problems are unorganized and include minimal explanations.

100-190

Below Basic

Generally, fourth-grade students who perform at this level demonstrate limited knowledge of grade-level content. These students demonstrate limited conceptual understanding, computational skills and problem-solving skills, as well as a limited ability to solve complex and abstract mathematical problems. Typically, the solutions these students provide to math problems are inadequate and lack explanations.

Source: SBE, 2009 Interpretive Guide: Connecticut Mastery Test: Fourth Generation, 2009.

Connecticut's “goal range” includes the two highest levels, Advanced and Goal. SBE's position is that scoring in the goal range “is a challenging, yet reasonable expectation for Connecticut Students.” In other words, students scoring in the goal range have met GLEs.

CMT PERFORMANCE COMPARED TO PERFORMANCE ON THE NATIONAL MATH TEST

The federal government has tested academic achievement in the 50 states since 1969. The federal test is known as the National Assessment of Educational Progress (NAEP), and has a portion devoted to testing math skills. The NAEP math test is based on the Mathematics Framework, which contains what hundreds of educators, curriculum experts, policymakers, and members of the general public across the 50 states thought students should know (National Center for Education Statistics, How Was the NAEP Assessment Developed?, Sept. 9, 2009).

Scores on the NAEP fall into four performance levels: Below Basic, Basic, Proficient, and Advanced. Table 2 shows the fourth grade performance levels.

Table 2: Grade 4 NAEP Math Performance Levels

Scaled Score

Performance Level Description

>282

Advanced

Fourth-grade students performing at the Advanced level should apply integrated procedural knowledge and conceptual understanding to complex and nonroutine real-world problem solving in the five NAEP content areas.

Fourth-graders performing at the Advanced level should be able to solve complex and nonroutine real-world problems in all NAEP content areas. They should display mastery in the use of four-function calculators, rulers, and geometric shapes. The students are expected to draw logical conclusions and justify answers and solution processes by explaining why, as well as how, they were achieved. They should go beyond the obvious in their interpretations and be able to communicate their thoughts clearly and concisely.

249-281

Proficient

Fourth-grade students performing at the Proficient level should consistently apply integrated procedural knowledge and conceptual understanding to problem solving in the five NAEP content areas.

Fourth-graders performing at the Proficient level should be able to use whole numbers to estimate, compute, and determine whether results are reasonable. They should have a conceptual understanding of fractions and decimals; be able to solve real-world problems in all NAEP content areas; and use four-function calculators, rulers, and geometric shapes appropriately. Students performing at the Proficient level should employ problem-solving strategies such as identifying and using appropriate information. Their written solutions should be organized and presented both with supporting information and explanations of how they were achieved.

214-248

Basic

Fourth-grade students performing at the Basic level should show some evidence of understanding the mathematical concepts and procedures in the five NAEP content areas.

Fourth-graders performing at the Basic level should be able to estimate and use basic facts to perform simple computations with whole numbers, show some understanding of fractions and decimals, and solve some simple real-world problems in all NAEP content areas. Students at this level should be able to use—though not always accurately—four-function calculators, rulers, and geometric shapes. Their written responses will often be minimal and presented without supporting information.

<214

Below Basic

(No description)

Source: National Center for Education Statistics, Mathematics Achievement Levels by Grade,
Dec. 20, 2006.

In 2009, 46% of fourth grade students in Connecticut performed at or above the NAEP Proficient level. In contrast, that same year 84.6% of fourth grade students in Connecticut performed at or above the CMT Proficient level.

The National Center for Education Statistics has formally studied the comparative rigor of the 2007 Connecticut CMT for fourth and eighth graders and the 2007 NAEP (Mapping State Proficiency Standards onto NAEP Scales: 2005-2007, October 2009). To determine the comparative rigor, researchers (1) obtained from each school the proportion of students who achieved proficiency on the CMT; (2) estimated, using NAEP school weights, the proportion of students in Connecticut who achieved proficiency on CMT; (3) estimated the weighted distribution of NAEP scores for the state as a whole; and (4) found the point on the NAEP scale at which the estimated proportion of students in the state who scored above that point equaled the proportion of students in the state who met the state's own performance standard.

The study found that a fourth grade student who scored Proficient on the math CMTs would score Basic on the math NAEP (id. at 20). Thus, it found that the CMT is less rigorous than the NAEP fourth grade math test. According to the study, of the 48 state math tests for fourth graders studied, Connecticut's 2007 fourth grade math CMT is less rigorous than tests in 26 states.

In addition, the study found that the CMT is less rigorous than the NAEP eighth grade math test. It found that an eighth grader who scored Proficient on the math CMT would score Below Basic on the math NAEP. Thus, the study concluded that the CMT is less rigorous than the NAEP eighth grade math test. According to the study, of the 47 state math tests for eighth graders studied, Connecticut's eighth grade math CMT is less rigorous than tests in 42 states (id.)

JH:df