Assessment

The Merck Institute for Science Education and its partner school districts have refined and implemented a comprehensive plan for assessing students' science learning. This plan was initiated in the fall of 1998 by representatives from the partner school districts. They identified five key questions to be answered with assessment information:

• To what extent are students achieving the district, state, and national standards?
• Where are the strengths and gaps in student understanding of science concepts?
• Are students increasing their ability to solve problems and apply knowledge?
• What is the correlation between professional development and student achievement?
• How well is the science program meeting the needs of all students?

After considering assessment strategies that might help to answer these questions, a four-component Partnership Assessment Plan was formed:

1. nationally recognized, standardized tests, that may include multiple-choice and/or short-answer items
2. assessment tasks including performance and/or short-answer items that reflect classroom practice and that are standardized in format, rubric and administration
3. summative assessments specific to the instructional modules districts are using and are standardized in format, rubric and administration
4. informal assessments that teachers use on a regular basis

The following summarizes the work the Partnership has done in implementing the assessment plan.

Component 1 - Nationally Recognized, Standardized Tests

For this component of the plan, partner districts initially consulted with each other and selected the nine open-ended science questions on the Stanford 9 as the nationally recognized assessment tool. It was administered to students in grades five and seven.

However, after several years of administration, concern about the relationship between nationally normed assessment tools and standards-based outcomes persisted within the Partnership. This concern was also expressed in the larger arena of NSF-funded Local Systemic Change (LSC) projects. The Merck Institute and its district partners had been the recipients of an LSC grant. In 2001, the LSC projects moved to develop an assessment tool that would be aligned with standards-based goals. For the 2001-2002 school year and for 2002-2003, the LSC Science Program Study was conducted using pre-test/post-test model at grade six. This Partnership-wide assessment provides the Partnership a data set that is better aligned with standards-based expectations than other available normed assessments. Partner districts no longer attempt to use nationally normed tests on a Partnership-wide basis, but use them for local purposes.

With a view toward making Component 1 assessments useful to classroom teachers, Institute staff and school district science education leaders work on format and presentation strategies for sharing data with teachers. Teachers identify possible interpretations of results and formulate questions that can be asked and answered with the assessment data. This is aimed at helping teachers use assessment data as a guide in their planning.

Component 2 - Performance or Short-answer Assessments That Reflect Classroom Practice

A Performance Assessment Project was initiated in the spring of the 1998-1999 school year. A Partnership-wide team selected performance tasks to be administered in grades three and seven. These performance tasks came from the TIMSS performance item bank and were selected for Partnership-wide administration beginning in the spring of 2000. In grade three, students used various objects to determine which of two magnets was stronger. In grade seven, students designed and carried out an experiment to determine the relationship between the temperature of water and the rate at which an Alka-Seltzer™ tablet dissolved.

In February of 2001, Institute and school district staff coordinated the second administration of the two tasks to approximately two thousand students in each grade level. The papers were scored by a scoring service using the TIMSS rubrics for the tasks. Afterward a Performance Assessment Team of about 30 teachers and science supervisors met to review the results and to design sessions for sharing the information with district teachers and administrators.

Within each district, teams of teachers took responsibility for sharing the results of the performance assessment with third, seventh, and other grade teachers during after-school sessions that were held in May. Classroom results were handed out at these meetings along with district results and information from the project team on how to interpret the results. Feedback from the meetings indicated that teachers felt they gained additional insight into how students respond to problem-solving challenges.

Similar information sessions were held for principals, vice principals and superintendents according to each district's strategy for communicating and using assessment results. The goals of these sessions were similar to those for teachers. The results and information on how to interpret them were shared and discussed. Principals then used the performance assessment results as a starting point for discussions with teachers about the development of student problem-solving abilities.

Component 3 - Summative Assessments Specific to Instructional Modules

The development or selection of summative assessments specific to instructional modules is a work in progress. Within several Partnership districts, committees of teachers have begun to move toward standardizing the format, rubric, and administration of end-of-module assessments. Their work has included grade-level review of the results of the assessments and a commitment to adopt specific assessments at defined grade levels. Some Peer Teacher Workshops and the follow-ups have been devoted to creating and using end-of-module assessments as envisioned in the assessment plan.

Component 4 - Informal Classroom-based Assessments

The Institute continues to disseminate copies of An Assessment Sampler, a selection of assessment tasks and student work samples together with teachers' insights regarding the assessment design process. Since its publication about 2300 copies of the Sampler have been distributed. The role of classroom level assessments was also emphasized in the Fall 2001 issue of Explorer, the Institute newsletter. The role of on going classroom level assessments, the Component 4 assessments, is re-emphasized in Peer Teacher Workshops. Partnership teachers are using assessment as a tool for improving their teaching and for monitoring the science learning of their students as envisioned in the Partnership Assessment Plan.