Spring 2004

Talking Science

A Thousand Resources for Science Teachers

GAYL O’CONNOR outlines the background to the Science Education Assessment Resource program.

Science Education Assessment Resource (SEAR) project links directly to a recommend ation and suggested action of the Commonwealth Department of Education, Science and Training (DEST) research report: The Status and Quality of Teaching and Learning of Science in Australian Schools.

In relation to science assessment:

It is recommended that the Commonwealth assist educational jurisdictions to reform assessment practice so that assessment more effectively serves the purpose of improving learning. Assessment must focus on the learning outcomes associated with scientific literacy.

One of the suggested actions identified was that:

The Commonwealth fund a national project to develop a resource bank of science assessment techniques and tasks that can be used by teachers to assess learning outcomes associated with scientific literacy.

Subsequently, DEST funded the development of an online resource bank of science assessment tasks for the compulsory years of schooling (P–10). The project commenced in June 2002, and will be completed in September 2004. The Australian Council for Educational Research (ACER) is responsible for the overall management of the project, working in close collaboration with Curriculum Corporation, Edith Cowan University, The Australian Science Teachers’ Association and The Academy of Science.

A steering committee and a science policy officers’ advisory committee, comprising members of each jurisdiction and sector, were established to ensure the underlying conceptual framework to which the assessment tasks were to be developed was acceptable and agreed to by all. The draft framework was developed by Professor Mark Hackling of Edith Cowan University. The framework is consistent with—and builds on— the Organisation for Economic Cooperation and Development’s (OECD) Programme for International Assessment (PISA) definition of scientific literacy.

Scientific literacy is the capacity to use scientific knowledge, to identify questions and to draw evidence-based conclusions in order to understand and help make decisions about the natural world and the changes made to it through human activity.

The scope of the project was such that the final resource bank was to contain at least 1000 items (questions).

Building the resource bank

There are two main components to the resource bank: the assessment tasks and the website through which teachers can access the tasks. The website also contains additional teacher resources, including an elaborated progress map for scientific literacy, based on the draft conceptual framework. The progress map describes the path of typical student progress through an area of learning—in this case, the development of scientific literacy. There are six levels (1–6) in the progress map. The items and tasks in the bank are linked to these levels.

Each component underwent a series of quality assurance stages. The website was tested at two stages—both stages involved primary and secondary teachers from each jurisdiction and sector across Australia. The feedback informed modifications to the website (design, functionality and content recommendations). The website (including sample tasks) was also circulated to the Steering Committee and Science Policy Officers’ group for comment at both stages.

The assessment tasks were developed and peer reviewed by the ACER test development team, and then subjected to review by the Fellows of the Academy for accuracy of science content. Critical friends to the project also provided feedback on the framework and on selected sample tasks.

Depending on the type of task, the assessment tasks were either trialled with whole classes (involving approximately 10 000 students in total) or piloted (involving smaller numbers of students).

The assessment tasks

To be consistent with the nature of the assessment of scientific literacy, the items were arranged in contextualised tasks. Each task has, on average, three or four items associated with it, linked by a common scenario or everyday context that is familiar to students.

In general, tasks contain a variety of item types, reflecting the real-world nature of the contexts selected for the task. The resource bank includes a variety of items that can be used for diagnostic, formative or summative assessment purposes. Given that tasks may contain more than one type of item, teachers are best placed to decide how to use a particular task with their students.

Diagnostic assessment

In general, diagnostic items would be used early in a learning/teaching sequence. Students are provided with a scenario (in a cartoon-style format) and asked to provide an explanation for the situation shown. Student responses can then be scrutinised against some common misconceptions drawn from the literature. Teaching implications and some suggested follow-up classroom activities are provided to address identified student misconceptions.

Summative assessment

Summative items are objective items, which require students to respond in a closed manner: providing a one- or two-word response; completing a matching exercise; or answering a multiplechoice question. The information gleaned from the student responses may be most useful to teachers at the end of a learning/teaching sequence for reporting purposes. However, such items can also be used during a learning/teaching sequence to inform progress towards learning outcomes.

Formative assessment

Formative items are intended to provide information during a learning/teaching sequence. They require an open response from students. Items of this type are expected to elicit a range of student responses that can be aligned with more than one level within the scientific literacy progress map. The information gleaned from student responses can be used by teachers to determine the next appropriate learning experience for students.

Searching the resource

In addition to assessment purposes, teachers search according to several additional criteria as shown in the screen shot of the search page:

• Conceptual strand/context
• Scientific literacy level
• Task type
• Learning outcome focus

Further explanation of these options is provided in the FAQ section of the website.

It is also possible to search by keyword, or to quickly locate a task that has been previously accessed by its task code.

Additional support materials

The items within the tasks are supported by rich marking guides, illustrated with actual student responses collected through the trialling or piloting phase.

For example:

The task Arthropods requires students to identify similarities in structure for a group of pictures of arthropods. Students then constructed a generalised arthropod. Students were asked to: Draw a picture of another animal (real or imaginary) that also has these common features. Also, label your picture pointing out the arthropod features.

Level 1 response. Describes one aspect of data.

Body, (but not segmented) with or without legs (if legs present, not jointed). Other features may or may not be present. Labels may be absent.

Actual student response:

The task Bird Watchers requires students to consider the impact of removal of forest on the bird life on a farm. Students were asked: What could be done to increase the number of bush stone-curlews on the farm?

Actual student responses:

• To increase the number of bush stone-curlews, burrows could be dug for nesting. Also, more trees could be planted to produce more leaf litter.
• Put more trees back in so there will be leaf litter and the insects that live in leaves will also come back so the bird will have food.
• Trees could be planted for leaf litter to build up and ground space could be allocated for the birds’ nests.

There is also a teacher resource section on the website that provides further resources to assist teachers to expand their assessment repertoire. This section includes links to specific illustrative tasks within the resource bank, and additional resources to provide ideas for other types of assessment techniques such as work samples, portfolios, observation checklists and peer assessment.

The SEAR project is due to be completed in September 2004. Once released by DEST for teacher use, the site can be accessed through www.curriculum.edu.au/science/.

References

Goodrum, D, Hackling, M & Rennie, L (2001). ‘The Status and Quality of Teaching and Learning of Science in Australian Schools’, a research report prepared for the Department of Education, Training and Youth Affairs, available at www.detya.gov.au/schools/publications/2001/science/index.htm.

OECD PISA home page for scientific literacy at www.pisa.oecd.org/pisa/science.htm.

Gayl O’Connor is Science and ICT Item Development Manager at Curriculum Corporation, and formerly SEAR Project Director at ACER.

The author owns the copyright in this article. For information related to the reuse of this work in any form please contact the publisher denise.quinn@curriculum.edu.au