Education Development Center, Inc.
Center for Children and Technology
Domain Projects as Assessment Vehicles in a Computer-Rich Environment
CTE Technical Report Issue No. 5
August 1990
Prepared by:
Joseph Walters and Howard Gardner
Project Zero, Harvard Graduate School of Education
Introduction
This paper reviews recent research that has been conducted at Project Zero
at the Harvard Graduate School of Education on the topic of assessment.
In the paper we argue that an assessment built around an interesting and
challenging project that explores a particular domain (such as music, writing,
drawing, mathematics, science, or programming) offers an important alternative
to traditional standardized tests. As students work through such projects,
they reveal most vividly the skills and aptitudes that we want to assess.
We refer to these assessment instruments as domain projects.
The theory of multiple intelligences provides much of the conceptual underpinnings
of our view of assessment, and the paper begins with a brief synopsis of
this theory. Next, we review examples of domain projects under development
in two different research initiatives at Project Zero--Arts PROPEL and Catalyst.
Drawing from these examples, we list the salient characteristics of this
project-based assessment and outline the design of one such project that
uses computer technology.
The paper concludes by indicating directions for future research and development.
Assessment at Project Zero
For the past twenty years, Project Zero has taken as its research agenda
the systematic study of children and their symbol-using skills by exploring
such diverse areas as drawing, metaphoric language, musical ability, storytelling,
written language, mathematics, and computer programming. The diversity of
this research is responsible in part for the development of the view of
human cognition that we call the theory of "multiple intelligences"
(Gardner, 1983).
With respect to assessment, we draw two conclusions from this particular
view of human cognition. First, an evaluation of achievement or aptitude
must be specified within a domain of human activity. Second, the assessment
must be drawn based on a true performance in that domain. In this section,
we will build the argument by reviewing the central ideas of multiple intelligences
and then outlining the implications of the theory for assessment.
Multiple Intelligences
The theory of multiple intelligences claims that human beings have evolved
at least seven different forms of knowing or processing information. These
different forms, called "intelligences," include the skills for
manipulating language, logic and mathematics, musical ability, spatial information,
bodily-kinesthetic information, knowledge of other persons (interpersonal),
and knowledge of oneself (intrapersonal). All normal human beings possess
some capacity in each of these intellectual spheres, but the interaction
of genetic and environmental factors produce marked differences in the profiles
of the various intelligences in individuals.
This theory was designed to serve two goals: (1) to synthesize a large set
of findings about human cognition, including neurobiological evidence, cross-cultural
analysis, and developmental milestones; and (2) to provide an alternative
to the widespread belief in a single faculty- -intelligence--that can be
adequately assessed by paper-and-pencil "intelligence tests."
In the theory of multiple intelligences, the specific skills of human cognitionthe
intelligencesare mobilized for solving problems within particular domains
of activity. Each intelligence is an evolved biopsychological potential
that is manifested as a particular cognitive skill. In contrast, the domain
of that potential is defined by the culture as an arena in which the various
cognitive skills are mobilized. In the theory of multiple intelligences,
analysis of problem solving requires consideration of both the cognitive
functioning of the intelligences (and their combination) and the context
of a domain specified within a particular culture.
The theory of multiple intelligences stipulates that the various intelligences
are independent. For instance, when an individual displays a high degree
of competence with one intelligence, this ability does not imply similar
competencies in other areas. Similarly, disability in one intelligence does
not imply disabilities in the others.
Although the intelligences are independently structured in this way, they
do not function independently. Any reasonably complex adult task requires
the simultaneous functioning of several of the intelligences. For example,
the task of writing a research paper makes primary use of the linguistic
intelligence, but it also taps the logical-mathematical intelligence at
the same time. Playing the violin for an audiencefirst and foremost a musical
task also makes demands on bodily-kinesthetic and interpersonal faculties.
How an individual combines these separate intelligences is part of that
individual's personal endowment, just as are the intelligences themselves.
These two outcomes of the theory of multiple intelligences that the intelligences
are independent and that they operate in concert has important implications
for the problem of assessment. First, an assessment must pose problems in
which individuals work with the actual materials of the domain being examined.
Second, a complete assessment must pose a number of problems that yield
to a variety of solutions in order to reveal an accurate picture of the
talents and skills of a given individual.
To illustrate the first point, consider the task of assessing an individual's
musical competence. If the assessment consists of a number of questions
in a multiple-choice format, the assessment is less a measure of the musical
intelligence and more a measure of linguistic facility or test-taking skill.
As an alternative, the assessment might ask the student to compose an ending
to a simple melody. To tackle this problem, the student must manipulate
musical notation, select musical sounds, and use musical terminology. In
this task, the student makes demands directly on the musical intelligence
the problem is not filtered through a linguistic or logical-mathematical
assessment instrument, as it is in the multiple-choice example. This is
what we mean when we say that to evaluate an underlying intelligence, an
assessment must pose problems that require the individual to manipulate
the actual materials of the domain of that intelligence. We must examine
true performance and not just a verbalization of a problem solution.
Second, the fact that cognition is composed of several independent intelligences
operating in concert implies that an assessment must pose a variety of problems
that yield to different types of solutions if it is to establish a complete
picture of the profile of intelligences for an individual. For instance,
by asking a student to solve our music composition task, we do not learn
much about that student's ability to write short stories. Similarly, we
cannot infer that individuals who do not write well are equally disinclined
to perform well on tasks that require interpersonal skills, spatial abilities,
or musical prowess. Any simple task, even when it requires genuine performance
with the materials of a domain, does not reveal the complete profile of
an individual's talents. Therefore, a complete assessment of an individual
must make demands on all the intelligences, not just a select few.
Finally, the theory of multiple intelligences underscores the importance
of the "personal" skills in daily life. Working cooperatively
in a group is recognized as an important feature of many adult situations,
from manufacturing to family dynamics. Also, understanding one's self, the
intrapersonal skill, is also highlighted in the theory. For instance, a
strength in the musical realm may be underutilized by an individual who
does not fully understand that strength, whereas an individual with a strong
intrapersonal intelligence may be better equipped to combine modest strengths
or to compensate for weaknesses efficiently. A complete assessment should
take both of the personal skills into account, and the assessments we describe
below consistently feature both small-group collaboration
and reflection on learning.
Assessment through Projects
To summarize, assessments of ability and learning must engage students in
performances in which they handle the actual materials of a given domain
and mobilize a number of the different intelligences. We believe that problems
that meet both criteria can often be formatted as projects.
At Project Zero, we have been devising a number of exemplar projects in
a variety of domains. These projects are at once open-ended, structured,
and oriented towards products. In order to pose rich, complex, and engaging
problems, the projects are open-ended they offer opportunities to find alternative
solutions and unexpected outcomes using different strategies and different
combinations of intelligences. At the same time, the projects are structuredthe
results can be analyzed in terms of what students are learning. Finally,
as students work through projects, they are called on to create original
products songs, essays, drawings and this gives them a stake in the outcome
of the project.
The products that students create in working on a project are evaluated
by the teacher and the student together. We follow the work of Collins and
Frederiksen (1989) in designing these evaluations. First, the evaluations
consider only the performances that reveal directly the skills and competencies
that we are interested in; again, we ask students to exhibit skills, and
not simply to describe those skills. Second, the scope of each project is
designed such that every skill that we are interested in is exhibited; we
do not design evaluations that test only a sample of the desired skills.
Finally, all of these criteria are articulated at the outset of the project,
making them transparent to the students. In the final section of the paper,
we will return to these principles of problem design and evaluation.
Currently, two research initiatives at Project Zero are developing specific
examples of assessment projects. Arts PROPEL is designing instruments in
the arts; and Project Catalyst has created a series of computer-based projects.
We will outline these projects in the next two sections. Following this,
we will describe a project that we are developing for the Center for Technology
in Education that embodies these several design principles.
Domain Projects in Arts
PROPEL
Arts PROPEL is a five-year research effort sponsored by the Rockefeller
Foundation. Its primary goal is to create and test new techniques for fostering
and assessing artistic development in secondary school students. The project
brings together researchers from Project Zero and the Educational Testing
Service and teachers of music, visual arts, and imaginative writing in the
Pittsburgh public schools.
Arts PROPEL has designed two assessment techniques for arts classrooms:
portfolios and domain projects. A portfolio is a collection of materials
that the student assembles during the process of creating a finished piece
and includes sketches, early drafts, and notes as well as the final piece
itself. These selections may be supplemented with a written journal in which
the student reflects on the process of creating the piece.
It is the second technique, the domain project, that is directly relevant
to the present discussion. Each domain project in Arts PROPEL is composed
of a clearly defined set of classroom tasks that focus students on a central
issue in the art form. We will illustrate Arts PROPEL domain projects with
examples drawn from the domains of music and imaginative writing, but projects
are also developed in the visual arts. These descriptions begin with a summary
of the project along with a description of the products that the students
create. This is followed by a discussion of the various assessment techniques
which includes reflection exercises and teacher judgments.
Sample Projects
Musical Performance: Ensemble Critique
In musical performance, rehearsal is the main avenue for developing instrumental
skill. In a typical rehearsal session, the teacher leads a number of student
instrumentalists, guiding their development by pointing out errors, highlighting
certain areas for additional private practice, adjusting the ensemble sound,
and suggesting stylistic interpretation.
In the Ensemble Critique project, students begin to make judgments about
their performances of the sort that were previously made by the teacher.
At the beginning of the project, the teacher leads a discussion of critique
and gives the students special scoring sheets that
they will use to critique a performance. The class then examines the score
of the piece they will perform, considering the style, the key, and other
important features.
Next, the students perform the piece as an ensemble and tape record their
performance. They evaluate their performance from memory; then they listen
to the tape and compare what they hear with what they remember. Finally,
they fill out their score sheets as a formal critique of the performance,
focusing on both the performance of their part as well as the entire ensemble.
This performance and critique process is repeated two more times during
the semester. In later performances, students can compare their performance
with earlier recordings. As the student musicians become increasingly proficient
at the process of critique, they begin to take over other tasks of the rehearsal,
including conducting the ensemble in rehearsal, determining which portions
of the piece require additional attention, and even making stylistic changes
in the performance.
Assessment of the Ensemble Critique project consists of the reflection activities
that students carry out at each of the three check points. As they fill
out the critique score sheets, they assess the performances of individuals
as well as the ensemble as a whole. The teacher also evaluates these performances
and students can compare their self-assessments with those of the teacher.
With practice, students become more precise in their ability to evaluate
the performance, and this precise evaluation in turn makes them more proficient
in their ability to rehearse themselves.
This project is not designed as an assessment of performance. Instead, it
is an assessment of the students' ability to critique a performance and
to adjust their rehearsal accordingly. In this project the students begin
to share responsibility for decisions that are usually made by the director
alone.
Imaginative Writing: Writing Dialog
All studentsread plays in English class but very few write them.
In the typical class, for example, the plays of Shakespeare are treated
as literature that can be studied and analyzed, not as scripts to be performed
and interpreted. The PROPEL project called Writing Dialog provides students
with a set of structured activities that develop their skill at creating
original dramatic dialogue. Brief passages from contemporary drama are examined
and judged using the same criteria that are used to critique the students'
work.
The Writing Dialog project is conducted over seven class sessions. Throughout
these sessions, students are learning to articulate the criteria by which
dialogues can be critiqued. In each session, written work is read out loud
and discussed.
First, the teacher introduces the project by circulating several clipboards
around the class. Each clipboard has written at the top a single line of
dialogue and each student adds an additional line. When the sheet is filled,
the class reads the resulting dialogues out loud and discuss the results.
Next, working in pairs, students imagine a setting and write a simple scene
with two characters that takes place in that setting. The students in each
pair write alternate lines in the dialogue. Again, the class reads the scenes
out loud and discusses the results. This process of writing and rewriting
continues for several class sessions.
All writing done for the project is saved in a folder, and in this class
session the students select a piece from that folder to work from, adding
approximately ten lines of dialogue to the scene. At the conclusion of the
project, students review their collection of dialogues. They record their
observations of the changes they have seen in their own writing over the
course of the project.
In this project, students begin writing dialogues with no instruction. They
can then use these first efforts to begin to explore those features that
make for a "good" dialoguesense of character, scene, motivation,
and so on. As their dialogue writing continues, they elaborate these criteria
and practice using them to evaluate their own dialogues as well as dialogues
from published plays. By looking back over their earlier work, students
can see a general improvement through the project; they can also see that
they have a deeper understanding of what makes a dialogue work. In this
way, the project helps the students to use their own productions to construct
the criteria they need later to reflect on their own work.
Assessment in this project focuses on the collection of written material
that the students create in the seven working sessions. At the end of the
project, students evaluate their entire collection, looking for evidence
of their developing skill at writing dialogue.
This self-examination is then compared with the teacher's review of the
same material.
These sample projects, Ensemble Critique and Writing Dialog, illustrate
the breadth of projects that have been created in Arts PROPEL. Currently,
these domain projects, as well as a number of others, are being extensively
tested in the Pittsburgh Public Schools. They will be available for broader
dissemination in 1990.
Analysis
To summarize, PROPEL projects are sets of activities that are presented
over the course of the school year. Each project poses problems that students
solve by creating original work. In producing original work, the students
use the work of others or reproductions of master works for points of contrast.
Throughout this process students critique their own work.
Teachers can use the PROPEL projects to assess students by evaluating the
final products, by documenting the change in skills displayed over the course
of the project or between two projects, and by measuring their students'
growing abilities to recognize positive and negative elements in their own
work. The PROPEL project assessments are not designed to measure static
competencies nor to determine precisely what has been "learned"
in a specific curriculum unit. Instead, they demonstrate learning over a
period of time.
These projects are designed to incorporate the features of assessment drawn
from the theory of multiple intelligences. The PROPEL domain projects require
students to deploy a variety of intelligences as they engage specific problems.
Students produce a product in each project, such as a musical piece or a
dramatic dialogue. They also work in groups throughout the project, featuring
the interpersonal skills of collaboration and group discussion. Finally,
the PROPEL tasks tap intrapersonal skill by asking students to reflect on
their unique approach to the problems posed in that project. In the PROPEL
model, assessment is the continuing process of taking stock of one's current
position and comparing that state to the desired state. As students become
more sensitive to their current state and how it compares with the desired
state, they begin to take on more responsibility for their own learning.
Second, the PROPEL projects provide students with multiple opportunities
for problem solving. In
this program assessment does not rest on the results of a single PROPEL
project but is drawn from students' performances over a series of projects
given throughout the year.
Catalyst Projects
Catalyst is a five-year investigation funded by the John and Mary R. Markle
Foundation to explore how children and adults learn with microcomputers.
We focused this investigation on the experiences of individuals as they
approached some area of endeavor for the first time, examining novices'
performance in music and visual art, and more recently in mathematics, writing,
and computer programming. Since the difficulties that novices experience
in various areas are well documented, we wondered if the computer could
alleviate some of those problems. We felt that this research question would
shed some interesting light on the general issue of computer-aided learning.
This initial research effort demonstrated that even when assisted by powerful,
straightforward, and inexpensive software tools, novices did not make a
sustained entry into a domain when they worked on their own. Although these
novices did not have difficulty with the software itself, their lack of
experience with the domain prevented them from using the software effectively.
At the same time, however, this early research determined that novices could
use such software most productively when they were presented with an inviting
task and considerable help.
Here is an example. We asked novice adults to use a music composition computer
program to solve a number of simple harmony tasks. We (and the subjects
themselves) were surprised to find that even with little musical training
these beginners could solve the problems quite proficiently. However, when
we then posed a more open-ended problem, like composing an original melody,
we discovered that our novices were much less successful. Even with the
computer as an aid, they could not tackle the more unstructured problem
effectively on their own.
We took this research finding as a starting point for the design of computer-enhanced
projects that would enable novices to work effectively in a new domain.Three
prototype Catalyst projects have been developed in the past year to test
this idea. What follows is a description of these projects along with the
principles we used in designing them. Our current research initiative, also
funded by the Markle Foundation,
is to evaluate the efficacy of these projects.
Design of the Catalyst Projects
Each Catalyst project consists of three essential components:
1. A powerful software tool. Each project is built around a powerful
computer program. We select software that is inexpensive, readily available,
and powerful enough for professionals in the domain. Examples include music
composition tools, computer program editors, and word processing software.
The software tool reduces the barrier posed by the prerequisite skills that
make up the craft of the domain, which an expert would hone through years
of practice. In music, for example, the computer produces the performance
of the composition and can even flag several kinds of syntactic errors.
2. An on-disk library. Each project contains detailed information
in the form of computer files. In the projects described below, this library
can contain songs, programming procedures, or historical information.
The library of examples in the project gives the novice ready access to
a repertoire of experiences, examples, or illustrations relevant to the
problem at hand. The expert has internalized this repertoire through experience.
3. Strategies and instructions. The project also provides strategies
in the form of step-by-step instructions along with expert advice and sample
solutions. The user follows these instructions to complete the project.
The strategies and instructions of the project guide the novice through
the solution of the problems inherent in each step of the task. The ability
to find effective solutions to common problems quickly is the hallmark of
expert problem solving.
Three Illustrations of Catalyst Projects
Three projects that meet these criteria have been created and are currently
being tested at Project Zero: SongSmith (Walters, Meyaard, &
Scripp, 1989); Just Enough Pascal (Walters & Morrison, 1988);
and Immigrant 1850 (Project Zero, 1990).
Catalyst Project in Music: SongSmith
The Project: To write a short poem (a limerick)
and set it to music by composing an original melody, duet part, bass line,
and chords. Working alone, an individual with moderate musical experience
can expect to spend 10 to 15 hours completing the project.
The Computer Tool: Deluxe Music Construction Set (Electronic
Arts), an inexpensive but powerful music editing program that can perform
the composition in up to four voices using a variety of synthesized timbres,
and can print the composition as sheet music.
The Library: Examples of good solutions illustrate each step in the
process of writing a song. Illustrations from experienced composers link
the task to music history. Using nontechnical language, the project also
makes connections with concepts in music theory.
The Instructions: The project guides the user through a step-by-step
process of writing a song. It begins with the task of creating the lyric
and setting it to an original melody. The project concludes with the creation
of harmony with a bass line.
The Song Smith project poses the central problems of music compositiontext
setting, voicing, contour, and harmony. It offers a chance for the beginner
to explore various solutions to these problems and structures the experience
in such a way as to guarantee a product (an original song) as an outcome.
Catalyst Project in Programming: Just Enough Pascal
The Project: To assemble the Pascal program "GridWalker"a
game in which small creatures intelligently find their way through mazesfrom
a set of Pascal procedures supplied with the project. An individual with
no Pascal programming experience can complete the project in about 25 hours.
The Computer Tool: THINK Pascal (Symantec), an integrated
system for writing, editing, and debugging compiled Pascal programs. The
debugging features of this development system pinpoint many of the the novice
programmer's errors. Special windows give the programmer valuable information
about the current state of the program and display the values assigned to
variables at any given point.
The Library: The kit includes all the necessary Pascal pieces to
assemble the complete program GridWalker. These program pieces are
also fully explained in the accompanying manual.
The Instructions: The Just Enough Pascal kit provides concise
instructions on how to assemble the pieces of Pascal code in the editor.
The kit also includes instructions
for "tinkering" with the program, making those changes to the
program that reveal important underlying concepts such as loops, variables,
conditionals, and events. It also exemplifies modular program design and
issues of user interface.
Just Enough Pascal gives the novice an insider's look at programming
through the process of building an elaborate program one step at a time.
Catalyst Project in Social Studies:
Immigrant 1850
The Project: To explore the experiences of the Irish immigrants to Boston
in 1850 by making the same decisions these immigrants made when they arrived
in the United States. After a brief introduction to the computer software,
an individual can complete the project in 5 hours.
The Computer Tool: Either Microsoft Works (Microsoft) or AppleWorks
(Apple). These integrated programs include modules for word processing,
database, and spreadsheet calculations.
The Library: Immigrant 1850 provides a complete list
of the actual passengers of four ships that arrived in Boston from Ireland
in 1850. It also provides information about available jobs (location, wages,
necessary experience), housing (location and rent), and transportation costs
of the time. A spreadsheet document displays a market basket of food and
clothing prices in 1850.
The Instructions: The project poses the problem of adopting one family
from the passenger list, and making several important decisions for that
familyfinding them a place to live, jobs, transportation, food. The users
record these decisions in the family diary (a word-processing document)
and they summarize the family budget on a spreadsheet.
The Immigrant project takes on many aspects of a simulation in a
game-like format. In working through the project, the user makes decisions
and reviews the results. The full impact of each decision can be assessed
by playing the game a second time and making particular decisions differently.
Immigrant also demonstrates how structured information accessed through
a powerful tool can be used to gather insights about an historical era.
The Immigrant project provides the individual with "raw"
or unstructured information about a period in history, and the individual
must organize that information to make decisions and to generate insights
about the period and the social group that is being studied.
These three Catalyst projects were designed to give novices firsthand experiences
with the materials and problems in three domains. At the end of each project
the novice has created a finished piecea song, a computer program, or a
diary. This product provides a "stake" for the student, the motivation
and context for following the instructions of the task. For example, in
SongSmith the discussions of musical notation, terms, and concepts
are presented in the context of an individual's own emerging song. In Just
Enough Pascal, the student creates a computer program, and in Immigrant
1850 a personal diary of experiences.
These three examples are different from the PROPEL projects in some important
ways: They are not designed to facilitate assessment; they are not designed
to fit seamlessly into the school curriculum; and they make explicit use
of computer technology. Nevertheless, the Catalyst projects offer important
features for our assessment projects by illustrating how projectssustained
activity on a single problem to produce an outcomecan tap into specific
intelligences and reveal how individuals approach the same task differently.
Designing Projects That Combine Technology and Assessment
As a partner in the Center for Technology in Education at Bank Street College,
Project Zero researchers are developing assessment instruments that make
the best possible use of technology. In our approach, technology functions
in two distinct ways. First, technology is used directly in solving the
problem at hand. For example, the music composition software actually performs
the composition as it is being written, allowing the composer to judge by
ear the quality of the work. Writers can use outlining software to study
and manipulate the structure of an essay and use word processing programs
to simplify the task of rewriting and editing a manuscript.
Technology operates in a second way in the assessment of student progress.
The computer "opens up" the process of problem solving, making
it more visible to both the problem solver and to the evaluator. For
instance, students can easily revise and save drafts as they complete them
for later review. When students work in small groups at the computer, their
decisions, made visible on the computer screen, can be easily observed by
both the members of the group and the teacher. In these ways, using the
computer to compose allows the student and the evaluator to study the process
of composition as well as the final product.
To develop assessment instruments that use technology as a facilitating
instrument and that build on the design principles drawn from the theory
of multiple intelligences, we are creating domain projects that embody features
taken from the projects in both PROPEL and Catalyst:
- Assessment is drawn from student performance as they complete projects
that focus on central issues in the curriculum.
- To complete a project, the student creates one or more original products.
- Assessment includes both interim and final evaluations of student products.
- Students evaluate their own work throughout the project and compare
these self-evaluations with the teacher's judgments.
- The computer is an indispensable tool which students learn to use effectively
during the project.
Domain projects designed to these specifications allow students to work
directly with the materials of the domain. These projects pose problems
that can be solved in a variety of different ways. They provide an opportunity
for students to develop a stake in the outcome of that problem solving.
Finally, projects designed in this way incorporate technology as a tool
that facilitates problem solving in the domain. The use of technology opens
up the process of completing the project to assessment.
A Prototype Project
The domain project Immigrant 1850 provides a starting point
for developing one such domain project. Like the other Catalyst projects,
Immigrant 1850 uses a powerful computer tool (AppleWorks)
to analyze information. The design of the assessment component is drawn
from the domain projects constructed for Arts PROPEL. Because we are presenting
this project as a prototype, we would like to describe it in a little more
detail.
In the Immigrant 1850 project, students work in small groups.
First, they examine the passenger list of a ship that arrived in Boston
in 1850. This list includes the name, age, occupation, and country of origin
of the passengers, all of whom list Ireland as their country of origin.
Students sort the list to divide the names into groups that might make up
individual families. For example, they find that the list contains several
adults with the same last name, but by using the age and occupation information,
they can make decisions about which individuals might plausibly be considered
a family. Each working group "adopts" one family and begins to
make a number of decisions from the point of view of this adopted family.
Students begin a first-person-account diary for the family, which describes
their voyage, their arrival in Boston, and the family's plans for the future.
Next, the students peruse other database documents. One document lists jobs
that were available in Boston during that period, including the location
of each job, the wage it pays, and the skills required. A second database
records typical housing, its location, cost per week, and a brief description
of the property. A third database covers the costs of commuting from one
location in the city to another. Using these three sources of information,
the students make the decisions that their adopted family had to make. The
students locate their selected jobs and housing on a map of the Boston area
and they calculate how much it would cost and how long it would take to
commute from home to work. Using a spreadsheet document, students "purchase"
food and dry goods supplies for their adopted family. As they select items
from the list, the spreadsheet calculates the total cost.
With all of this information in hand--family members and their ages, jobs
and salaries, the location and rent of housing, the cost of food and clothing,
the distance and charges for transportation--students begin to calculate
a yearly budget for their family. They enter all of the information they
have gathered into the budget spreadsheet, which then computes the results
of these expenditures over the course of one year. Students examine the
result, and if the family is losing money, they return to the other documents
to reconsider their decisions.
For example, the students may decide that the children in the family should
be sent to work in factories instead of going to school. Or they might send
older children to work as servants, because these jobs pay room and board
in addition to a small wage. The
parents in the family can decide to work six days a week instead of five.
The family may select less expensive housing. In this way, the students
adjust their decisions for the family so that income and expenses balance.
Throughout this process of getting their family settled, students enter
the decisions they make into their family diary. They also compare the decisions
they make for their family with the decisions of other working groups. For
instance, the class can plot where individual families decided to live on
a single map of Boston. This summary displays the distribution of this particular
group of Irish immigrants across the city, which can be compared with published
demographic maps of Boston of the period. Student also compare the experiences
and decisions of different families. For instance, some families will have
many children to support, others may have a single parent (many immigrants
died at sea), and some families will consist only of young adults. The decisions
and experiences of these different families will themselves be very different.
This core activity--using the various tools in AppleWorks to make decisions
for an adopted family--is supplemented with several other activities. In
one, students review newspapers written for the Irish immigrants during
this period and use these as models for writing their own newspapers. In
another activity, students produce a skit depicting some aspect of the Irish
experience. In a third activity, students compare the cost of feeding a
family of four in 1850 with the cost of making the same purchases in 1990.
Student Products
Immigrant 1850 supports the production of a range of products
in a variety of formats. To illustrate, we will consider three written products
that students create. First, they keep track of their decisions by writing
a diary from the point of view of one member of their adopted family. To
complete this diary, students supplement the factual information they are
given in the various databases with historically realistic but fictional
details. In writing this diary, students exercise their "historical
imagination."
Students write a second essay as an historian or sociologist of the period.
The questions that guide this piece may require additional research. For
example, as historians, students would be asked to explain why so many Irish
immigrants came to Boston at this particular time, and how their arrival
changed the city. As sociologists, students could be asked to explain why
the immigrants lived where they did, and to speculate on what impact adding
a trolley system to the city might have, or how prejudice toward the immigrants
was manifested.
In a third writing assignment, students consider questions about the unit
itself. For example, they reflect on how they made decisions, how effectively
their group functioned, and what they would do differently if they were
to work through the project a second time. They consider what information
is missing from the simulation and how the project would change if that
information was added. Or, they discuss how an immigrant's experience today
compares with that of the Irish immigrants 140 years ago.
Techniques for Assessment
Three different techniques are brought to bear in assessing this student
work: (1) evaluation of the three written pieces; (2) assessment of group
work using a checklist; and (3) review of students' self-assessment activities.
Written Work
As teachers read the written work, they offer several specific points for
improvement. Students redraft the assignment taking these points into account,
and the new draft is graded. Checklists of writing performance used in Arts
Propel tasks will be adapted to this portion of the assessment. As in Arts
Propel writing assignments, students create a portfolio of drafts that lead
to the completion of the finished piece, which they use when reflecting
on changes in their writing.
Checklist on Group Activities
In a second assessment technique, the teacher evaluates how the various
teams of students function as a group to accomplish the assigned tasks.
To facilitate this, the teacher uses a detailed checklist to record pertinent
incidents and interactions. The aspects of group work that are listed on
this checklist include interpersonal dynamics, group efficiency and creativity
in problem solving, flexibility of roles, and responsiveness to suggestions.
This checklist is important for several reasons. First, it attempts to capture
the interpersonal dynamics of the project, which may otherwise remain undetected
because students rarely comment spontaneously
on such features of their work in their own writing. Second, the checklist
gives the teacher the opportunity to step back from the class and observe
classroom interactions in a structured and purposeful manner. The checklist
also provides the teacher a chance to observe the work of students who may
not write well or speak up in class frequently.
Finally, the checklists offer yet another dimension to the grading processhere
the teacher can take stock of the classroom atmosphere, how people are working
as groups rather than individuals, and the differences in working style
among these groups. Since group work is a large part of the project, it
is important that assessment procedures take it into account. The results
of the checklist will be shared with students as groups and individually
when appropriate.
Student Reflection
Students' reflections on the project take two forms. They evaluate their
own work (both process and product), and they comment on the project itselfits
function, limitations, and assumptions. In both instances, reflection is
linked directly to what students produce over the course of the project.
To structure the process of self-assessment, students create a portfolio
of their drafts of the essays they write as they work through the project,
along with their teacher's comments and suggestions. At the end of the project,
they review all of the materials collected in the portfolio. They compare
what they accomplished on this project with those of previous projects;
or they contrast their observations and reflections with those of other
students.
At the conclusion of the project, students critique the materials of the
project itself, including the relationship of specific activities of the
project to general questions in the subject area. Students can consider
the nature of the simulation, its limitations and implications, and its
relationship to history.
The goal of this reflective component is to get students to play a significant
part in their own education by casting a critical eye toward their accomplishments
and toward the project itself. To do that, they must think about what they
are doing, why they are doing it, and how good the results are when compared
with their capabilities and expectations. As these reflections become more
precise, they can be used to guide students' work in later stages of the
project.
Analysis of the Assessment Techniques
As a tool for assessment, the Immigrant 1850 project is designed
with the theory of multiple intelligences in mind. In the project, students
work with the materials of social studiesthe historical records and documents
of the period being studied. Furthermore, students can approach the project
from a number of different perspectives and engage the variety of intelligences
in different combinations. They can mobilize the logical-mathematical intelligence
to analyze quantitative data or to select from among competing sources of
evidence. They can use the linguistic intelligence to fashion poignant written
documents, the interpersonal intelligence to collaborate with their peers,
and the spatial intelligence in locating neighborhoods on a map.
Although the theory of multiple intelligences informs the design of this
assessment project, it does not speak to the issue of the actual assessment
itself. How do we distinguish between an "adequate" solution of
the Immigrant problem from an "inadequate" one? To consider this
question, we will follow Frederiksen and Collins' analysis of assessment
by considering three important features of assessment: directness,
scope, and transparency.
First, as an assessment instrument, the domain project blends learning and
evaluation and it targets directly the skills and concepts
that the teacher wants the students to learn. For example, in a social studies
class, students are evaluated on their ability to gather and collate information,
to analyze and interpret the results, to get help when necessary, and to
write a description of what they have uncovered. These are precisely the
tasks they must perform to complete the project.
Because the skills of interest are evaluated directly, the domain project
approach to assessment stands in sharp contrast to the traditional test
situation. With direct assessment, both teacher and student focus on the
skills themselves; in contrast, with tests the teacher must infer the presence
of desired skills from students' performance on tests that measure those
skills only indirectly. When a student answers a multiple-choice question
correctly, the teacher must infer from that response that the student has
some understanding of the concepts that provide a basis for the correct
answer. Furthermore, as Frederiksen and Collins
point out, indirect tests of skills often lead students to concentrate on
the skills that are directly related to taking the test rather than on the
skills that can only be inferred from the tests.
Second, the domain project assessment is more complete in scope than
the traditional test in that it attempts to capture most of what is required
to complete the desired activity. Interpersonal skills are manifested in
the group interactions, mathematical and spatial abilities appear in the
tasks of manipulating data and working with maps, and performance styles
can be observed throughout the project.
Of course, different students will display very different profiles of performance
along the dimensions represented by these different skills. The challenge
to the assessment procedure is to document the breadth of these skills in
useful detail. For example, the checklist of group activities directs the
teacher to many skills that might easily be overlooked in the written components
of the project. Similar instruments are yet to be devised for observing
and recording other skills that are represented in the project.
In contrast, a traditional test must select a representative sample of skills
for evaluation. From this sample, the teacher then infers the degree to
which the untested skills have been mastered. Furthermore, certain important
skillssuch as planning, negotiating, collaborating, revising, reflecting,
or perseveringare rarely represented at all on multiple-choice tests.
Finally, the terms of assessment must be defined explicitly for both the
teacher and the students. This makes the assessment transparent.
In the Immigrant domain project, for example, all criteria of the assessment
are shared with the students when the project begins. As the students complete
portions of the project, they use these criteria to gauge their own work.
They can even use the group process report card to evaluate the performance
of their group.
Practice with Assessment
Because the domain project is direct in method and broad in scope, it allows
students to "study for the test" or to practice with the features
of assessment that permeate the project from beginning to end. Furthermore,
if several domain projects are given throughout the year, an individual's
accomplishments in one project can serve as signposts that direct learning
on later projects. We hope that as they complete a series of these projects,
students will gain confidence and
experience with the demands of assessment. And because these assessments
tap the very skills that the teacher is interested in, this practice with
assessment becomes part of the educational process itself. Ultimately, the
goal of domain projects is to teach students to assess their own work efficiently
and to use this self-assessment to improve their performance on similar
tasks in the future.
Although adults might find reflection in a domain project to be straightforward,
high school students probably will not. Therefore, the teacher must model
the process by posing typical questions and then offering answers. As the
students attempt to assess their own work, the teacher can review these
first attempts and make suggestions where appropriate. Again, students will
develop sophisticated skills of self-assessment only through repeated practice
with feedback.
Concluding Remarks
Our goal in developing domain projects for assessment is to create a realistic
model of an alternative to the traditional test-based approach to measuring
competency or achievement. We will be examining this model in the classroom
next year. We will attempt to determine if the assessment is practical and
reliable and if it yields information about students efficiently with a
reasonable amount of teacher effort. We will also consider the subjective
nature of teacher judgments. We must determine which assessments capture
insights and aspects of student performance that may have been overlooked
otherwise and to what degree teachers agree with one another in these judgments.
The domain projects we are developing are not designed as large-scale replacements
for what currently transpires in the classroom. Instead, they serve as models
of how one might develop strategies for assessment that meet the criteria
we have identified and can coordinate with existing curricula.
We hope that these domain projects will move the discussion of assessment
away from tests, measurement, and large-group comparisons toward a consideration
of careful documentation of student learning, self-assessment, and growth.
References
Collins, A. & Frederiksen, J. (1989). A systems approach to educational
testing. Educational Researcher, 18, 27-32.
Project Zero. (1990). Immigrant 1850. Cambridge, MA: Harvard Graduate
School of Education.
Garnder,H.(1983). Frames of Mind. New York: Basic Books.
Walters,J.(1988). Just Enough Pascal.Bedford, MA: Symatec.
Walters,J., Meyaard, J., & Scripp, L. (1989). SongSmith: Songwriting
on the Macintosh. Cambridge, MA: Walters Associates
[ Home | About CCT | Projects | Newsletters | Reports | Staff | Links | EDC Home ]
Last Update: 11/18/96
Comments on the CCT Web site: Webspinner.
©1996 Education Development Center, Inc. All Rights Reserved.