The Academic Achievement and Accountability Commission's Recommendations on Accountability; Citizens' Minority Report

To the House and Senate Education Committees

January, 2001

Introduction

The Accountability Commission, in their current recommendations, includes this statement.

Based on comments received during the public forums, on public response forms, and on other information learned during the process, the commission recommends that a series of actions be taken by the Legislature, the Governor and the Superintendent of Public Instruction.  (A+ Accountability Commission, November 15, 2000, p. 8)

This is not true.  During the entire history of education reform in Washington State, the education establishment has used a process called the Delphi Technique.  This process can include a series of controlled meetings led by trained facilitators, usually employees of the establishment responsible for implementation of the recommendations.  Their role is to lead those in attendance to agree to their predetermined agenda.  The final report always refers to the public forums and insists that the final recommendations be based on everyone's comments.  This is done to give the illusion of legitimacy to their recommendations.  Yet, the testimony, oral and written, of those who disagree with any part of the predetermined agenda are not recorded in the final report.

Therefore, Citizens United for Responsible Education (CURE), in response to the fulfillment of the intent of Education Reform Legislation and increased student learning, insists that a minority report shall be filed.  The purpose of this minority report is to give a summary of common comments made throughout the state at public forums and in written response, that were either not included in the Academic Accountability and Achievement (A+) Commission Recommendations or disagree with their recommendations.  By so doing it will help validate the first principle adopted by the A+ Commission,

Commission collaboration, integrity and public service: The commission will conduct its work in the spirit of collaboration with the highest standards of integrity and public service. (p. 9)

Issue 1:  The WASL will be the sole source for recognition for improvement.  During presentations, the A+ Commission also listed Norm Referenced Tests as another source for recognition of student improvement.  This was important to educators and parents throughout that the state, opposed to using the WASL as the sole source to determine student improvement.  Unfortunately, the Commission ignored the people.  "Recognition for Improvement" (p.17) recommends the WASL as the sole source used to determine who receives recognition for improvement.  Authentic accountability demands that the tool used must be proven to be valid and reliable.  The WASL has not been proven to be either, as defined by both the CSL and science.  The first goal of the commission should have been to demand an authentic assessment.  They failed.  Therefore, this responsibility falls to the legislature.  The WASL must be proven scientifically valid and reliable.  All other accountability goals are meaningless as long as the WASL remains flawed.

The WASL is not valid.  It is revised every year at all levels.  It does not measure consistent results nor are the circumstances similar.  Its scoring rubrics are adjusted to the children’s responses each year and measure different standards each year.  Yet assessment scores are compared as if they are equivalent.  Presently, the fourth-grade math assessment is being revised, in recognition of the problems with that section.

The WASL is not developmentally appropriate.  It demands formal logic at all grade levels for many questions.  Since this ability does not develop until after elementary age, these questions are more appropriate for more mature students.  The majority of students have been set up for failure.  The work of Dr. Catherine Taylor, University of Washington, paid through special grants to oversee the WASL, is the only opinion recognized by SPI.  Independent analyses of the WASL must be studied, including those of Professor Donald Orlich, of Washington State University, and Dr. Doug Carnine, of Oregon State University.  Prof. Orlich has predicted that only about 40% of the students will pass.  Dr. Doug Carnine has found that the 4th grade math assessments regularly exceed the developmental ability of students.  Necessary computational knowledge is not being tested.  Findings from both the OSPI study of the Grade 4 Mathematics Assessment and the Northwest Regional Laboratory support the analyses of both Professor Orlich and Dr. Carnine.

No authentic proof of student performance based on the WASL is available.  The only scored results (the assumed proof of performance-based assessment) are scanned into each school district’s computer database by National Computer Systems (NCS).  The scored assessments themselves are not returned for teacher use.  The legislature, in fulfillment of the intent and mandate of ESHB 1209, must demand that all scored WASL’s be returned to Washington State for students, parents, and teachers to use in determining areas of student deficiency and study.

The scoring is not reliable.  The recent inaccurate scoring of the Writing assessments is not a fluke.  It speaks clearly to the subjectivity and lack of reliability in this type of assessment.  To achieve a rating of .84 to .92, each essay must be read by up to five highly trained scorers.  In a high-stakes assessment, nothing less than 100% accuracy in scoring is acceptable.  Washington employs only one scorer, with minimum training, to skim each assessment in as little as three minutes, an average of 2 1/2 minutes for writing essays and 20 seconds per math essay.  An average of almost one out of every four assessments (24.3%) that were evaluated by two scorers had scores that were discrepant by one or more points.  The discrepancies were ignored.  In addition, scorers are not required to meet the standards set for Washington State.  Dr. Catherine Taylor, U of W, working with SPI Bergeson’s office, under a grant, wrote:

Qualifications of Readers

Highly qualified, experienced readers (scorers) were essential to achieving and maintaining consistency and reliability when scoring student-constructed (open-ended) responses.  Readers selected for the Washington Assessments were required to have the following qualifications:

(Washington Assessment of Student Learning, Grade 4, 1998, Technical Report, p. 4-1)

National Computer Systems scorers in Iowa are:

Other states have had to regularly address the same performance assessment scoring inaccuracies that have occurred in Washington State.  In addition, the entire issue of the effectiveness of performance-based education, under any name, is questionable.  The legislature should research the findings in other districts and states regarding performance-based education and scoring accuracy.  The State of Kentucky is mentioned often.  Washington State reform is based on Kentucky's reform.  Since 1993, much has been written about the continuing failure of Kentucky's reform and testing.  The legislature must demand nothing less than 100% accuracy in assessment scoring.

Reporting is not valid, reliable, or fair.  SPI, without legislative rule, has approved a set of testing accommodations for teacher-selected students.  They are used extensively in schools throughout the state for both the standardized tests and the WASL assessment.  These accommodations are not reported.  Reliability issues demand that complete information be included in all measurement reports including how many were accommodated and exactly what the accommodations are.  SPI allows teacher-selected students be excluded from both testing and reporting.  It is as though they do not exist.  Their numbers are not included in the Learning Improvement Index.  State law mandates that all students must pass the assessments in order to earn a Certificate of Mastery (CM).  The legislature must determine what is to be done with students who are being denied the opportunity to receive the CM.

The WASL is not yet mandated for students.  Students whose parents choose to opt their children out of the WASL have received erroneous reports stating their children have not met the standards, with no other explanation.  National Computer Systems, using assigned student-identification numbers, has scanned all the scores into the students’ permanent electronic portfolios.  These electronic portfolios are not kept at the local schools.  Instead, they are part of a district database, soon to be fed into the K-20, Washington Virtual Education Space (WAVES).  Students and parents are punished for making a legal choice.  Assessment results have also been returned to the wrong students.  As an example, one tenth-grade student who did not participate in any part of the 2000 assessment received three different scores.  His parents had requested that he be excused (recognized as absent.)  Instead, the student received three other scores — an incomplete for his math score, refusal for his writing score, and numerical scores in both reading and listening.  These numerical scores suggest that he received someone else’s reading and listening assessment scores, someone who took the assessment.  That unknown person received either refusal or incomplete for his score.

NCS pulls the assessment booklets apart to score the reading and listening.  Then, they have made mistakes putting the results back together.  Parents who noticed discrepancies, such as the one described above, have questioned this to no avail.  Districts either blame NCS or refuse to give any explanation.  SPI is unresponsive.  Parents are not allowed by SPI to see the assessment, scoring, or scanning.  They have no way of determining whose scores (their child’s or another child’s) have been scanned into their child’s permanent electronic portfolio, used to track their progress over the years.  Discrepancies and lack of response to parental concerns are reported throughout the state.  Students’ entire futures are reliant on accurate scanning of scores into their permanent electronic portfolios.  The legislature must investigate these mistakes and develop steps to insure against future mistakes, as well as develop parental options for protecting their children.

The assessment is hostile to children.  The ambiguities of many questions set children up for failure and undue stress.  There are continued stories of cheating.  The practice of teaching to a narrow assessment robs children of a well-rounded education.  More affluent schools are able to offer Saturday classes during the weeks prior to taking the assessment.  However, providing this same opportunity is not possible all schools.  Equal access to assistance for all students becomes an issue.  The WASL is prejudiced against Hispanic, Native American, and African American students, as well as boys.  Dr. Robert Rayborn maintains that the 4th Grade WASL is .54 dependent on IQ and Reading ability.  SPI published the Study of the Grade 4 Mathematics Assessment - Final Report; and revealed:

Northwest Regional Education Laboratory (Portland, Oregon) found that:

The WASL is neither fiscally nor educationally cost effective.  Millions of dollars have already been spent writing the EALR’s, determining the standards, writing the assessments, and paying for scoring.  It is apparent that these costs will continue to grow as other assessments are added and reliable scoring sought.  Along with this, children are spending untold hours being taught the assessment, and then taking it.  The learning loss cannot be justified.  Recognition for improvement should be made by using other data including the standardized achievement tests that have a track record of reliability and validity.  The commission (p. 17) disagrees.  They recommend only the WASL for this purpose.

For good reasons, the Seattle Education Association, based on professional judgment, has rejected this assessment.  It is imperative that the legislature address this resolution.

Reform legislation requires SPI to provide an objective analysis.  The analysis must:

16.  Require students to use computational skills without the use of technology and

17.  Consult with recognized experts with differing views on the instruction of mathematics. (cf. SSB 5418. Chapter 388, Sect. 601.)

ESHB 1209 requires:

The assessment shall be designed so that the results under the assessment system are used by educators as tools to evaluate instructional practices, and to initiate appropriate educational support for students who have not learned the essential academic learning requirements at the appropriate periods in the students’ educational development.

Issue 2:  "General Assistance" (pp.15-16) and "C: Funds and Capacity Needed for Focused Assistance" (pp.26-27) continue to support the mantra of the educational establishment, a plea for more and more money and resources in order to implement education reform.  Facilitators at all forums called for more of both.  The commission, without hesitation, question, or condition, dutifully recommended that all schools and districts receive even more than what has been received in the past.  Yet, citizens throughout the state question both the necessity and effectiveness of these requests.  Overworked teachers are continually forced to attend in-service, accept changing curricula, and use mandated methods, whether they increase learning or not.  Local decision-making is being compromised.  Since the early 1980’s, the State Legislature and the Federal Government have poured money and resources into Washington in support of education reform.  Grants and pilots, including Schools for the 21st Century, teacher mentor programs, SLIG’s, and Chapter I Schools, are but a few of the many experiments that use the general assistance strategies the Commission recommends be increased.  It is time for the legislature to demand that SPI define words such as "skilled" and "specialized expertise", while including a series of requirements, for these skilled and specialized experts, based on validated research.

Determining how much additional time for those engaged in developing curriculum and strategies for students and for those assisting and training other professionals within their district is a local responsibility.  This falls to the district and each teacher.  It is inappropriate to expend resources, including teacher time, for development and training in unproven strategies and instruction programs.  Before devoting scarce resources to expand and implement this system, the Legislature must demand scientific evidence of its effectiveness in producing an academic education that will best serve the children living in the state.  Until this time, these funds would be better spent providing for classroom needs or reducing the tax burden on the citizens.

Issue 3:  "D: Statutory Changes Required for the Intervention Strategies"  The Accountability Commission wants the legislature to repeal or change many laws.  The legislature must carefully evaluate these for the protections they provide.

Conclusion

Washington education reform legislation demands that all adults in this state be held responsible for improvement of student learning.  It is imperative that all voices be heard and all recommendations be received equally.  This minority report serves that purpose.  It is based on three issues and a series of recommendations, presented by citizens throughout the state, but ignored by the A+ Commission.  Comments are based on the intent of State Reform Legislation:

o        Increased student learning

o        Involvement of the entire community

Therefore, this report is submitted to the Senate and House Education Committees for equal consideration, when forming future educational policy.

Citizens United for Responsible Education gratefully acknowledges the work of Joanne McCann, of Spokane, Washington, for compiling this report.

╪