Assessment Consortia: When, whether, and how to consort …San Diego, CAJune 23, 2015
Scott E. Smith, Ph.D.Kansas State Department of Education
1
“To the size of states[inserthereconsortia] there is a limit, as there is to other things, plants, animals, implements; for none of these retain their natural power when they are too large or too small, but they either wholly lose their nature, or are spoiled.”Aristotle, Book 7 ofPoliticsIn light of Aristotle’s assertion, we might ask, “How does a state ensure that a consortium entered into retains power, efficiency,and effectiveness.”
What’s to be said of size and scale as it applies consortia?
Chapter16:“Towardsa Theory of Large-ScaleOrganization”Nobody really likes large-scale organization; nobody likes to take orders from a superior who takes orders from a superior who takes orders…Even if the rules devised by bureaucracy are outstandingly humane, nobody likes to be ruled by rules, that is to say, by people whose answer to every complaint is: "I did not make the rules: I am merely applying them."
Small is Beautifulby E.F. Schumacher
Kansas over the last five years has moved from (1) being a member of a large, national assessment consortia, to (2) working alone,and finally to (3) consorting with a single state, Alaska. Of thethree arrangements,Kansas has found the most value in working with a single state. The advantages fall generally into four categories:(1) Communication(2) Assessment development decision-making(3) Legal/Organizational matters(4) Ownership
There are advantagestokeeping things small …
Reaching consensus on assessment development that can be used in Kansas and Alaska has been relatively simple.Dailycommunication on critical issuesin Kansas and Alaska has occurredwith simple email rather than withWebEx, conference calls, or Doodle Polls.Conversations withKansas StateBoard of Education regarding shared costs, joint assessment development agreements, and future planning have been well received and easily explained.
(1) Communication: There is a difference betweenreaching consensusandmaking concessions…
Shared contentstandards across Kansas and Alaskahave allowedCETE to fully measure state-specific contentstandards … i.e., surf content and turf content.Kansas has access to an interim assessment that it otherwise would not have been able to develop on its own …CETE has demonstrated the meeting the needs of two states is possible.
(2) Test Development
Efficiencydoes exist as Kansas does shareitems withAlaskaCustomizationexists as PLDsand cut scores aredeveloped just forKansasScorereports is a combination of shared + customized.
Test Development continued …
Consorting with only Alaska rather than with multiple other stateshas eased challenges of drafting data-sharingagreementsTest-security concernsthatmight exist with larger consortia areeasedTwo-state MOU and contract allows for targeted efficiencies and targeted assessment development
(3) Legal-Test Security Challenges
Chapter 18: Ownership“Whenwe move from small-scale to medium-scale, the connection between ownership and work already becomes attenuated; private enterprise tends to becomeimpersonal …”“Inlarge-scale enterprise, private ownership is a fiction for the purpose of enabling functionless owners to live parasitically on the labor of others. It is not only unjust but also an irrational element which distorts all relationships within the enterprise.”
(4) Assessment Ownership (and more Schumacher)
TheKSDE and KSBOE“own” the state assessment as per the authority vested in them through state law. Ownership should not bedeferred, and Kansas has been able to preserve a sense of ownership of its assessments.Local control in Kansas has been preserved and recognized in particular with the Kansas Assessment Advisory Council (36 Districts) working closely with CETE on test design issues.Educational stakeholders in Kansas want to see a Kansas assessment that measures Kansas standards and are willing to invest time and expertise along with the vendor to create a desirable “product” … hence, pride of ownership.
A vendor working with a small number of states is able to balance (1) the desires of a single state with (2) the reality of a common blueprint.CETErecognizes the importance of ensuring that the input of Kansas educators is reflected in their “product.”
Ownership continued …
0
Embed
Upload