Abstract (pp. 1-2 only; For entire paper, including footnotes and source material, request by
e-mail at: dennis_delong@yahoo.com)

The Assessment of Learning Outcomes: Needless Burden or Fruitful Opportunity?

[PAPER DELIVERED TO NYS POLITICAL SCIENCE CONFERENCE, SUNY NEW PALTZ  APRIL 2001]

Dennis R. DeLong, Ph.D, Assistant VP Academic Affairs/Dean, School of Graduate and Continuing Education
formerly, Dean, University College, Chapman University and
formerly, Deputy Chief Academic Officer, Connecticut State University System

 One of the primary issues facing all of higher education is accountability - the desire to ascertain the actual impact of a university or college education on individuals and society. It is, some would argue, the logical result of increasing concerns for quality. Increased concern for accountability also derives from growing concerns regarding the cost of higher education in an era of apparent declining public support. Other motivating forces which have contributed to a heightened salience for accountability are the growing awareness of workforce needs, greater customer demands for high quality educational services, increased competition and the infusion of technology into instruction which has caused more systematic attention to learning objectives.

1. INHERENT DIFFICULTIES

Despite the coalescence of pressures for accountability in higher education it is a singularly difficult process to undertake, especially when applied to the products of learning, usually labeled "learning outcomes." The adoption of performance assessment of learning for the purpose of enhancing accountability is considered in many quarters to be antithetical to the culture of higher education. It also is conceptually difficult and problematic.

  1. Cultural Barriers - Constraints impeding the acceptance of the systematic assessment of learning outcomes derive from the nature of academia at both the individual faculty and institutional levels.
Faculty Resistance – many faculty, especially in non-professional, liberal arts and sciences fields where it is not as common, feel the assessment of learning is unwise or inappropriate. Some faculty feel it is unwise, because it threatens to wrest control of academic content from Individual faculty members. Others feel it is inappropriate because it either attempts to measure matters which are not measurable or, in focusing on phenomena which are measurable, misses the subtle nature of a faculty member’s input in a teaching relationship. College teaching traditionally has been a "private contract" between faculty and students, protected by academic freedom and tenure; assessment threatens to open up this relationship to public scrutiny.

 Institutional/Program Resistance – academic entities resist the adoption of assessment, especially if it is imposed by higher levels of administration or pushed hard by external bodies, to avoid the loss of autonomy and to forestall faculty concerns regarding academic freedom. Resistance also stems from the believe that assessment will take much time and produce results which are not worth the effort extended to attain them. The possibility of destructive comparisons among seemingly similar institutions as a result of assessment activity is feared in particular.

B. Fundamental Measurement Dilemmas – similar to any measurement challenge, the assessment of learning outcomes confronts several fundamental dilemmas:

Definitional - Learning outcomes can be defined as the tangible products of the courses and programs offered by higher education institutions, i.e., the impact of the learning activities on the learner. As in any "cause and effect" scenario, however, one must determine the prior state before attributing change to an action. A focus on learning outcomes should acknowledge that learning is a "value added" concept which involves altering the knowledge or skills level of a learner by calculated actions.

Level of Analysis - the assessment of learning outcomes can be applied to an individual’s performance on a particular course element (exam or paper) or in a course (as in the case of assigning a letter grade or percentage or raw numerical score to symbolize performance on an element or in a course), in an academic program, major or entire college experience, i.e., a series of courses (as in the grade point average or grade average which summarizes performance for an individual across several courses). Assuming there is an acceptable mechanism to assess individual performance, scores then can be aggregated to assess performance by: course, program, academic major, division, school, college/university or system.

Type of Measurement - as in any measurement situation, the operationalization of a factor for which qualitative distinctions on a dimension are posited should reflect the nature of the factor itself. Except where the results can be standardized, for instance, the scores assigned to questions on a exam or qualitative differentiations based on observations or examinations of case analyses or problem solving activities, the measures at best can be treated as a ordinal scale. Aggregations of scores derived for ordinal distinctions must respect the limitations inherent in such measures. In addition, the disparity between the score assigned to a ordinal category and the actual meaning which can be attributed to the differentiation is magnified by the number of categories used, i.e., a dichotomy such as "Satisfactory" and "Unsatisfactory" is less problematic than 3, 4 or 5 item scales although these offer richer analytical possibilities.

Learning Objectives or Standards – the assessment of course or program performance requires the establishment of performance standards if the results of applying measurement instruments are to be meaningful. Accumulated knowledge ought to be compared against the core content of a course or courses, for example. An assessment of writing skills ought to draw on an array of expectations ranging from Excellent to Poor.

Performance Target or Base Line – instruments which purport to measure performance in accumulating knowledge or personal skills produce no inherent value unless compared to some sort of target or benchmark. A target can be drawn from past measurement (base line), a national or peer norm in the case of a standardized test, or a predetermined outcome which is a best estimate of performance expectations.