MDiv Education and Numbering the Levites

Presentation at the Conference on the Character and Assessment of Learning for Religious Vocation November 2002 by Daniel Aleshire, The Association of Theological Schools

ABSTRACT: The educational assessment movement has both external and internal aspects. This article provides an overview of educational assessment and the challenges it presents to theological education, the problems that the assessment of theological learning present to theological schools, and the current project of ATS on the Character and Assessment of Learning for Religious Vocation. The author outlines both the internal and external factors influencing this movement, principles for assessing the goals of the theological curriculum (with particular attention to the Master of Divinity degree program), and the Association's efforts to assist member schools in developing and refining their educational assessment practices.

Introduction

We are in the midst of an educational sea change. It is massive, pervasive, and gathering momentum. It has been going on for a quarter century, and the result is that a new issue is thoroughly and unmistakably on the table. It is not an educational fad or whimsey. It reflects changing cultural perceptions about education and the increasing expectation of observable results for educational dollars spent. "It" is educational outcomes assessment. This is my thirteenth year at ATS, and the outcomes assessment movement has ceased being a movement. It has established itself as a fixture in higher education, as it has in elementary and secondary education. It will not go away, and it cannot be avoided by theological schools if they wish to be a part of the ecology of higher education.

This paper addresses three issues related to the ATS project on the Character and Assessment of Learning for Religious Vocation: (1) educational assessment and the problems it presents for theological education; (2) theological education and the problems it presents for assessment; and (3) the work of ATS over the past decade, especially the ATS Character and Assessment of Learning project, related to the assessment of learning in theological schools.

I want to begin with a text in Numbers. It seems appropriate to use a text for a paper about assessment in theological education from a book in the Hebrew Bible that is called "In the Wilderness." The book begins with the Lord telling Moses to "Take a census of the whole congregation of the Israelites, in their clans, by their ancestral houses. . . ." The text proceeds by telling how the numbering would be conducted for each tribe. Then we get to the Levites, and the Lord tells Moses "only the tribe of Levi you shall not enroll, and you shall not take a census of them with the other Israelites." The Levites, whose job was to attend to the tabernacle of the covenant were to be left unnumbered. The census, it appears, was for determining military strength, and the Levites received a ministerial exemption. Even apart from military service, there is something about this exemption from numbering that theological education would like to claim. We are happy to have engineers assessed. (We would like to think they know how to design bridges that don't fall down as we cross them.) We would generally agree to numbering surgeons to determine if they can cut well and sew things back together correctly when they are finished. (We probably wouldn't even mind if student surgeons who failed the assessment were barred from operating on people). We would generally support numbering student pilots to make sure they know how to take off and to land, as well as to perform other tasks in between. We are less certain, however, about numbering the Levites. We are not confident that assessment is a good or desirable thing in theological education, particularly quantitative assessment. While there may be value in numbering other professional clans and tribes, our tribe is different. The idea of a ministerial exemption is attractive.

This is the post-modern era, or at least not the old-modern-as-it-used-to-be era, and it is appropriate for me to begin with some self-description so that the reader can determine the hermeneutical lens through which my words should be interpreted. My Ph.D. is in psychology, and the program in which I earned it was very empirically oriented. It required a significant number of graduate hours in research design and statistics, and I spent three years between my tenure as a pastor and professor as a research scientist at the Search Institute in Minneapolis, where all our work was statistical. My first semester in graduate school exposed me, in more than one class, to the short version of psychologist E.L. Thorndike's conclusion: "If something exists, it exists in some amount. If it exists in some amount, it can be measured."

I have, on the one hand, been intrigued my entire career about how things can be measured and the kind of understanding and insight that can accrue from appropriate analyses of statistical data. On the other hand, I do not think that all that is worth knowing can be known statistically. John Harris, a thoughtful assessment theorist, cites a quote that reportedly hung on the wall in Albert Einstein's Princeton office: "Not everything that counts can be counted, and not everything that can be counted counts." I commend Einstein's opinion. So, I bring a friendly but critical stance to the topic before us. I am ready neither to dismiss the knowledge that can be derived by empirical methods nor am I ready to assume that knowledge should be limited to what can be verified empirically.

The assessment movement and its problems for theological education

Like any social or higher education phenomenon, the assessment movement is the result of the confluence of many external and internal factors, and its story can be told in many ways. I will provide my perspective on it as a theological educator, observing the movement from the world of higher education accreditation.

External factors that influence this movement. I believe that the outcomes assessment movement has been heavily influenced by cultural factors outside of education. It is deeply rooted in an industrial, consumerist way of thinking and has been promoted more by external political forces than internal educational values.

Industrial images. In the 2002 gubernatorial race in Pennsylvania, the funding of education was a hot issue. The Democratic candidate's position was that education has to be less dependent on property taxes, and while not increasing other taxes, this candidate wanted to improve the "educational product" offered in Pennsylvania. It strikes me as odd that he thinks of education as producing a "product." Product is an industrial term. Factories make products like cars and widgets. It is less clear to me that schools make educational products. However, if they do, then one can determine whether or not the product is good, just as a well-made car can be distinguished from a poorly made car. I do not think people were talking about educational "products" thirty-five years ago when I graduated from high school. They may have talked about the quality of education, even good schools, but industrial language was not the normative discourse for education. Thinking about education as a product changes educational thinking.

Consumerist perspectives. If education needs students and products need consumers, then the more education becomes a product, the more students become consumers. In the case of schools, constituents also become consumers, and funders become investors. Consumers look to US News and World Report to find a best-value educational product. Investors want data that indicate what the return on their investment has been. ATS occasionally receives calls from persons asking if the Association ranks theological schools. Often, the real question is, in effect, which schools are the best values. The callers seem perplexed when we tell them there is no such ranking or value rating in theological education. These questions are new to the discourse of education. Every decade or so I buy a car, and when I do, I am a consumer concerned about product quality and value. I bought a grey Camry in 1990, and another grey Camry in 2001. I may be a boring consumer, but I am a consumer. In North America, we have imported a perspective very appropriate for widgets and cars into the world of education, where it may be less appropriate.

Political demands. The major influence in the assessment movement has been political. If American students do not perform as well on a test as Japanese students do, it is a political problem more than an educational one. When more money is needed for education, the problem is a political one more than an educational one. The politics that influence education demand observable results. Politicians want to be able to demonstrate value for tax dollars expended. The learning of education doesn't necessarily want these same things, at least in these same ways. Perhaps the most problematic aspect of the politics of education is the passion for simple criteria to determine educational success-like results on a test.My daughter completed two years of service in the Teach for America program last year and decided to stay in the same, very under-funded, school for a third year. It is a middle school, where she teaches art and one session of remedial reading. The eighth graders have to pass the state reading test or they cannot go on to high school. She has eighteen students in the remedial reading class, three of whom read at the third-grade level and all the rest at lower levels. Her job in that class is to prepare them to pass the test. These children have been failed by society in almost every way a society can fail its children. Their education has been severely under-funded, their parents have been under-employed, their teachers have been under-paid. Now, they will take a test, and if they under-perform, they fail. Students cannot move beyond middle school until they pass the test, and most will drop out of school before they pass it. The state legislature will talk nobly about standards and accountability. Meanwhile, a test becomes a prison that locks these children out of the future that many of them-with a well-funded education and eight years of instruction-could have entered. Richard Elmore, in a recent article in Harvard Magazine, worries that the most recent federal legislation is "now accelerating the worst trend of the current accountability movement: that performance-based accountability has come to mean testing alone." Elmore goes on to note that "relying only on standardized tests dodges the complicated questions of what tests actually measure and of how schools and students react when tests are the sole yardstick of performance."1 My daughter will do her best with her students, but she has not had a single course on teaching children how to read. She is the only hope the children have of passing the test. Her energy, imagination, and care for them will be enough for some. But the children who fail, or the school that fails, will take the rap for the society and the politicians who, in my judgment, have failed.

The assessment movement in higher education

Thus far, I have reflected on the assessment movement in education in general. The movement began with elementary and secondary education, and its greatest impact, in terms of social policies and social consequences, has been in elementary and secondary education. For the past twenty years, however, the movement has been descending on higher education, and once again, the primary impetus has been external.

Since the 1980s, and greatly intensified in the United States by the 1992 reauthorization of the Higher Education Act, accrediting agencies have been required to demonstrate that the institutions they accredit are assessing "educational effectiveness." In the ensuing reauthorizations of this bill, the regulation of accrediting agency standards on assessment has grown increasingly precise, demanding, and inescapable-which is exactly what Congress has wanted. Failure by ATS to address educational effectiveness will lead to withdrawal of recognition by the U.S. Secretary of Education, and without that recognition, ATS accreditation does not provide the eligibility needed for seminaries to participate in the federally guaranteed student loan program. The participation of theological students in this loan program is increasing. In 1991, fewer than half of the students in ATS schools graduated with seminary educational debt. In 2001, more than half of the graduates had debt. Of the students who graduated with seminary educational debt, the amount of debt more than doubled between 1991 and 2001. The external, federal influence, although indirect, is strong and getting stronger in theological education in the United States.

Internal influences A perceptive reader can detect that I do not warm up to the external factors that have pushed the assessment movement forward. I think these factors are philosophically flawed, educationally flawed, psychometrically flawed, and social-policy flawed. However, these flaws do not mean that assessment is a flawed concept. Elementary and secondary teachers have a long and good history of assessing their students, and the results have helped teachers learn more effective strategies for teaching. Assessment is a value in education. My critique is perhaps better identified with the "accountability movement"-an externally imposed effort that asks the wrong questions for the wrong reasons. I want to affirm the value of the "assessment movement" as an internally driven work of educators to improve student learning and increase the professional capacity of teachers. There are internal reasons for assessing theological learning, and I find them as important to affirm as I find the external factors for "accountability" reason for worry.

Assessing student learning is a long-standing practice. I was a professor for twelve years in a seminary prior to joining the ATS staff. In every course I taught, I devoted considerable effort to assessment. There were tests, papers, projects, and other activities in each course that, I hoped, would facilitate student learning and provide some assessment of that learning. The faculty was sufficiently concerned about assessment that the last week of each semester-and we only had fifteen weeks-was devoted exclusively to assessment. We called it final exam week. Combined with the other tests or assessment-oriented class sessions, my students probably averaged two weeks out of every semester-almost fifteen percent of their total educational "contact" time-in assessment. No one was making me do it. There was no institutional requirement that it be done. So why did my colleagues and I do it? The reasons were internal, not external, and these reasons are instructive for ATS accrediting expectations about assessment.

Internal reasons for assessing student learning. Consider the following as possible explanations for my tendency to assess student learning: (1) I cared about the subjects I taught and engaged in assessment as a way of encouraging students to learn what I thought was important for them to know. I may have been wrong, and no doubt, the subjects I taught were less central to ministerial capacity than I thought. Ministry should be an informed profession, if not a learned one, and educational assessment supports the process of learning. (2) I value critical thinking and would like to think that I could be self-critical about my work as a teacher. If the students never learned any of the things I thought they should know, then it seemed to me that, as a self-critical teacher, I would ask if there were something about the course design or my work as a teacher that impeded, rather than facilitated, learning. In this sense, assessment is a form of intellectual self-discipline, a willingness to stand aside from my work critically, in much the same way that I wanted students to stand aside from their ideas or others' ideas in order to make critical assessments. (3) I cared about the people I would never meet who would be served by my students after they graduated. I wanted students who could be good ministers for my mother and my children, and others' mothers and children. So, as an act of care for the people I did not know, I was interested in whether or not students were learning the sensitivities and skills, along with the knowledge, that good ministry requires.

I will admit that I spent most semesters just happy to make it through to exam week, let alone thinking that any good would ever come of it. I failed often as a teacher, including giving many tests that were ineffective assessment instruments. Most days I was too busy to function thoughtfully. But, when I was reflective about the work, I did wonder about what I was doing and if I were doing it as effectively as it should be done. These are reasons for educational assessment that are internal to our work as teachers, that grow out of the care and values we bring to our work. For me, the internal factors for assessment are compelling.

As students have become more diverse, as information has become more prolific, as ministry has become more complex, as religion in North America has become more stressed, theological educators need to become more skilled at assessing the attainment of educational goals for the sake of communities of faith and the faith that shapes those communities.

The problem of assessment for theological education. Frankly, I want theological schools to become more committed to, more intentional with, and more skillful about outcomes assessment. I do not think the Levites should get an exemption. I think their work is every bit as crucial as the work of any surgeon, any engineer, or any pilot. Because the work that graduates of ATS schools do is important work, it is our responsibility as theological educators to make sure they know what is necessary to do this work well. In the end, theological schools do not have a choice about developing more intentional and skillful practices of educational assessment. Our choice is either to learn to do this kind of educational work grumpily because it has been externally mandated, or to learn how to do it faithfully because we care about the work our graduates do and the communities they serve.

Theological education and the problems it presents for assessment

Assessing the outcomes of education is complex work, but it follows a simple form of logic: the activity of teaching has results, and if education has been conducted effectively, the educational results should be evident. So, if my daughter educates her special class of eighth graders effectively, they will pass the eighth-grade reading proficiency test. If they pass the test, we will assume they can read and that, with no training and very few resources, my daughter did her job well. Education leads to results, and assessment is the simple task of determining what kind and how much of those results have been attained.

Consider this logic imposed upon the work of a theological faculty member. The professor teaches students about the love, justice, and mercy of God. Because education has results, learning about the love, justice, and mercy of God should have result. But what is it? Is it that the student will be able to say "God is a God of love, justice, and mercy?" Or that the student will know the theological meaning of these categories and the limitations with which we can claim any knowledge of God at all? Or is it that students will order their lives by this vision of God? Or is the preferred result that students will be able to help other people perceive of God in this way and order their lives by such perceptions?

Identifying outcomes to be assessed
The first problem with outcomes assessment in theological education is the outcomes, not the assessment. Theological faculties are not always certain what they want theological education to accomplish. In the 1980s, ATS and Lilly Endowment engaged in a variety of activities to discern the aims and purposes of theological education. The Basic Issues Research, as the effort was known, gave considerable attention to the goals of theological education, and it resulted in a small library of articles and monographs on the subject. Between 1992 and 1996, ATS conducted a major study to redevelop the ATS standards of accreditation, and considerable attention was given to this research. The statement on the theological curriculum in the ATS accrediting standards is a paragraph with a shelf full of books behind it. This is how it reads:

4.1.1. In a theological school, the over-arching goal is the development of theological understanding, that is, aptitude for theological reflection and wisdom pertaining to responsible life in faith. Comprehended in this over-arching goal are others such as deepening spiritual awareness, growing in moral sensibility and character, gaining an intellectual grasp of the tradition of a faith community, and acquiring the abilities requisite to the exercise of ministry in that community. These goals, and the processes and practices leading to their attainment, are normally intimately interwoven and should not be separated from one another.

The article I cited earlier by Richard Elmore argues that improving student learning requires that "internal accountability. . . precede external accountability. That is, school personnel must share a coherent, explicit set of norms and expectations about what a good school looks like before they can use signals from the outside to improve student learning." ATS, in its effort to identify the core of the theological curriculum and to adopt an accreditation standard about the curriculum, took at least the first step in saying what good theological education should do. The accrediting standards implement this general statement about the curriculum through specific statements for each degree program offered in ATS accredited schools. For the MDiv, the educational goals include: "knowledge of the religious heritage; understanding of the cultural context; growth in spiritual depth and moral integrity; and the capacity for ministerial and public leadership."

Given these educational goals for the MDiv program, the proper answer to the question posed earlier about the "learning" professors might hope would result from teaching students about the love, justice, and mercy of God is "all of the above." The goals for the MDiv require students to learn the theological affirmations and limitations related to a God of love, justice, and mercy (intellectual grasp of the tradition), that they order their lives by this vision of God (deepening spiritual awareness), and will be able to help other people perceive God in this way (acquiring the abilities requisite to the exercise of ministry).

While schools are free to adapt and implement these goals in ways that are appropriate to their educational and ecclesial contexts, they are not free to ignore these goals. This lack of freedom is externally imposed by ATS only in the sense that the schools, as a community, adopted these standards as an act of mutual accountability.

The Association has a clear and comprehensive statement about what theological degree programs should accomplish educationally. That leaves us, however, with the second and more vexing question, how do we assess what we hope is accomplished? That is the problem that theological education poses for the assessment movement: How do we assess what we most care about achieving in MDiv education?

Principles for assessing the goals of the theological curriculum
The more I have worked in this area and observed schools struggling with assessment, the more I have concluded that the outcomes that theological educators most want to be attained are the most difficult to assess, and the goals that are more secondary are the easiest to assess. We can assess, for example, the ability to match thirty-five dates to the dates of birth of thirty-five historical figures. While I do not want to minimize church history, matching dates to people may not be as central to the work of ministry as the ability pastorally to help grieving parents deal with the death of their child. The second ability is much more difficult to assess than the first.

There are several general principles that schools need to keep in mind as they wander into the complex world of assessing the attainment of goals in the MDiv program. I want to share a short list with you.

There is no one way. Unlike the narrow way the Gospels talk about, the road that leads to good assessment is a wide one. Good assessment uses many indicators in many ways to arrive at nuanced judgments about educational effects.

Numbers and quantitative information can be very helpful forms of assessment. You can determine if someone gets thirty or ninety answers right on a 100-question Bible content exam. This can be very helpful, and my statistical background leads me to think that quantitative assessment can be very revealing and informative. But, numbers cannot tell a school if the right answers to these 100 questions truly reflect mastery of biblical content, nor can numbers tell what an acceptable score should be. Quantitative information can be helpful, but it has always been the case that numbers do not make decisions; people do. In fact, one of my worries about the assessment movement is the tendency to think that numbers can make decisions. That is like thinking that a stop sign should stop your car. A driver will do better to step on the brake. Stop signs indicate an appropriate action; they do not take the action.

Qualitative methods of assessment are as good as, sometimes better than, quantitative methods. "Assessment" is not synonymous with "quantitative," even though there is increasing pressure on accrediting agencies to value numerical assessment over qualitative assessment. This would be tragic in theological education. At the heart of ministerial work are practices that are more like art than technical skill. While the performing arts have always been evaluated, they have seldom been evaluated by numbers. Theological education needs efforts of assessment that are useful, truthful, and qualitative, and not necessarily metric.

The goals of theological education need to be carefully distinguished. I remember a faculty lounge conversation from a few decades ago. One of our seminary's graduates had just devastated a congregation with unprofessional behavior and sexual misconduct. One faculty member expressed surprise because the graduate was such a fine theologian. His statement assumed that if graduates know the right or good, they will do what they know, but, as we discerned in that faculty lounge conversation, bad professional performance or sexual misconduct is not necessarily related to inadequate knowledge of theology or Bible content. Something else is at work. The educational goal to know theology differs from the educational goal to learn responsible professional conduct. Effective assessment requires a careful delineation of the differences in educational goals.

Different forms of assessment are needed for different educational goals. Abraham Kaplan, an influential philosopher of science, wrote about the law of the hammer: "Give a child a hammer and suddenly everything needs pounding." Because assessment is difficult, we have sometimes wanted to use one form of assessment to assess everything. It is as if we had a good scale for measuring weight, and then used it to measure height and hair color. We need to appreciate the multiple ways of knowing that vocational religious work requires, and the multiple forms of assessment these multiple ways of knowing require.

Assessment specialists can help us with our efforts, but they do not have our answers as theological educators. We do not have to invent everything, but not everything that we need has been invented. We need to pay careful attention to what has been done in other professions and in other theological schools, but we will need to develop skill in doing this work ourselves.

Educational assessment does not end with the assessment of student learning

It also requires analyses of educational programs. The ATS degree program standards expect educational evaluation in two areas. The first is the one I have discussed the most, assessing student learning, but the second is also important-assessing educational programs.

The standards read:

A.5.1 The institution offering the MDiv shall be able to demonstrate the extent to which students have met the various goals of the degree program.

A.5.2 The institution shall also maintain an ongoing evaluation by which it determines the extent to which the degree program is meeting the needs of students and the institution's overall goals for the program, including measures such as the percentage of students who complete the program and the percentage of graduates who find placement appropriate to their vocational intentions.

It is altogether possible that a student could graduate knowing everything she was taught, and because she was taught all the wrong things, not function thoughtfully or well in ministry. So, as if assessing educational effectiveness were not enough, the standards add the expectation that a school evaluate its degree programs to determine how they are succeeding in meeting their goals.

Evaluating the effectiveness of a degree program points to one of the greatest difficulties in this work. Theological faculties seem to be comfortable with assessment as an individual activity: An individual faculty member develops a test for the content in his or her course, and individual students take the test, which forms the basis for assigning individual grades. Our discomfort rises when faculty members, as a whole, must decide on the educational goals of the degree program and develop corporate evaluation practices that help faculty make decisions about the outcomes achieved by all the students. Each point in this corporate evaluation is riddled with more difficulty than all the points combined in individual assessment.

ATS efforts and the Character and Assessment of Learning for Religious Vocation Project

The assessment expectations in the ATS standards are unavoidable. We can argue about their theological correctness, about their implausibility, or about the philosophical flaws in the assessment movement. These arguments will not make the standard go away, and the Commission on Accrediting cannot reaffirm a school's accreditation on the basis of the elegance of their theological argument against assessment. The task of ATS as an association of schools is to reach deeply into the things we care about in theological education, find the internal accountability that will guide efforts to develop appropriate and effective means of assessment, and use our intellectual and institutional skills to become good at this work. ATS seeks to serve the schools as well as to accredit them, and so we have been engaged in work for a decade of which this project is the culmination.

Redeveloped accrediting standards
A decade ago, ATS began the difficult task of rethinking its accrediting standards. This was a comprehensive review and redevelopment of accrediting standards, perhaps the most comprehensive review that ATS has ever undertaken. The result is an articulation of the goals of theological degree programs and an articulation of what ATS means by evaluation, educational assessment, and degree program evaluation. As a result, we have agreed-upon expectations about educational goals and educational evaluation.

Pilot School Project
Following the adoption of the redeveloped accrediting standards, ATS invited eight of the schools that were among the first to be reviewed according to the redeveloped standards to participate in the Pilot School Project. This effort provided guidance by consultants about the development and implementation of models of educational assessment. The eight schools developed models of evaluation and wrote case studies describing their work that were published in Theological Education in spring 2000 and presented at the 2000 ATS Biennial Meeting. These schools have been trying to implement systems of assessment for several years now, and they know about the problems of starting a comprehensive assessment program and the even greater problems associated with continuing it over time.

Increasing accountability to the standards
The Commission on Accrediting has taken a phased approach to the implementation of the full expectations of the educational assessment elements in the standards. This is a new educational practice, and ATS schools are acquiring the in-house expertise that colleges and universities also are developing. For the past four years, the Commission has been emphasizing the level of expectation in this area. Most of the schools participating in this Character and Assessment of Learning project were chosen because they are preparing for accrediting visits in 2004, 2005, and 2006. They will be among the last schools to be evaluated by the standards that were adopted in 1996. ATS has both enlisted the schools' help in this current project and intends to help them attain the knowledge necessary to meet the full expectations of the standards.

The Character and Assessment Project
The Character and Assessment of Learning for Religious Vocation project is a four-to-five year effort, funded by Lilly Endowment, that has several goals, including two central ones: (1) increased understanding of the character of theological learning and (2) increased skill in assessing the attainment of learning for religious vocation. In addition, the hope is that this project will provide better understanding about the relationship between the goals of learning and some characteristics of the students who are the learners, and the development of resources that will both help the schools participating in the project and help other schools in the Association.

The project uses the term "learning for religious vocation" as a means of focusing on the learning that theological educators most care about and the learning that the communities of faith most need their leaders to achieve. "Vocation" is not a reference to office or role, but to the deep ordering of ministerial and priestly work. This is the kind of learning that we want to learn how to assess, because this is the learning, or lack thereof, that can have the largest impact in communities of faith. The project entails two major areas of work: research and institutional implementation of effective assessment strategies.

Research projects
The project includes five research projects. (1) The first one was completed by Charles Wood and Gordon Smith and is reported in this issue of Theological Education. ATS schools were surveyed in an open-ended manner to identify what they are currently doing about assessment. Some interesting skills have been developing, although the report shows there is still much work to do. (2) Carolyn Jurkowitz, in the second study, reviewed the rather massive amount of literature on assessment in higher education. The report of her research is contained within this issue as well. (3) A third study, conducted by Victor Klimoski, examined the ways in which other helping professions, such as social work, education, and nursing, are addressing the issue of assessment. His report is published herein. (4) The fourth study is the most complex and is being coordinated by Katarina Schuth and Gary Peluso-Verdend. They, with their colleagues, are interviewing individual MDiv students and groups of faculty to get at the deeper and substantive issues that influence learning for religious vocation. This study is seeking to identify the ways in which characteristics of students contribute to or detract from the educational goals of the MDiv degree program. The results of this study will be available in 2004. (5) A fifth study will involve an assessment of the literature on ministerial performance of graduates of ATS schools. It will explore some denominational studies of clergy and provide a second-order analysis of the findings of those and other research projects that can inform the evaluation of current seminary students. This fifth project is not yet under way but will be conducted in 2003. Together, this research will make a range of information available to the participating schools to help them invent, implement, or reinvent their efforts to assess student learning and degree program effectiveness.

Participating schools
In addition to the research components, the project is involving forty schools that are, or soon will be, preparing for a comprehensive ATS accreditation evaluation. The primary role of schools as participants in this project is anchored to focused work on their self-studies. As part of that work, schools will carefully evaluate the educational goals of their degree programs in the context of the ATS accrediting standards and develop and implement systems for assessing student learning.

As a result of this project, a significant number of ATS schools will have thought about the outcomes they most want to be the results of their educational efforts, ways of assessing the attainment of these outcomes or educational goals, and ways of assessing the overall effectiveness of their school's degree programs. As a peer organization, ATS plans to share the learning of the schools participating in this project through publication in this journal of their case studies and in other venues, and thereby increase the amount of useful information that is available to ATS member institutions. As an accrediting agency, ATS will seek to use its accrediting process both to motivate and to help schools to identify goals that are at the heart of theological education and to assess them with skill and sensitivity.

Conclusion

Identifying the goals that theological educators most want their students to attain and developing strategies to assess learning are critically important for theological schools. The importance is not vested so much in the external expectations of educational institutions as it is in the schools' internal commitments to educate students well. Educating students well is important not so much because educators value good work that is done well, as it is that the quality of education of seminary students has a direct bearing on communities of faith and their work in the world. In the end, for theological schools, assessing educational attainment is an act of stewardship because so much that is good is at stake.

Daniel O. Aleshire is executive director of The Association of Theological Schools in the United States and Canada.