lurie

Student Learning and Effective Teaching

Learning:
Learning Goals

Introduction

Core Component 3a: The organization’s goals for student learning outcomes are clearly stated for each educational program and make effective assessment possible.

In this section, once again we begin by framing this section of the report within the context of a highly decentralized academic environment. The University offers programs and majors as diverse as Social Computing, Movement Science, Chemical Engineering, Periodontics and Oral Medicine, Finance, Educational Studies, Judaic Studies, Romance Languages and Literatures, Biology, Neuroscience, String Instruments, Environmental Justice, Midwifery, Public Policy, Pharmaceutical Sciences, and Social Work. In view of this academic diversity and richness, for academic programs to identify learning goals and how to measure whether their students meet these goals is an activity that differs greatly from one unit to the next.

An important part of the University’s success lies in the commitment to intellectual freedom that the nineteen schools and colleges enjoy (twelve of those involve undergraduates). This freedom can exist only within the context of a strong administrative infrastructure that ensures that all University units meet the highest standards of professionalism and institutional integrity. In keeping with this culture, the academic units were queried about the ways in which they articulate their academic missions, how they expect their students to grow, how they use the evidence about student learning that they gather, and the processes by which they make curricular innovations and change. These questions were purposely phrased to allow for answers that reflect the many differences across academic discipline.

The Office of the Provost asked the deans of the schools and colleges to respond to, or where applicable, to have each of their departments respond to, five key questions about student learning, which are provided below:

  1. What processes are in place for faculty members to discuss whether the curriculum is designed to promote students’ growth and learning, and to make revisions to the curriculum and program requirements?
  2. What are the key ways you expect your students to have grown by the time they complete the program (your unit’s educational goals)? Such measures of growth could include skills acquired, attributes developed, experiences, and “products” that could include course work, artistic work, and theses.
  3. How is the curriculum/program designed and structured for your students to meet these goals?
  4. In what ways does the unit measure whether students achieve your stated goals? Beyond grades and graduation rates, other ways can include portfolios, projects, theses, and capstone experiences.
  5. How do you communicate your goals (or mission) to students and other constituents, and the ways in which the curriculum/program is designed to help students achieve them, both on and off campus?

The full set of responses to these questions, which varied widely, as anticipated, is available in the 2008 report, Units’ Assessments of Student Learning. As a result of the way in which we asked these questions, the schools and colleges’ responses are the farthest thing from “one size fits all.” Rather, the responses show a wide array of goals, values, ideas, and approaches that are a rich representation of the University’s diverse and well-regarded academic enterprise. Additional analysis of the responses is included in the supporting report.

Schools, Colleges and Departments

The few examples below touch on a few of the ways the academic units articulate their learning goals for students, as described for the entire campus in the Units’ Assessments of Student Learning report that was produced in support of our reaccreditation process.

College of Engineering

In its Bulletin for Undergraduate Education, the College of Engineering describes its undergraduate education mission, its undergraduate educational objectives, and its undergraduate education outcomes. The mission of the undergraduate degree programs of the College of Engineering is to prepare graduates to begin a lifetime of technical and professional creativity and leadership in their chosen fields. The college’s Undergraduate Educational Objectives are below:

  • Prepare students for professional creativity and leadership in their chosen fields of study.
  • Providing students with a comprehensive education that includes in-depth instruction in their chosen fields of study.
  • Emphasizing analysis and problem-solving, exposure to open-ended problems, and design studies.
  • Fostering teamwork, communication skills, and individual professionalism, including ethics and environmental awareness.
  • Providing adequate co-curricular opportunities that cultivate lifelong learning skills.

For undergraduate educational outcomes, the objectives state that graduates of the College of Engineering’s undergraduate programs will have:

  • An ability to apply knowledge of mathematics, science, and engineering within their chosen field.
  • An ability to formulate engineering problems and develop practical solutions.
  • An initial ability to design products and processes applicable to their chosen field.
  • An ability to design, conduct, and interpret the results of engineering experiments.
  • An ability to work effectively in diverse teams and provide leadership to teams and organizations.
  • An ability for effective oral, graphic, and written communication.
  • A broad education necessary to understand the impact of engineering decisions in a global/social/economic/ environmental context.
  • An understanding of professional and ethical responsibility.
  • A recognition of the need for and an ability to engage in life-long learning.
  • A broad education necessary to contribute effectively beyond their professional careers.
  • A sense of responsibility to make a contribution to society.

Department of Asian Languages and Cultures, LSA

The Department of Asian Languages and Cultures in LSA expects their students to learn about the regions and disciplines covered by the department; to develop skills in Asian languages, along with an appreciation of the place of languages in definitions of cultures; to acquire a critical understanding of the assumptions and methodologies that underlie the study of Asia; and to progress toward the larger goals of liberal studies in the humanities, which include fostering interest in and understanding of other cultures, assisting in the development of critical thinking, and training students to read and write with nuance.

Department of History of Art, LSA

At the curricular level, the undergraduate program in LSA’s History of Art program is designed to provide students with:

  • A broad knowledge of the world’s visual cultures and artistic traditions.
  • An understanding of the connections and interrelationships among the visual practices of diverse cultures.
  • A full comprehension of the ways in which artworks function in their own socio-historical contexts.
  • A highly developed capacity to employ the skills—including critical reading, sustained visual analysis, advanced independent research, and persuasive writing—necessary to understand and communicate the ways in which artworks, images, objects, and built environments “speak” to their audiences, past and present.

As students advance through the concentration or minor, they learn to deploy a wide variety of interpretive methods devised to approach visual artifacts as elaborate acts of communication. They also learn the importance of face-to-face encounters with original objects, of close formal analysis, and of the complex relationships between visual and verbal forms of representation. In a larger sense, the students gain a thorough understanding of the discipline of art history and a heightened level of visual literacy that extends across diverse cultural and historical traditions.

Organizational Studies Program, LSA

In the process of developing their program, the faculty in LSA’s Organizational Studies Program paid particular attention to designing the program’s goals to promote “intellectual exit competencies” through a variety of different means. These goals are described below.

  • Theoretical knowledge and skills. To be able to draw upon theoretical concepts and methods from a range of social science disciplines to interpret organizational phenomena.
  • Engagement in planning student course of study. To strengthen student ability to understand and articulate their planned course of study through the curriculum and how it relates to their career and/or continuing educational aspirations.
  • Research Skills. To develop research skills, including 1] problem analysis and solution generation, and 2] the ability to pose theoretically informed questions and use research methodologies to analyze organizational questions.
  • Participate in a variety of learning opportunities. To create learning opportunities outside of the classroom and to recognize that peers can be a learning resource.
  • Writing and presentation skills. To develop strong writing and formal presentation skills to communicate in both academic and nonacademic settings.
  • Teamwork Skills. To develop skills required to work in diverse teams and be comfortable in both leadership and subordinate team roles.

School of Public Health

The School of Public Health articulates its learning goals for students and each of the school’s five departments through a set of student competencies. Each of the departments has articulated competencies specific to its professional discipline. In addition, the school has articulated competencies in the core areas of public health that any Master’s of Public Health graduate should possess. Based on these competencies, measures of growth include skills in that discipline and in the broader area of public health, recognition of the role of professionalism in that discipline and in broader public health, and exposure to interdisciplinary learning. Products include course work and capstone experiences. Indeed, the capstone experience varies by discipline to best fit the needs of that discipline but include critical reflective papers, theses, or work that reflects students’ growth in combining all the individual skills and pieces of knowledge acquired.

Learning Goals of Units

Taking a broader look at the learning goals that the academic programs include in their responses to the five questions listed above, we sorted these goals into four general categories: knowledge and understanding, competencies, experience or engagement, and character.

Within each of these categories, goals can be found that often appear on lists of learning goals or objectives. For knowledge, common goals include knowledge, theory, independent thinking, the ability to think synergistically, and understanding the value of diversity. In the category of competencies, these goals include critical thinking; team building; identifying, formulating, and solving problems; leadership; and creative thinking. With regard to experience and engagement, common goals include the ability to both create and apply knowledge, relationship building and networking, and practical experience. In the fourth category of character, we find such common goals as ethical behavior, tolerance, and discipline.

Perhaps even more interesting are the goals or learning objectives that the academic units articulate beyond the generic. In the category of knowledge, these goals include the ability to read with nuance, the ability to do “close reading,” intellectual and academic breadth, and a deep understanding of the world. In the category of competencies, some of the more unique goals include scientific reasoning; change dynamics; the ability to resolve complexity and uncertainty; and passionate, innovative, and flexible leadership. For experience and engagement, such goals include contextual awareness, cultural engagement, professional identity, exposure to a range of possibilities, and the ability to derive maximum satisfaction and fulfillment from future activities. Finally, with respect to the category we define as character, we find these goals: the ability to express the uniqueness of one’s vision with clarity and insight, a well educated sense of citizenship, becoming a sophisticated consumer, and strength of character.

There are many ways in which academic units at the University assess whether students have met the academic goals their departments, schools or colleges have set for them. Below are a few responses from individual units that illustrate this range, edited for brevity. Full descriptions can be found in the Unit’s Assessments of Student Learning report.

School of Social Work

The School of Social Work uses a number of strategies to assess each objective in the Master’s of Social Work program. The faculty also frequently makes special efforts to assess emerging needs and to evaluate innovations within the school. The faculty collects process as well as outcome data and uses both types of data to plan and implement program changes.

The school routinely gathers data through direct assessment methods of student learning (e.g., pre and post measures of competencies, use of students portfolios in Alternative Reaffirmation Proposal implementation, field learning contracts, and instructor graded course assignments) and indirect assessment measures (e.g., end-of-semester evaluations, focus groups, Foundation Year Measure, Second Year Competency Exit Measures, and alumni surveys). The Office of the Associate Dean for Educational Programs administers the assessment tools and analyzes the data. Standing committees for action and continuous program improvement (including the curriculum committee, the executive committee, and the community advisory board) review these reports.

In recent years, the faculty has made numerous changes in assessment strategies. First they re-examined and recast the curricular objectives and competencies for foundation and advanced concentration courses. As a result of this process, the faculty then changed and added items to the school’s ongoing assessment instruments to better determine whether students are meeting the stated objectives. In addition, the curriculum committee, working with curriculum workgroups, has begun to set up levels of proficiency for each competency. If a majority of students report that they are unable to meet minimal levels of competency, the curriculum committee shares this information with the relevant curricular workgroups for them to take action.

Through the Alternative Reaffirmation Plan proposal, the faculty is in the process of testing students’ use of portfolios to assess the degree to which their learning is integrative. Although assessment isn’t the primary force behind the school’s interest in portfolios, they could provide a comprehensive measure of how well students meet the school’s learning outcomes. The school is also participating in a pilot project across a number of units at the University to test and further develop electronic, portfolio-based learning software to help facilitate student learning and curricular assessment in ways that integrate curricular, co-curricular, and professional practicum experiences.

Through a range of assessment strategies, the school demonstrates its commitment to evaluating students systematically and to increasing confidence in the data by using multiple sources of reporting. The school also provides a table with the foundation and advanced educational objectives for the Master’s of Social Work, which also describes the data used to evaluate each objective.

School of Natural Resources and Environment

The School of Natural Resources and Environment (SNRE) tracks completion of degree requirements for Masters and Ph.D. students. Students in landscape architecture also prepare a formal portfolio of their work for use in employment applications. Although employment is a rough measure of program quality, the school initiated an annual employment survey of the most recent graduates, which they conduct six months after graduation. For the last two graduating classes, 90% of respondents said they were employed at the time of the survey. The school also monitors the progress of its Ph.D. students in comparison to a timeline by which it would like students to complete the preliminary examination, defend the dissertation proposal, achieve candidacy, and complete the degree. The school sometimes penalizes students who don’t reach mileposts (e.g., by naming them ineligible for Graduate Student Instructor positions or, in the extreme, expelling students from the program).

For certain segments of the school, assessments are put into place through the process of review by an external accreditation entity, such as the American Society of Landscape Architects and the Society of American Foresters. In addition, the SNRE Student Government has organized sessions with students to assess whether the courses had achieved their goals. The students condense the results into a report to the dean, which has been shared with faculty members who teach the courses. Each year, SNRE’s Office of Academic Programs administers the “Exit Survey of Graduating Students” to assess students’ satisfaction with the school’s academic program, climate, and career services. The director of the office compiles and summarizes results from the survey. Finally, SNRE maintains an active network of alumni working in the conservation and environment field. The school solicits feedback from this group on whether its learning goals are being achieved. Alumni have a perspective on their own education and on the training of recent graduates whom they either employ or view as prospective employees.

Department of Communication Studies, LSA

The Department of Communication Studies in LSA relies on a variety of informal indicators, including course evaluations and specific questions about students’ introduction to and grasp of key concepts in the field. As part of prospective students’ applications to be admitted to the concentration, the department asks students to write an essay about their prerequisite classes and what they learned. The department also sponsors a variety of shared forums in which faculty members evaluate and share information about the curriculum.

Each May the department holds a retreat during which faculty members discuss the undergraduate program. Also, smaller groups of faculty who teach the same course in alternate semesters or years meet periodically to discuss readings, curriculum, overlap between courses, and the like. Faculty members also discuss curricular issues as they arise. Many faculty members ask students about the curriculum during their advising hours and bring issues raised by students back to their faculty colleagues. Thus, the department seeks to continually assess and revisit how the curriculum is working.

The department’s various 400-level classes, nearly all of them seminars with a strong emphasis on analytical thinking and writing, help faculty members to see how far their students have come in their studies. For example, in these classes students are required to produce visual work, a series of seminar papers, or a final research paper. Such work can show faculty members whether the students have achieved the department’s goal of having them think about the media more critically and with increased oral and written sophistication.

Since 2004 the department has also begun more aggressive efforts to build ties between its alums, its current students, and the department. Events bring alums back to Michigan to talk to current students about how to find internships, network, apply for jobs, and discover what they would most like to use. During these events, the alums talk specifically about the ways in which the department’s curriculum connected to what they learned from the program and how it has served them on the job. These various events and activities have been very important to the department’s desire to drive home to its current students why the curriculum is designed the way it is.

Additional Approaches to Assessment

An overview of the academic units’ responses to the question about methods of assessment reveals again both a common set of approaches as well as less common methods (Units’ Assessments of Student Learning). Among typical ways to assess learning outcomes are written work, laboratory work (e.g., student lab reports), examinations (including practical exams), group work, skills tests, student projects, clinical experience, thesis, artistic and creative works (e.g., designs, recitals, performances, and exhibits), publications (e.g., in University and external publications), oral presentations (in a variety of settings), results on proficiency exams, degree audits or assessments, presentations, internships, community engagement or service learning experience, capstone experiences, qualifying exams, job placement information (including how long it takes students to find jobs), results of graduate and professional school examinations, admission to graduate or professional school, and alumni accomplishments and successes.

Other methods for assessing students by the University’s academic units include clinical simulations in health fields; student-taught seminars; active-learning pedagogical activities (e.g., one-minute papers, peer review of draft papers, and small group discussion); local and national awards and honors; student leadership positions (on and off campus); student-designated pathways for the concentration; enrollment in multiple fields; evidence that a student’s progression through the major has a clear shape and direction; progression through a building block sequence; the ability to link theory to models, processes, and practice; faculty reports on how prepared students are for upper level courses; external funding; broad analysis of a student cohort’s writing and research presentations; and evidence of clinical judgment.

Curricular and Assessment Efforts

In response to the question “What processes are in place for faculty to discuss whether the curriculum is designed to promote students’ growth and learning, and to make revisions to the curriculum and program requirements?,” the schools, colleges, and departments described many different types of mechanisms for faculty members to review and revise the curriculum, and whether they help students to grow and learn in ways that match the unit’s learning goals and objectives, as described below.

Faculty members discuss, review, and make decisions about the curriculum and student learning in regular or special all-faculty meetings, in groups with designated responsibility for discussing matters related to the curriculum, and in a variety of other types of settings and structures. Depending on the size of the unit, all governing members of the faculty are invited to participate in regular faculty meetings, typically with the dean or department chair. At times, special meetings of the faculty are called to focus on the curriculum. The purpose of such meetings is to review and make recommendations or decisions about existing core courses, new courses, program requirements, and the overall curriculum (e.g., integration among offerings). Groups with designated responsibility for curriculum include executive committees, curriculum committees, steering committees, education policy committees, ad hoc committees, and department faculty representative groups (for undergraduate and graduate education). In addition, some academic units hold periodic retreats for curriculum discussion. At times, units undertake an initiative to review and revise a major, concentration, or program. Most of the units have an associate dean for education in place (undergraduate and/or graduate), and many of the departments appoint an associate chair for education.

Deans, department chairs, and faculty committees often take advantage of CRLT services and expertise in evaluation and curriculum development when planning their curriculum discussions. CRLT consultants gather and analyze data (through surveys and focus groups with faculty, students, and alumni), consult about ways of assessing student learning, and serve as outside facilitators for discussions of curricular issues. Other mechanisms the academic units have used include curricular workshops, teaching innovation exchanges, selecting a core course coordinator (e.g., where faculty members teach similar courses), developing a curriculum management plan, appointing a course coordinator or “faculty of record,” mapping learning outcomes to courses, creating reports on the portion of students who successfully achieve the unit’s stated learning outcomes, asking instructors to submit end-of-term reports about their courses, creating shared teaching archives for faculty members to draw from, and designating a faculty teaching mentor (e.g., available to all faculty members but assigned to faculty members whose classes aren’t meeting departmental goals).

Other activities the units list include appointing faculty teaching mentors, conducting a needs analysis of departments and students across campus, creating electronic sites where faculty members can share teaching materials and discuss the curriculum, writing a unit-specific guide for new instructors, scheduling faculty member visits to the classrooms of graduate student instructors (GSIs), creating teaching circles for new GSIs, conducting an internal academic program review, offering teaching seminars, doing a survey of courses taught at other universities, involving students in the work of the curriculum committee, creating a curriculum across the disciplines, and seeking external funds for curriculum change or innovation.

Centralized, University-wide Assessment Efforts

In addition to the school, college, and department activities described above, the University engages in Universitywide assessment activities, several of which are described below.

National Survey of Student Engagement

The National Survey of Student Engagement (NSSE) collects information annually from samples of first-year and senior students about the nature and quality of their undergraduate experience. Since NSSE was first administered in 2000, approximately 1,200 baccalaureate degree-granting colleges and universities in the U.S. and Canada have used the instrument to measure the extent to which students engage in effective educational practices that are empirically linked with learning, personal development, and other desired outcomes such as student satisfaction, persistence, and graduation. Faculty, administrators, researchers, and others use NSSE data for institutional improvement, accountability, and related purposes.

The University of Michigan has participated in the NSSE survey in 2000, 2001, 2003, 2006 and 2009. Administered by the Office of Budget and Planning to freshmen and seniors, the survey gauges students’ experiences inside and outside the classroom. The NSSE data can be classified into five benchmark categories: (1) academic challenge, (2) active and collaborative learning, (3) student-faculty interaction, (4) enriching educational experience, and (5) supportive campus environment. In March 2006, the Office of the Provost distributed detailed NSSE results to the campus in the first issue of its publication M Know Blue. With this information, faculty and student affairs administrators examine what students gain and what organizational practices can be improved to enhance the undergraduate experience.

First Year Student Survey and College Senior Survey

The First Year Student Survey, which the University has a long history of administering, and the complementary College Senior Survey, which the University began administering in 2007 after more than a decade, are part of the Cooperative Institutional Research Program (CIRP), a national longitudinal study the American Council on Education began in 1966, which it now conducts jointly with the University of California, Los Angeles. Through the Division of Student Affairs, the University of Michigan participated in the first years of the First Year Student Survey (1966-1976), and resumed participation in 1993. Over the last fourteen years, the DSA has administered this survey to more than 60,000 entering University students. The items in this survey include students’ selfreported reasons for attending college and the University in particular, as well as student hopes and expectations for their educational experience. The CIRP survey results create a snapshot of new undergraduate students that informs key faculty and staff members about University students and their needs.

The First Year Student Survey serves the University in two main ways. First, it provides us with a picture of our new undergraduate students, including:

  • One year snapshots of entering student groups.
  • Trend analyses of entering student experiences and attitudes over the years.
  • Comparison of University of Michigan entering students to the nation and to public and private peer institutions, which tells us about the particular needs of our students and how they are alike and different.
  • Education of faculty, staff, and administrators about our student population through presentations to various groups and units.
  • Information provided to the colleges and schools about their newly admitted students.
  • Projection of future trends, needs, and traits of undergraduate (and graduate) students.

In addition, many programs, units, and staff actively seek out this information to plan in a responsive manner. Some examples are below:

  • The Career Center uses this information with other information about student destinations after graduation to learn more about the outcomes of their work.
  • The Ginsberg Center for Community Service and Learning adjusts offerings by understanding how many entering students have been previously involved in community service and learning experiences.
  • The International Center uses data about student study abroad experiences before college to inform their work with undergraduate study abroad programs.
  • The Office of Multi-Ethnic Student Affairs uses data about targeted populations to inform their planning and programming for particular student needs.
  • University Housing and the University Health Service uses information about student smoking and alcohol use before college to target health education programs and living arrangements.
  • Several areas with questions about first generation student needs and traits have been able to move forward with more targeted programs to provide a better university experience for these students (which translates into gains for all students).

For more information, see the website of the Office of Student Affairs Research in the Division of Student Affairs.

Graduating Senior and Alumni Surveys

In 2008 the University of Michigan joined other institutions in the Association of American Universities Data Exchange (AAUDE) in administering a survey of graduating seniors. AAUDE, of which the University of Michigan is an active member, is a public service organization consisting of members of the Association of American Universities (AAU) whose aim is to improve the quality and usability of information about higher education. AAUDE institutions exchange data and information that inform their decision-making, and issue data and reports on both public and confidential topics. The organization also meets at least once annually to discuss new developments and continuing priorities.

In the spring of 2008, the Office of Budget and Planning administered the Graduating Senior Survey to University students who were about to receive their baccalaureate degrees. In spring 2009 this was followed by a survey of alumni cohorts. These surveys included a core set of questions that participating AAUDE institutions agreed to (allowing for cross-institution comparisons) and additional items directly related to the University’s review for reaccreditation, with a focus on internationalization, involvement in research and creative works, and service and engagement.

Graduating Senior Survey

Below are two charts that show the responses of our students to questions about how well the University prepared them in select skill areas and how well the University prepared them to meet a select set of personal and professional situations. The full report, the Michigan Experience I: Perspectives from the Class of 2008, is posted. The results are a good example of the type of information that helps units at the University to assess their academic programs and the services and other opportunities they provide to our students. Parallels can be seen with a number of the goals for student learning discussed above.

When asked to evaluate how well the University prepared them for specific skills, seniors gave their highest scores to thinking logically and analytically, acquiring new skills and knowledge on their own, using the knowledge gained from their major field, and ability to judge the value of information (see below).

learnonown
How well did U-M prepare you to: (n=1535)

Similarly, when asked how well they were prepared for personal and professional situations, seniors gave high marks to feeling prepared for teamwork, getting along with people from diverse backgrounds, and decision making (see below).

situations
How well did U-M prepare you for situations that require you to: (n=1524)

Alumni Survey

The responses of alumni to the questions above are very similar to those of 2008 graduating seniors; the full report, the Michigan Experience II: Perspectives from Recent Alumni Cohorts, is posted. When students were asked about how well the University experience prepared them for their careers, the responses from two cohorts were overwhelmingly positive (figure below).

studprep
How well did University prepare for career: (n=2379)

The Collegiate Learning Assessment

In 2008 the Association of American Colleges and Universities (AAC&U) invited the University of Michigan to participate in a national project on assessing student learning outcomes, a substantial joint venture by the Association of American Colleges and Universities (AAC&U), the American Association of State Colleges and Universities (AASCU), and the National Association of State Universities and Land Grant Colleges (NASULGC; now called the Association of Public and Land-grant Universities). This project is supported in part by the U.S. Department of Education’s Fund for the Improvement of Postsecondary Education (FIPSE). The goals of the project are to build campus leadership and capacity to assess student learning outcomes and put the results to use.

As part of its participation in this project, during the 2007-08 calendar year the University administered a pilot of the Collegiate Learning Assessment (CLA), a standardized test intended to measure higher order skills such as critical thinking, analytic reasoning, problem solving, and written communication. The CLA was administered to 200 first term, first-year undergraduates and 200 undergraduate seniors in LSA. Among first year students who took the CLA test at 176 institutions, the group of LSA first year students scored in the 99th percentile (CLA score 1302). The LSA seniors scored in the 97th percentile (CLA score 1322). These results indicate that students who are admitted to the University of Michigan have already acquired the skills the CLA tests for. With entering first year students scoring so high, the CLA is not likely to be an effective instrument for the University of Michigan to assess the effect of our learning environment on its students. Rather, we favor the distributed and diverse campus approach currently in place.

Campus-wide Course Evaluations

Another important University-wide assessment effort is the University’s campus-wide course evaluations, which the Office of Evaluations and Examinations (E&E) conducts through its Teaching Questionnaires (TQ). In 2008- 09 the system moved from a paper-based format to online. Each academic unit must include four University-wide questions in its questionnaire, including the four items below that the instrument asks students to respond to on a five-point scale of agreement-disagreement. These questions are:

  1. Overall, this was an excellent course.
  2. Overall, the instructor was an excellent teacher.
  3. I learned a great deal in this course.
  4. I had a strong desire to take this course.

Of these four common questions, the faculty and academic administrators typically use responses to the first two questions about quality of the course and the instructor in three key ways: to consider ways to improve the curriculum, in faculty dossiers for tenure and promotion, and as a source of information to consider when setting annual merit increases.

A campus-wide compilation of these results shows a high degree of satisfaction with courses and instructors among our students. Here we focus on the student learning experience (TQ Q3), which shows high and steady marks from 2000 to 2008, especially at the upper undergraduate level and graduate level.

satisfaction
“I learned a great deal in this course.” (TQ Q3)

In addition to campus-wide evaluation, staff members at E&E help individual instructors, departments, and the schools and colleges to design custom questionnaires. Each academic unit may add up to 26 agreementdisagreement questions and up to five open-ended questions on each TQ. Faculty members design questionnaires online, individually or in groups, depending on the procedures individual departments follow. Some departments generate questionnaires based on a pre-defined set of core questions for all their courses. Other departments allow faculty members to design their own questionnaires for each individual course.

To further assist academic units and individual faculty members, E&E offers a catalog of hundreds of questions about teaching improvement from which academic units can select. These questions cover the topics of student development (knowledge, interests and values, participation, social awareness, self concept, and vocational skills and attitudes) instructor effectiveness (skill, climate, interaction, feedback, organization, and difficulty), assignments (written, reading, laboratory, and other), textbook, audiovisual material, instructional computing, and exams. In addition, E&E offers fifteen open-ended questions from which academic units can choose. Here, too, the topics of these items reflect back to the learning goals discussed above, with the academic units able to customize their questionnaires. The full set of optional questions is available on E&E’s website.

Academic units may also choose whether to include on their questionnaires a set of eight questions that the Michigan Student Assembly, the central student government at the University, uses to compile Advice Online, an evaluation guide for students. In the fall of 2008, the TQ program became a paperless operation allowing students to fill out electronic forms online. Systematic results from this change in procedure are not yet available.

ePortfolios at the University of Michigan (MPortfolio)

mportfolioThere are two main types of ePortfolios (electronic portfolios): professional or career portfolios, and portfolios that focus on students’ integrative learning. Professional portfolios consist of “artifacts” that showcase a person’s knowledge, skills, and abilities to others, increasingly using multimedia. Such artifacts can include papers, thesis, lesson plans, personal statements, evaluations, letters of recommendation, written reflections, designs, and musical compositions and resumes, as well as photographs, videos, and audio recordings. In ePortfolios whose purpose is integrative learning, students reflect on their learning, identify evidence of their learning, connect their personal and professional values and goals to their formal and informal learning experiences, apply knowledge and skills across different contexts, and demonstrate their ability and achievements. The University is piloting ePortfolios (called MPortfolio) and recently developed a website that includes examples produced by undergraduates from a range of educational levels, backgrounds, and disciplines.

ePortfolios can be used by academic programs, by units that provide co-curricular learning experiences, and in a wide variety of settings. Several units at the University that have piloted ePortfolio efforts include the School of Social Work, University Housing, the Ginsberg Center for Community Service Learning, the Office of Student Activities and Leadership, the Spectrum Center (with a focus on LGBT students, faculty, and staff), the Career Center, and the Program on Intergroup Relations. All of these units and programs have decided to continue beyond the pilot phase. In addition, many individual students create ePortfolios using Sitemaker, a University-supported software environment that lets non-technical people create customized websites and web-databases.

Individual Faculty Member Learning Assessment

As mentioned above, some units allow individual faculty members to tailor the Teaching Questionnaire to their individual courses, providing them with an opportunity to assess their students’ learning. In addition, faculty members from units across campus investigate aspects of student learning, in some cases with the support of grants from the University’s Center for Research on Learning and Teaching (CRLT), as described below. In a later section that focuses on IT and learning, several other faculty-based examples are described.

Investigating Student Learning Grants

Since its inception in 2008, 20 faculty members have received Investigating Student Learning Grants to investigate aspects of student learning in their courses or programs. The competition is open to all tenured and tenure-track faculty, clinical instructional faculty, and lecturers who have continuing appointments and course development responsibilities. Grants may be made to individual faculty members or to teams of faculty members and up to two graduate students to undertake a joint project.

The grant’s main objective is to support faculty members who want to explore if and how their pedagogical strategies are working, or to do research related to new innovations in teaching. To be approved, projects must be inquiry based, use methods appropriate to the discipline, and the results must be made public to inform the work of their colleagues and the discipline more broadly.

A few sample project titles are provided below to give a sense of the topics that faculty grant recipients have decided to focus their efforts on:

  • The Art and Science of Story-Telling in the Engineering Classroom.
  • Do Student Presentations Enhance Student Learning in Pathology Labs?
  • Fostering Critical Engagement and Cross-Cultural Comparison.
  • Beyond Clickers - A Web-Based System for Lecturing.
  • Structuring Evaluation to Improve Mathematical Writing, Reading, and Problem Solving.
  • Pedagogical Implications of Individual Interview Assignments through International Teleconference Using Skype.

Faculty Teaching Innovations (Faculty Development Fund Grant)

In the fall of 2008, twelve faculty members (eight individual faculty members and two two-person teams) received funds through the Faculty Development Fund Grant for innovations to enhance the quality of student learning. Administered by CRLT, this competition is open to all tenured and tenure-track faculty, clinical instructional faculty, and lecturers who have continuing appointments and course development responsibilities. When choosing between projects of equal merit, priority is given to proposals submitted by applicants who have not previously received funds and to proposals that incorporate multicultural perspectives and that use active learning strategies.

Sharing Ideas and Practices for Assessment

Although individual faculty members across campus are engaging in the study of learning assessment in their courses and in innovative approaches to teaching, as with so many other types of activities at the University providing venues for faculty members to share these ideas and approaches with their colleagues can be a challenge. Below, however, are examples of such information-sharing efforts.

Teaching Grants Information Sharing

As mentioned above, as part of its grant programs to support research on and innovation in teaching, CRLT asks faculty members to provide summaries of their findings, which are posted to CRLT’s website.

Examples of Faculty Members Using Technology in Teaching

The University provides descriptions online of how more than 50 University faculty members from a wide crosssection of disciplines use technology in their teaching. The projects are grouped into several categories: experiential and collaborative learning; using multimedia in classroom teaching; using the web for students to publish their work, using web-based training, tutorials, and simulations to engage students; using technology tools and strategies to promote active learning; and using inline tools to engage students with course content and to help them interact with others.

Research and Assessment of Teaching and Learning Posters

In September 2008, the Medical School, School of Education, and CRLT co-hosted a “walking dinner” in which faculty and research groups presented posters summarizing 62 separate research and assessment efforts related to teaching and learning. Also in fall 2008, the College of Engineering and the Center for Research on Learning and Teaching-North hosted a lunch event that featured 19 posters summarizing research on teaching and learning currently underway in the College of Engineering.

Provost’s Seminar on Teaching

Since the mid-1990s the provost has been hosting a seminar on teaching that covers a broad range of topics (see topics list on website, described in further detail below in the section on support for teaching). For example, in October 2008 the Provost’s Seminar on Teaching featured three concurrent sessions on effective teaching approaches for the millennial generation. In September 2009 the Provost’s Seminar focused on assessment, including an address from the provost on the need for faculty members to examine what students learn, a keynote address from Carol Schneider, President of AAC&U, and panels of University faculty members sharing their approaches to assessment in various settings.

LSA Assessment Symposium

The College of LSA sponsored a symposium bringing together faculty leaders in the college (chairs, associate chairs, and key staff) to exchange ideas about ways to further a culture of assessment. In addition to remarks from the dean, faculty members presented approaches to assessment across a wide range of disciplines. A second symposium in 2009 examined assessment activities currently underway at the department and college levels, and considered ways to further embed assessment of student learning into the college culture.

Enriching Scholarship Series

Taking place usually during one week in early May, the Enriching Scholarship Series, part of the Teaching and Technology Collaborative, offers pedagogical and hands-on, skill-building sessions for faculty members and other University instructors. This series offers sessions that explore ways to effectively integrate information and technology with teaching, learning, and research. One of sessions during the May 2009 Enriching Scholarship Series included posters or presentations by three 2-person faculty teams and two individual faculty members who are the inaugural 2009 Teaching Innovation Prize Winners. This prize will be described more fully below in the section on teaching awards.

The examples described above reflect a high degree of distributed activity and reveal a considerably amount of information sharing about teaching and learning across the schools and colleges, especially for such a diverse and widespread campus.

 

umseal