American Sociological Association Statement on the Use of SRIs for Retention and Promotion

By -

September 2019


Most faculty in North America are evaluated, in

part, on their teaching effectiveness. This is

typically measured with student evaluations of

teaching (SETs), instruments that ask students to

rate instructors on a series of mostly closedended

items. Because these instruments are

cheap, easy to implement, and provide a simple

way to gather information, they are the most

common method used to evaluate faculty

teaching for hiring, tenure, promotion, contract

renewal, and merit raises.


Despite the ubiquity of SETs, a growing body of

evidence suggests that their use in personnel

decisions is problematic. SETs are weakly related

to other measures of teaching effectiveness and

student learning (Boring, Ottoboni, and Stark

2016; Uttl, White, and Gonzalez 2017); they are

used in statistically problematic ways (e.g.,

categorical measures are treated as interval,

response rates are ignored, small differences are

given undue weight, and distributions are not

reported) (Boysen 2015; Stark and Freishtat

2014); and they can be influenced by course

characteristics like time of day, subject, class

size, and whether the course is required, all of

which are unrelated to teaching effectiveness.


In addition, in both observational studies and

experiments, SETs have been found to be biased

against women and people of color (for recent

reviews of the literature, see Basow and Martin

2012 and Spooren, Brockx, and Mortelmans

2015). For example, students rate women

instructors lower than they rate men, even when

they exhibit the same teaching behaviors

(Boring, Ottoboni, and Stark 2016; MacNell,

Driscol, and Hunt 2015), and students use

stereotypically gendered language in how they

evaluate their instructors (Mitchell and Martin

2018).


........


3. SETs should not be used to compare

individual faculty members to each other or

to a department average. As part of a holistic

assessment, they can appropriately be used

to document patterns in an instructor’s

feedback over time.

4. If quantitative scores are reported, they

should include distributions, sample sizes,

and response rates for each question on the

instrument (Stark and Freishtat 2014). This

provides an interpretative context for the

scores (e.g., items with low response rates

should be given little weight).

5. Evaluators (e.g., chairs, deans, hiring

committees, tenure and promotion

committees) should be trained in how to

interpret and use SETs as part of a holistic

assessment of teaching effectiveness (see

Linse 2017 for specific guidance).

Comments

Post a Comment

Popular posts from this blog

A REPORT ON THE UVU “FINAL INVESTIGATION REPORT”: THE CASE OF PROFESSOR MICHAEL SHIVELY

By -
This week I read Utah Valley University’s Final Investigative Report on Professor Michael Shively, dated July 31, 2019. With a set of appendices, it is a public  document, filed in a lawsuit against UVU by the family of Professor Michael Shively (Case 2:20-cv-00119-DB-PMW Document 17-1 Filed 03/26/20; Utah Federal District Court).  I discuss the Report here because it reveals serious flaws in a procedure that should afford due process to members of the faculty while also enabling the University to dismiss faculty when there is proven cause. A UVU Faculty Senate Committee, of which I am a member, is currently writing a new policy on “Faculty Remediation, Sanction, and Dismissal for Cause.” Mistakes made in the Shively case can help us as we develop guidelines that are more reasonable than those in the earlier policy and more conducive to a mutual sense of trust and collegiality. Professor Shively passed away on August 19, 2019, never having seen this report that discounts ma
Read more

Statement on the Suspension of Professor Mike Shively at UVU

By -
Image
Mike Shively training on the San Marcos River for the 2019 Texas Water Safari (courtesy of the Shively family) Our colleague, Professor Michael Shively, passed away on the evening of August 19, 2019. Mike joined the UVU faculty in 1992 with advanced degrees in anatomy and veterinary science. He served as President of the Faculty Senate and as Department Chair. He was the Faculty Athletic Representative, a role he relished as he played his trumpet at many athletic events. Over the course of his teaching career he taught thousands of students in his anatomy and physiology classes and was honored with numerous teaching awards. Students found his demanding classes key components for admission into medical, dental, and nursing schools.  Although “Rate My Professors” is an imperfect source, it is informative to see what students posted there about Mike’s anatomy class. Five recent, representative examples:  AWSOME, 5 out of 5 rating Anatomy is the hardest class you will prob
Read more

Begging the DEI Question

By -
On January 19, House Republicans voted to pass HB 261, Rep Katy Hall’s diversity, equity, and inclusion bill. Their votes suggest that they did not pay attention to Andy Larsen’s thoughtful analysis of the issue (Salt Lake Tribune, January 13): “One of the most popular anti-DEI studies is full of problems.” Or perhaps they were simply more interested in the politics of the issue than in careful analysis in the service of Utah’s students.  Larsen writes that the question the bill purports to answer is what to do with the “DEI bloat in academia,” as reported by the conservative Heritage Foundation in 2021 (and in a flurry of subsequent pieces there). After pointing out serious methodological problems in the report (“it’s an insane way to draw conclusions”), Larsen concludes his analysis: “In the end, the anti-DEI folks have a problem: Their argument just isn’t that well-considered. They’re using shoddy methodology [and] they don’t address the research that led to DEI programs in the fi
Read more