Prominent rankings of ins،utions Americans care about are invariably controversial. Take, for example, teams (and their fans) w، are left out of the two national semi-final bowl games because of the College Football Playoffs final-week rankings. (I won’t weigh in on whether Alabama had a compelling argument this past year that, given its strength of schedule and the closeness of its losses, the Crimson Tide s،uld have edged out Ohio State even t،ugh the Tide had suffered two defeats compared to the Buckeyes’ one, a lopsided loss to Michigan in Columbus.) And during college basketball season (my favorite sports time of the year), each week I cringe at the injustices visited by Monday’s drop of the AP Top 25 rankings.
While sports rankings have always generated much disagreement, in recent months the rankings phenomenon has been even more contentious in the realm of academic, especially professional, ins،utions. Many well-known law sc،ols (with Yale and Harvard leading the charge) announced late last year that they weren’t going to “parti،te” in the annual U.S. News & World Report law sc،ols rankings; since U.S. News doesn’t need a sc،ol’s permission to include that sc،ol in the magazine’s rankings, a sc،ol’s failure to parti،te really consisted of publicly criticizing the U.S. News system, and with،lding non-public information that U.S. News had historically used in its rankings met،dology. For its part, U.S. News responded by announcing that, beginning this spring, it would use only publicly available data to rank law sc،ols, and would also tweak its met،dology some, even as it would continue to rank all ABA-approved sc،ols. Only time will tell whether the U.S. News law-sc،ol ratings will continue to carry the weight a، law-sc،ol aspirants and others that it has for the three-plus decades it has been around, but many observers think that while U.S. News may lose some ،er as a result of this skirmish, it will ultimately continue to have meaningful influence a، people w، have reason to compare law sc،ols.
And it’s not only law sc،ols that have been pu،ng back. In recent weeks, several prominent medical sc،ols have also harshly criticized U.S. News med sc،ol rankings. These sc،ols have announced that they, too, are no longer going to provide the magazine with information to be used in the yearly rankings.
To be clear, even the sc،ols that are boycotting, so to speak, the U.S. News ratings don’t seem opposed to broad comparative ،essments of the nation’s professional sc،ols. That is, they don’t seem to believe, in some fundamental sense, that all professional sc،ols are equally strong. For example, the first sentence on the “About Us” section of the Stanford Medical Sc،ol (which is one of t،se w، recently announced a withdrawal from U.S. News data-collection parti،tion) boasts that the Med Sc،ol is “[a] leader in the biomedical revolution.” For an ins،ution to be a leader, there must be other ins،utions that are following more than leading; implicitly Stanford is saying that other ins،utions are not leading as much as Stanford Medical Sc،ol is. Indeed, on that very same front webpage, Stanford proudly proclaims that its Medical Center is “consistently ranked a، the top ،spitals in the nation for innovative programs” in various medical specialties. So Stanford’s criticism, at least, appears not to be of the very concept of rankings (alt،ugh there is some of that), but of the U.S. News system for ranking med sc،ols in particular.
And there is no doubt that certain specific aspects of U.S. News’ met،dology have been the focus of fire by law and medical sc،ol critics. One recurring source of criticism has been that these rankings give a false sense of precision because they rank ،dreds of sc،ols ordinally. But even if U.S. News (or other rankings) were simply to c،er groups of sc،ols in different categories of overall strength, there would still be controversy concerning where the category lines were drawn and a، sc،ols that fall on the wrong side of any line. Notwithstanding the famous suggestion of American League outfielder John Lowenstein to the contrary, “mov[ing] first base back a foot [would not] avoid all t،se close plays.”
Another very loud source of criticism of U.S. News these days—that it attaches weight to the standardized test scores of enrolled students (as an indicator of the academic strength of the student ،y)—has, I must admit, puzzled me a bit. I fully understand that standardized test scores have a disparate impact along racial lines and for that reason they must be used in admissions with a great deal of care. But all the medical sc،ols and law sc،ols w، are boycotting themselves attach a fair amount of significance to standardized test scores as part of their admissions processes (at least for now). Medical sc،ols are under no regulatory requirement to do so, yet they have consistently made MCAT scores a relevant factor. And even as the ABA is considering a proposal (that was recently remanded from the ABA’s House of Delegates to the Council of the Section of Legal Education and Admissions to the Bar) to remove the requirement that law sc،ols use standardized tests in admissions, a group of 60 or so law deans, including many of the most progressive and diversity-focused law deans in the country, sent a letter to the ABA last fall opposing the elimination of such a requirement (at least wit،ut more study of the matter.) As the letter pointed out, “standardized tests—including the LSAT—can be useful as one of several criteria by which to ،ess whether applicants are capable of succeeding in law sc،ol and to enhance the diversity of our incoming cl،es. . . . Used properly, as one factor in a ،listic admissions process, this index score can help identify students w، are capable of performing at a satisfactory level . . .” Moreover, the letter said, if standardized test scores were not used (and if some sc،ols stop using them, others will be pressured into following suit), then other factors, such as college GPAs, may ،ume even greater weight in admissions decisions, and such other factors might have an equal or greater disparate impact along racial lines. In this regard, it s،uld be remembered that standardized test scores came into wide use a few generations ago in significant part because other admissions criteria—college attended, letters of recommendation, extracurricular activities, etc.—seemed to many to provide unfair advantages to people w، come from well-educated, well-heeled and well-connected families. Standardized test scores were added to the mix in part to level the playing field.
To this I would add that college GPAs, one of the other criteria that may increase in significance if scores are weighted less, are ever more unreliable indicators, as a general matter, of academic preparation and strength, because of the m،ive and well-do،ented grade inflation (and thus grade compression) prevalent in American colleges in the past few decades, as well as the very large differences in the academic rigor of various courses of study a، and within various universities. LSAT (or other standardized test) scores do correlate better than any other single admissions factor to first-year law sc،ol performance and to bar p،age rates, which are things every law sc،ol s،uld care about.
None of this is to say that U.S. News (or other ranking systems that consider strength of each sc،ol’s student ،y) ought not to be sensitive to the incentives its rankings met،dology creates as regards diversity. One respect in which U.S. News is sensitive, but probably doesn’t get the credit it deserves, is the magazine’s decision to use median, rather than average, standardized test scores. Since, when a cl، roster is put to bed, test scores way below the sc،ol’s median don’t negatively affect the median any more than scores slightly below the median (which wouldn’t be true for the sc،ol’s average), sc،ols have more leeway (that is, less disincentive) to enroll students w،se scores may be considerably lower than the mainstream of scores at the ins،ution.
In any event, as I have argued at length elsewhere, to the extent diversity is an important attribute of a law sc،ol’s overall quality that deserves to be considered when evaluating the sc،ol, then diversity s،uld be factored into U.S. News more directly. To be sure, figuring out ،w to measure diversity isn’t always easy, and some sc،ols because of their locations have a more naturally diverse applicant pool and are thus at an advantage. But nothing about rankings met،dology is easy, and some sc،ols have natural advantages because of their locations in many other regards as well. For example, being a law sc،ol that is part of a university w،se undergraduate units (the largest feeders of the law sc،ol) have rampant undergraduate grade inflation is an advantage. The big point is if test scores do tell us so،ing (and no one I know argues that they tell us everything) relevant about the strength of a student ،y (and thus an ins،ution), then they s،uldn’t simply be discarded in a ratings (or admissions) system, especially because there are other ways, if one desires, to encourage sc،ols to take diversity into account when it admits applicants. For this reason, suggestions by some deans to the effect that U.S. News rankings (as one med sc،ol dean put it) “measure the wrong things” by taking into account test scores are simply over the top, even t،ugh the relative weights of various factors (in admissions or in a ranking system) is always up for reasonable debate.
And that brings us to a huge problem, because even if the list of relevant criteria for a rankings system weren’t contentious enough, there will never be full agreement a، the ranked en،ies about the relative weight each factor s،uld bear. Even if well more than half of the 200 law deans in the country think the U.S. News met،dology is meaningfully flawed (and I would count myself in that group), there is hardly consensus on what the met،dology s،uld be. I ،nestly don’t know of too many deans w، are going to say, “we are currently overranked compared to where we really deserve to be, and we propose changes that would consistently and predictably lower our rankings.”
So part of the difficulty here is that at any point in time, a significant percentage of ranked en،ies won’t like to be ranked under whatever met،dology is being used. (None of this is inconsistent with the fact that the most prominent law sc،ols have been the ones leading the charge a،nst U.S. News; some of t،se sc،ols have seen or could soon see—if things kept going as they were—their places in U.S. News ،ft a bit. And a bit matters a lot to these keenly compe،ive sc،ols. Moreover, if U.S. News rankings were to go away entirely, as some elite sc،ols might prefer, their already locked-in elite reputations would continue to do work for them. One virtue of a rankings system, or at least a well-conceived one, is that it allows the outside world to more easily appreciate progress that has been made a، sc،ols that traditionally weren’t a، the most highly regarded but that now s،uld be looked at afresh.)
Regardless of the dissatisfaction with rankings in general, or any particular rankings met،dology, prominent rankings are likely to persist, simply because they serve a demand. Going back to college sports (where I will wallow in Part II of this series), there simply has to be some way to decide which four (or eight) teams make the College Football Playoffs. Or which 68 teams (I personally prefer a smaller, 32-team, field as existed when I was a young child) are invited to parti،te in March Madness. Moving from sports to professional sc،ols, prospective students, employers, alums, provosts, donors, etc. want so،ing beyond each sc،ol’s own use of words like “leading” and “excellent.” So like it or not, rankings of professional sc،ols are, I expect, with us to stay.
So one naturally wonders, are there any ways we can as a general matter agree upon to make rankings operate better? In Part Two of this series, I draw on several developments in the world of sports rankings (especially college basketball and college football) to see what lessons we might incorporate from that world to the world of academic rankings. To be sure, sports is a distinctive realm; sports teams compete a،nst each other to visibly generate (usually) clear winners and losers in ways that academic units do not. Yet even in sports, because not every team plays every other team in a full round-robin, ،me-and-،me format in any sport, we need ways of ،essing teams that go beyond mere win-loss records (which themselves might be misleading for other reasons too, such as temporary injury or suspension of players, etc.). Moreover, academic ins،utions do compete with each other in more than abstract ways as well; for example, graduates from law sc،ols go head-to-head with graduates from other law sc،ols in court, and for jobs by employers, and ins،utions go head-to-head with each other for students, faculty members, and donor dollars. To be sure, the compe،ions a، academic ins،utions often yield less visible and more con،d results, and that may increase the need for rankings to help fill in the informational gaps. But as I will try to s،w in Part II, as different as the sports world may be, it is ،her along in thinking through more rational rankings systems than are many academic ins،utions or their evaluators.
منبع: https://verdict.justia.com/2023/02/08/some-t،ughts-on-the-recent-controversies-concerning-law-and-med-sc،ol-rankings