#when what's REALLY needed is. like. 1-2hr long interview and review of history

[queenlua]

"Our results suggest that between 6.5% and 16.9% of text submitted as peer reviews to these conferences could have been substantially modified by LLMs ... the estimated fraction of LLM-generated text is higher in reviews which report lower confidence, were submitted close to the deadline, and from reviewers who are less likely to respond to author rebuttals." (emphasis mine)

this is just a rando arxiv thing so like grain of salt, but

(1) lmao

(2) it is interesting (and depressing) to me how, like. there's a realm of stuff that we Have Not Been Able To Automate, right, where you actually need A Real Fucking Human Being to sit down and Use Their Cognitive Faculties to determine Is This Real Or Is It Bullshit. and for a while we've been relying on, like, some human Typing Up Some Paragraphs as a sort of proof-of-work that can't really be replicated by an automated system

now admittedly, all these systems that fucking *hate* the "ineffiency" of spending human time and expertise to properly analyze a situation, so the people in those roles are inudated with Too Much Work, and thus they've already been trying to skirt proof-of-work in various ways (writing lazy/bad peer feedback, skimming the paper, etc). so, y'know, "use of chatgpt to churn out the peer reviews you despise doing and don't have time for anyway" is just a step function in a trend that already clearly existed. but it does seem sort of troubling how much everything pushes toward doing things in this "efficient" way (that is, ways that look good in ways that are easily legible) instead of, like, are we doing something real here. because the real doesn't go away. and when quality just goes to shit because ppl are too lazy/tired to properly vet stuff, augh

#ai #i also just read an article in Undark about how PANDAS diagnoses are a bitch and a half #partially because the way medical billing works docs literally don't spend more than 15min with you #when what's REALLY needed is. like. 1-2hr long interview and review of history #which tbh hit a little hard #since yeah. had to see a specialist a month ago #i wrote all my questions down in advance sprinted through them as fast as possible #and STILL got cut off so like. what was accomplished here lol