Did Stanley have anything to say about the SAT getting rid of analogies? My husband, who teaches MBA students, says the inability to make analogies is the besetting flaw in his students. They'll understand something in one context but can't transfer it to another.
I wonder if there would be a market for a rigorous standardized test that would do the kind of separation Stanley was looking for.
Same objection (they were removed later): he was appalled that the College Board thought it better to test achievement than aptitude. It never made sense to m. Well-resourced families and school districts would always do better with an achievement test. The point of aptitude tests is to find under-resourced sheer ability.
I'm reading this piece, nodding along with it... uh huh, uh huh... It always seemed to me, a product of one of a long series of under-resourced (shitty) rural schools, that the tests were slanted toward (what we called) the rich families. At a young age, it was more a curiosity. Nowadays, I can see it for the travesty it is.
Just so you know, I'm also a product of an under-resourced (shitty) rural school. Where I grew up we didn't even have rich families. I remember my 5th grade reading textbook talking about kids growing up in a city where they could walk to an ice cream shop and the playground -- might as well have been Narnia.
Yeah, that’s how it felt. I shouldn’t have said families; it was “the” rich family that owned the local machine shop (shut down in the 70’s when things got offshored.)
Asians and whites averaged about the same scores on the SAT until around the turn of the millennium. Since then, Asians have been pulling away from everybody else like Secretariat coming down the home stretch in the 1973 Belmont. Has anybody looked into how much of this change is due to changes in the population versus how much due to changes in the SAT?
“The point of aptitude tests is to find under-resourced sheer ability.”
Ok, but imo the point of the SAT (whatever the A stands for) and similar tests should be to measure both.
Or at least, it is useful indeed to measure both aptitude and achievement.
So I agree with the claim that it is badness that they removed those things that better measured aptitude, but I don’t think finding “sheer ability” independent of work ethic and diligence should be the only, or even main, goal.
Context matters. A disadvantaged student who does well on the SAT is showing something of value. Whether it’s that he and his parents cared to “game” the test or not.
Which is different from suggesting that the way the system works now is best for societal value. I’m just saying that a test system that primarily measured “aptitude” and nor achievement would likely be even further from optimal.
Thanks for explaining in detail what deeply irritates me about current default AI text, far more of a tell than the use of an em-dash.
I note it creeping in, even when it is doing a plausible job of emulating a specific writers voice.
I have a friend who uses ChatGPT as editor - specifically to downgrade the reading age of his text, as he wants to reach a mid-market audience.
Even though I know he wrote the original, the rewrite always has this AI stylistic voice.
I didn’t use to hate it - there is always room for an unnecessary flourish versus following Orwell’s dictums to the letter. Just not when you use the same flourish multiple times in every piece.
Thanks for pointing out explicitly what I’d realized intuitively.
Whenever I see more than a single “not X but Y” in some text, I know that it was LLM generated.
Which means that - unless I’m using the LLM myself - it is a quick way for me to discount the author and their text.
Where when using an LLM myself, I don’t always love it but it usually doesn’t bother me all that much.
So unlike you, I am actually *grateful* that the LLMs do this, because it is the single best marker to separate human generated output from LLM output.
I recently went back to my 1999 PhD dissertation (in evolutionary biology). I used the "not x but y" (with a lot more words) surprisingly commonly. The extra words hide the pattern but it's there once you are carefully looking for it.
It's also how I speak. I work now in a realm where I am purposely flipping client's self-descriptions, changing their frames of self-reference, so it's almost a verbal tic at this point. I have caught myself using the pattern at home 🫣. I'm currently writing a book and being a whole lot more careful about that pattern usage; once in a chapter is my limit.
Thanks for this engaging and informative article, which helped me understand both the source of the locutional epidemic "not this, but that“ and why I find it so annoying. The graphic Gemini came up with includes a weirdly inventive typo, or is it one? Does "scoffolding" refer to a type of argumentative scaffolding based on scoffing?
Thank you for the beautiful explanation of this phenomenon. I work on the mathematics of “embeddings”; these cosines and sines you mentioned. It’s fascinating to me to see the downstream, psychological impact of mathematical choices.
In a tangential direction (ha!), I have found that the GRE verbal scores are decent predictors of success in math grad school (for native speakers of English). Stanley was on to something!
Fascinating. Analyses like this make the case for why dialogue with chatbots can be used as a time-saving initial research method but never should be a ghost writer. In good prose, phraseology is a major part of personality.
I just read this latest Holly post, and loved the first ~25% of it.
Then the rest is *so* clearly written with AI, and I thought of this comment and this Hollis’ post.
Partly it made me sad.
And partly I wondered about the irony of Holly - whose Substack I’ve generally enjoyed, though I’m not a regular there - clearly using AI so much to help with this piece in particular.
I didn't know that substack and you are right and you have saved me the time ever reading it again. That's just too bad. It also has that single-sentence paragraph slam poetry form.
I spent way too much time (glorious time, but lots) in a class laying out the complexities of what I would call an unbalanced opposition that begins the following epitaph that Christina Sharpe deploys at the beginning of her essay “Beauty is a method.” Here it is.
"Beauty is not a luxury, rather it is a way of creating possibility in the space of enclosure, a radical act of subsistence, an embrace of our terribleness, a transfiguration of the given. It is a will to adorn, a proclivity for the baroque, and the love of too much." —Saidiya Hartman
Not a luxury but a way… Now, the point of sharing this lovely bit of prose in this context is that the opposite of luxury might be necessity, but to get the point here requires thinking about the mismatch in the middle of this opposition, not their obvious mirroring. Good writers, as you say above, might occasionally and for rhetorical purposes state a clear antonymy, especially when staging disagreement. Someone say, “Beauty is a luxury.” Their interlocutor then says, “Beauty is not a luxury, it’s a necessity.” Right. I get that. But better writers, or more demanding writers, work two oppositions at the same time and let them overlap as they do here. E.g., you might leave the first clause thinking “Beauty is a necessity,” but you don’t know why or for what purpose until the next clause. And if you only say, “Beauty is a way of creating possibilities…” you might not even bring the luxury/necessity opposition to the table; the latter are several of myriad available predicates. So, a good or better "Not x but y" uses the y to complicate the former.
But why did it take time in class to work this out? It was already difficult in discussion to pull each of the oppositions into the ring. Even defining the luxury against necessity was difficult.
Finally, just as with my beloved em dash, better to best writers must now give up on moves that are made too easily by LLMs, so easily that there are larded with the connotations of no-one-home. Which is weird. LLMs are being trained on good writing. So, to be better, writers have to go weirder. Which will also find its way into the training data. Endgame?
I had no idea why the "this isn't x, it's y" syntax annoyed me so much! I assumed it was just a peccadillo (like my disdain for the grocer's apostrophe, or my outrage when someone types "loose" when they mean "lose"). But thanks to this article I have neurological science to substantiate my opinion.
I'm not a grammar nazi after all -- I'm merely a cognitive miser, just as natural selection made me.
Right? Like you're being proactively mansplained, "Now you probably think x, don't you? That's so cute. No, of COURSE not, silly rabbit, it's y..."
Just sets my teeth on edge. And makes me wonder: Could it be that all text on the internet is just a steaming heap of preemptive contradiction served up with a soupçon of condescension?
Thanks for putting into words something that irked me but I couldn’t quite explain why. That negative construction is always a tell that LLMs were used to write a LinkedIn post. Also, I appreciated this as a former gifted student (and all the baggage that implied!).
NLP identified one of the functional reasons for use of antonym pairings in persuasive speech is that, properly placed, they tend to induce trance states. If you are arguing a case, the listener’s (or reader’s) critical faculties can be switched off by a sufficiently intriguing juxtaposition, increasing susceptibility to subsequent suggestion.
NLP is just a theory or rationalization for phenomena of hypnosis, some of it derived from observation of the clinical work of Milton Erickson. Calling it “pseudo-science” seems gratuitous slander, based mostly on distaste for the style of associated hype.
I think LLM has also internalized the dictum that "the hallmark if adult education is repetion". Either that or is given the instruction to write a 5000 word essay on mixing water and dirt to make mud and runs out of material.
I think you should mention why antonyms are close in distributional space: true antonyms must be the same in every dimension except one, the one in which they are opposite. You can use antonyms in the same place in most sentences.
Did Stanley have anything to say about the SAT getting rid of analogies? My husband, who teaches MBA students, says the inability to make analogies is the besetting flaw in his students. They'll understand something in one context but can't transfer it to another.
I wonder if there would be a market for a rigorous standardized test that would do the kind of separation Stanley was looking for.
Same objection (they were removed later): he was appalled that the College Board thought it better to test achievement than aptitude. It never made sense to m. Well-resourced families and school districts would always do better with an achievement test. The point of aptitude tests is to find under-resourced sheer ability.
I'm reading this piece, nodding along with it... uh huh, uh huh... It always seemed to me, a product of one of a long series of under-resourced (shitty) rural schools, that the tests were slanted toward (what we called) the rich families. At a young age, it was more a curiosity. Nowadays, I can see it for the travesty it is.
They used to be better. They have gradually gotten worse. It is not an either-or however. They are still better than most other measurements.
I'll take you word for it; I'm not an expert. My overall general take is "the system" is awful.
Just so you know, I'm also a product of an under-resourced (shitty) rural school. Where I grew up we didn't even have rich families. I remember my 5th grade reading textbook talking about kids growing up in a city where they could walk to an ice cream shop and the playground -- might as well have been Narnia.
Yeah, that’s how it felt. I shouldn’t have said families; it was “the” rich family that owned the local machine shop (shut down in the 70’s when things got offshored.)
Why uncover aptitude when it will just end up being competition for your own progeny?
They might discover a cure that will save my progeny's life!
They think the pie is finite. What they don’t understand is that every human that blooms makes the world bigger — more pie for everyone.
Asians and whites averaged about the same scores on the SAT until around the turn of the millennium. Since then, Asians have been pulling away from everybody else like Secretariat coming down the home stretch in the 1973 Belmont. Has anybody looked into how much of this change is due to changes in the population versus how much due to changes in the SAT?
As a Randian I abhor this kind of thinking.
“The point of aptitude tests is to find under-resourced sheer ability.”
Ok, but imo the point of the SAT (whatever the A stands for) and similar tests should be to measure both.
Or at least, it is useful indeed to measure both aptitude and achievement.
So I agree with the claim that it is badness that they removed those things that better measured aptitude, but I don’t think finding “sheer ability” independent of work ethic and diligence should be the only, or even main, goal.
Context matters. A disadvantaged student who does well on the SAT is showing something of value. Whether it’s that he and his parents cared to “game” the test or not.
Which is different from suggesting that the way the system works now is best for societal value. I’m just saying that a test system that primarily measured “aptitude” and nor achievement would likely be even further from optimal.
I really enjoyed this. Thank you.
Thank you!
Thanks for explaining in detail what deeply irritates me about current default AI text, far more of a tell than the use of an em-dash.
I note it creeping in, even when it is doing a plausible job of emulating a specific writers voice.
I have a friend who uses ChatGPT as editor - specifically to downgrade the reading age of his text, as he wants to reach a mid-market audience.
Even though I know he wrote the original, the rewrite always has this AI stylistic voice.
I didn’t use to hate it - there is always room for an unnecessary flourish versus following Orwell’s dictums to the letter. Just not when you use the same flourish multiple times in every piece.
And now the twist.
Right? It helps to know why!
Thanks for pointing out explicitly what I’d realized intuitively.
Whenever I see more than a single “not X but Y” in some text, I know that it was LLM generated.
Which means that - unless I’m using the LLM myself - it is a quick way for me to discount the author and their text.
Where when using an LLM myself, I don’t always love it but it usually doesn’t bother me all that much.
So unlike you, I am actually *grateful* that the LLMs do this, because it is the single best marker to separate human generated output from LLM output.
Excellent point
I recently went back to my 1999 PhD dissertation (in evolutionary biology). I used the "not x but y" (with a lot more words) surprisingly commonly. The extra words hide the pattern but it's there once you are carefully looking for it.
I suspect it's common in dissertations where the point is trying to explain something new and original. This would be an interesting research project.
It's also how I speak. I work now in a realm where I am purposely flipping client's self-descriptions, changing their frames of self-reference, so it's almost a verbal tic at this point. I have caught myself using the pattern at home 🫣. I'm currently writing a book and being a whole lot more careful about that pattern usage; once in a chapter is my limit.
Thanks for this engaging and informative article, which helped me understand both the source of the locutional epidemic "not this, but that“ and why I find it so annoying. The graphic Gemini came up with includes a weirdly inventive typo, or is it one? Does "scoffolding" refer to a type of argumentative scaffolding based on scoffing?
Ha! Smart and funny. Gemini always makes one mistake that a reader catches (I never do) and you're the first one...
Thank you for the beautiful explanation of this phenomenon. I work on the mathematics of “embeddings”; these cosines and sines you mentioned. It’s fascinating to me to see the downstream, psychological impact of mathematical choices.
In a tangential direction (ha!), I have found that the GRE verbal scores are decent predictors of success in math grad school (for native speakers of English). Stanley was on to something!
Fascinating. Analyses like this make the case for why dialogue with chatbots can be used as a time-saving initial research method but never should be a ghost writer. In good prose, phraseology is a major part of personality.
Thank you and yes!
Curious what @HolyMathnerd thinks about your point...
https://hollymathnerd.substack.com/p/the-democratization-of-nothing
I just read this latest Holly post, and loved the first ~25% of it.
Then the rest is *so* clearly written with AI, and I thought of this comment and this Hollis’ post.
Partly it made me sad.
And partly I wondered about the irony of Holly - whose Substack I’ve generally enjoyed, though I’m not a regular there - clearly using AI so much to help with this piece in particular.
I didn't know that substack and you are right and you have saved me the time ever reading it again. That's just too bad. It also has that single-sentence paragraph slam poetry form.
I spent way too much time (glorious time, but lots) in a class laying out the complexities of what I would call an unbalanced opposition that begins the following epitaph that Christina Sharpe deploys at the beginning of her essay “Beauty is a method.” Here it is.
"Beauty is not a luxury, rather it is a way of creating possibility in the space of enclosure, a radical act of subsistence, an embrace of our terribleness, a transfiguration of the given. It is a will to adorn, a proclivity for the baroque, and the love of too much." —Saidiya Hartman
Not a luxury but a way… Now, the point of sharing this lovely bit of prose in this context is that the opposite of luxury might be necessity, but to get the point here requires thinking about the mismatch in the middle of this opposition, not their obvious mirroring. Good writers, as you say above, might occasionally and for rhetorical purposes state a clear antonymy, especially when staging disagreement. Someone say, “Beauty is a luxury.” Their interlocutor then says, “Beauty is not a luxury, it’s a necessity.” Right. I get that. But better writers, or more demanding writers, work two oppositions at the same time and let them overlap as they do here. E.g., you might leave the first clause thinking “Beauty is a necessity,” but you don’t know why or for what purpose until the next clause. And if you only say, “Beauty is a way of creating possibilities…” you might not even bring the luxury/necessity opposition to the table; the latter are several of myriad available predicates. So, a good or better "Not x but y" uses the y to complicate the former.
But why did it take time in class to work this out? It was already difficult in discussion to pull each of the oppositions into the ring. Even defining the luxury against necessity was difficult.
Finally, just as with my beloved em dash, better to best writers must now give up on moves that are made too easily by LLMs, so easily that there are larded with the connotations of no-one-home. Which is weird. LLMs are being trained on good writing. So, to be better, writers have to go weirder. Which will also find its way into the training data. Endgame?
I had no idea why the "this isn't x, it's y" syntax annoyed me so much! I assumed it was just a peccadillo (like my disdain for the grocer's apostrophe, or my outrage when someone types "loose" when they mean "lose"). But thanks to this article I have neurological science to substantiate my opinion.
I'm not a grammar nazi after all -- I'm merely a cognitive miser, just as natural selection made me.
When an LLM gives me that not x but y thing, I feel like I've been pushed around, told what to think.
Right? Like you're being proactively mansplained, "Now you probably think x, don't you? That's so cute. No, of COURSE not, silly rabbit, it's y..."
Just sets my teeth on edge. And makes me wonder: Could it be that all text on the internet is just a steaming heap of preemptive contradiction served up with a soupçon of condescension?
Weirdly enriching post
https://hollisrobbinsanecdotal.substack.com/p/how-to-tell-if-something-is-ai-written
It is weirdly enriching yes.
Thanks for putting into words something that irked me but I couldn’t quite explain why. That negative construction is always a tell that LLMs were used to write a LinkedIn post. Also, I appreciated this as a former gifted student (and all the baggage that implied!).
Thank you spread the word!
Genuinely a fantastic article!
Thank you!
NLP identified one of the functional reasons for use of antonym pairings in persuasive speech is that, properly placed, they tend to induce trance states. If you are arguing a case, the listener’s (or reader’s) critical faculties can be switched off by a sufficiently intriguing juxtaposition, increasing susceptibility to subsequent suggestion.
!!!!
Well, perhaps it is just trying to defend itself (😏), but ChatGPT says your claim ain’t true.
https://chatgpt.com/share/69acf826-dd58-8005-a5df-aa12a19ca401
NLP is just a theory or rationalization for phenomena of hypnosis, some of it derived from observation of the clinical work of Milton Erickson. Calling it “pseudo-science” seems gratuitous slander, based mostly on distaste for the style of associated hype.
This essay is not vague; it’s highly clarifying.
In all seriousness though this was a great explanation of why this pattern is annoying
I think LLM has also internalized the dictum that "the hallmark if adult education is repetion". Either that or is given the instruction to write a 5000 word essay on mixing water and dirt to make mud and runs out of material.
By the way, Dave Birss developed a tool to help identify such language. It’s called the “Bull Sheet.” Check it out here: https://davebirss.com/web-tools/the-bull-sheet/
I think you should mention why antonyms are close in distributional space: true antonyms must be the same in every dimension except one, the one in which they are opposite. You can use antonyms in the same place in most sentences.
I hope the conversation will continue by people who know more than I do about this…