‘But there is a difference between recognising AI use and proving its use. So I tried an experiment. … I received 122 paper submissions. Of those, the Trojan horse easily identified 33 AI-generated papers. I sent these stats to all the students and gave them the opportunity to admit to using AI before they were locked into failing the class. Another 14 outed themselves. In other words, nearly 39% of the submissions were at least partially written by AI.‘
Article archived: https://web.archive.org/web/20251125225915/https://www.huffingtonpost.co.uk/entry/set-trap-to-catch-students-cheating-ai_uk_691f20d1e4b00ed8a94f4c01
Don’t really know how to feel about this because 15 years ago, all I did was reword Wikipedia pages to make a good paper. I went to college because I was led to believe it was a requirement to do well in life. I still learned a lot, but that was mostly through the social interaction of coursework. And honestly, I don’t use anything from college in my current engineering job, it was all on-the-job panic learning. If I were to go back to college today, it would be such an enlightening experience of learning, but when you’re a kid getting out of high school, you’re just trying to get by with some gameplan that you’ve only been told about. Idk. I don’t blame them for using a tool that’s so easily accessible because college is about fun too. I guess I wouldn’t do it different at that age .
Apparently you learned to learn, which I suppose is one major goal of college.
I’m in the same boat, and for me personally no, in uni I learnt to do as minimal of a job as possible to “pass” the arbitrary goals set by uncaring world. I had to unlearn all of that very quickly when I got my first real job that I actually like. My uni broke me, for sure, and I’m lucky I fixed a little bit of that decades later.
I’m sorry to hear that.
Would you say that your experience was typical or was it especially bad for you (as in not designed for your needs) while other people were better off?
That’s a nice ideal but the reality is that this world is cruel and we’re burdening future generations with debt for their degrees and the job market sucks. If reality was different, then maybe kids could enjoy learning in college. But it’s not, so they need to make sure they are capable of being good little sheep that can do what the C suite wants otherwise they’re going to be in poverty and debt for the rest of their lives with very little safety net.
US here, in case it wasn’t obvious.
You hit the nail on the head.
The problem is the cost of education in the US.But not all of the world is such a capitalist hellscape as the US is, where people were embezzled of affordable living, healthcare and education.
That doesn’t make the concept of education a bad one. The framework in which it’s implemented is to blame and the people who created said framework.
I think that rewording wikipedia is slightly better though. It still requires you to digest some of the information. Kind of like when your teacher let you create notes on a note card for the test. You have to actually read and write the information. You get tricked into learning information.
Ai, just does it for you. There’s no need to do much else, and it’s reliability is significantly worse that random wiki editors could ever be. I see little real learning with ai.
With AI you cane solve an assignment without:
- reading the assignment
- reading the source of information
- reading the answer that “you” “wrote”
With the rewording Wikipedia approach you had to do all of those three things
Another thing is, you often gain interest on the topic, and Wikipedia indeed has the neat little thing of articles being related to each other, so it’s very plausible to start on Chandler Bing and end on the Atlantic slave trade, for instance. With LLMs, this is much, MUCH rarer, considering whatever you find interesting must be researched manually, since LLMs are more or less useless.
This refrain I keep hearing of “I don’t use anything I learned in college” is an INSANE take. Unless you went to some fly-by-night for-profit scam college, you learned way more than you think, even if it didn’t include some specific engineering technique. You mentioned social interaction, but critical thinking is the big one. We need to stop devaluing education-it’s critical for our future. We can’t dismiss it just because capitalist vultures are ruining it. We need to fight to make it what it should be.
Great read
and professors use AI TO try to suss how AI, and giving them wrongfull grades.
Have you read the article?
He runs right up to the actual problem but side-steps it:
“A college degree is not just about a job afterwards – you have to be able to think, solve problems and apply those solutions, regardless of the field.”
Problem: With generative AI, this is the LAST thing employers want. If you’re out there working right now, particularly in tech? It’s all about “leveraging” AI to “be more efficient.” They don’t want you thinking and solving problems on your own, they want you regurgitating solutions they presume are pre-vetted by AI.
I’ve had these discussions at my own job… “But, but, Generative AI makes it so easy to make and place Facebook ads!” - Agreed, and that’s not my job. “But, but, you can analyze data and generate reports!” - Yes, also not my job.
But the push by business to use it is HUGE, and in that environment, some student using it to cheat in a history class, ultimately, will benefit more from that experience in the “real” world than probably taking the history class in the first place.
Then again, my plan is to John Henry the shit out of this until I’m dead.
For those who never took a folklore class:
https://www.americanfolklore.net/john-henry-the-steel-driving-man/
What the business wants and what they say they want are two completely different things. They still want people to be able to think and solve their own problems, even if they end up assigning the praise for your hard work on the AI you only pretended to use
This is a new pattern that I am seeing. C suite is required to launch AI initiatives because of market expectation, tech staff expected to launder their work through Claude
And literally all of them can fuck right off and fire me before I do that, the only thing keeping me from being fully radicalized against this system is that I am still(just barely) able to afford food and shelter.
I think that most of all they want you to feel like AI could replace you if you misbehave.
It should be treated the same as if another student wrote the paper. If it was used as a research tool where you didn’t repeat it word for word then it’s cool, it can be treated like a peer that helped you research. But using it to fully write then it’s an instant fail because you didn’t do anything.
Okay, sure. But how can you identify its use? You’d better be absolutely confident or there are likely to be professional consequences.
Not to mention completely destroy your relationship with the student (maybe not so relevant to professors, but relationship building is the main job of effective primary and secondary educators.)
Have the student submit drafts with the first rough draft written in class and submitted at the end. Then weekly or daily improved drafts. If the finished paper is totally and material different then it’s a red flag. If the student wants to drastically change the paper then the teacher must approve.
“Chat GPT, this is my rough draft. I want you to polish it a little and add some, but not a lot. This is meant to be a second pass, not the final draft. Make a couple mistakes on the grammar.”
Can easily be faked with AI. You can just prompt AI to make progress, drafts, mistakes, fix the mistakes, etc.
I’ve presented on this at teacher conferences, for what it’s worth. There’s no effective way to detect AI usage accurately when the text-writing process isn’t supervised. The solutions need to accept that reality.
Then papers will all have to be written in person on an internet free machine or hand written.
that is an unrealistic solution
True, it has never been tried before.
Or we could focus on the core of teaching, which is building relationships with students. Then, with that rapport, students will trust their teachers when they explain why getting AI to do the work for them is hurting their own education. We can also change our assessment practices, so that students don’t feel the pressure to write a “perfect” essay.
And, yes; occasionally require students to do a bit of writing with invigilation.
Had trouble with this myself teaching. Students this semester have been good about it (probably because I’ve been very explicit in my contempt and also it kept blundering) but last semester was tricky.
One thing I learned was I need to also insist no Grammarly. That used to be allowed but it makes original writing sound very AI. I also riddled my assignments with short oral segments and personal stories.
It cuts into class time but I’ve managed to make those sessions educational since my “presentations” are always conversations w/ students. No ppts. Actually kinda fun and very much weeds out cheaters lol
*Words, phrases and punctuation rarely used by the average college student – or anyone for that matter (em dash included) – are pervasive. *
Hey, fuck you too >:(
This quote is particularly amusing because the author used en dashes where he should have used em dashes, while making a point about how no one uses em dashes.
Damn, I finally had to go learn what the em dash is, and by contrast what the en dash is. Your comment will haunt me to the end of my days.
And you don’t put spaces around em or en dashes.
Doesn’t MS word automatically change a regular dash to an em dash if there’s a space on either side and you keep typing the sentence?
Wonder how many false positives that’s caused.
Yeah it sure does, as does outlook.
The em dash used in MS word is different to the one used by ChatGPT as far as I can tell, it looks more like the en dash used by GPT.
I feel like I have to train myself out of using em dashes in my writing now to avoid being mistaken for AI, and it sucks.
I sometimes use different dilimiters [you know, like “(” {or also “[” or “(” (as they are used in programming – since the em dash is essentially one if these --)}].
I use the en dash – often mistaken for an em dash — quite regularly but I think it can be legitimately used to get a hunch, if he then seeks proof via hidden prompt injection and student confessions.
I use a dash all the fucking time - it proves a point and keeps the train of thought going
I’m going to need to see you passing a capcha test
I think AI would be better at captchas than me.
Oh fuck don’t tell jhex but dogs and bicycles look the same to me
That’s a regular dash. The em dash is longer.
Here’s the link to the actual article. I get that you’re trying to do users a favour to bypass tracking at the original URL, but the Internet Archive is a Free service that shouldn’t be abused for link cleaning as it costs a lot of money to store and serve all this stuff and it’s meant as an “archive”, not an ad-blocking proxy.
I’m posting this in part because currently clicking that link errors it with a “too many requests” error. Let’s try to be a little kinder to the good guys, shall we?
If users wasnt a cleaner/safer/faster browsing experience, I recommend ditching Chrome for Firefox and getting the standard set of extensions: uBlock Origin, Privacy Badger, etc.
Yeah, especially if it’s not paywalled.
It deprives the original source of traffic too, even if it’s Adblock traffic.
Any free service is bound to be exploited to the fullest possible extent. It’s the depressing fate of so many internet projects.
Fuck it. Let’s make the Internet Archive only accessible from public libraries. And you will have to physically go to a library to access it. No accessing the archive through your library’s website.
I could also be convinced to make the Internet Archive only accessible from a series of elaborate temples we build just for this purpose.
Regardless of the method, the point is that the Internet Archive still exists and serves its core purpose. It loses some convenience of scholarly access, but in turn it now becomes useless as a paywall bypass mechanism.
Damn it sucks that the system that says you need a diploma to not be an impoverished slave your whole life encourages people to get the diploma but not to care how they get it.
I’ve been using it for a personal project, and it’s been wonderful.
It hasn’t written a word for me. But it’s been really damn helpful as a research assistant. I can have it provide lists of unexplained events by location, or provide historical details about specific things in about 5 seconds.
And for quicky providing editing advice, where to punch up the language, what I can cut, or communicate more clearly. And I can do that without begging a person for days to read.
Is it always perfect? Not at all, but it definitely helps overall, when you make it clear to be honest, and not sugar-coat things. It’s definitely mostly mediocre for creative advice, but good for technical advice.
It’s a tool, and it can be used correctly, or it can be used to cheat.
Do you then check those historical details against trusted sources? If so, how often do they need correction?
And the issue is that for the people who call for a Butlerian Jihad, we are part of the problem.
when you make it clear to be honest
It has no idea what honesty is. It has no idea what bias is.
It is fancy auto-complete. And it’s wrong so often (like 40% of the time) that it should not be used to seek out factual information that the prompter doesn’t already know.
it should not be used to seek out factual information that the prompter doesn’t already know.
Eh… Depends on the importance and purpose of the information.
If you’re just trying to generate ideas for fiction from historical precedents, it doesn’t matter if it’s accurate. Or if you’re using it as a starting point, then following the links to check the original source (like I do all the time for Linux terminal commands).
Hell, I often use Linux terminal commands from Google’s search results AI box—I know enough to be able to parse what AI is suggesting (and identify when the proposed commands don’t make sense), and enough to undo what I’m doing if it doesn’t work. Saves a lot of time.
Copilot fixed some SQL syntax issues I had yesterday, too. 100% accuracy on that, despite it being a massive query with about a dozen nested subqueries. (Granted, I gave a very detailed prompt…) But, again, this was low stakes–who cares if a SELECT query fails to execute.
Exactly, and yes it’s for historical fiction, not a history book. So if it gets some details wrong about Spanish colonial forts, or Queen Anne’s Royal Court, and the role my witches may have had in it, I think I’ll survive.
But if I had to do all this research, or seek out editing help myself, I’d be on page 5, instead of 67, and my story wouldn’t be anywhere near as tight as it is.
Idiocracy is locked in. Rhey all should be expelled.
I heard of something brilliant though: The teacher TELLS the students to have the AI generate an essay on a subject AND THEN the students have to go through the paper and point out all the shit it got WRONG :D
Brilliant
This is discussed in the article
Interesting post, would be good to support the author/publisher with a source link. Especially since it isn’t pay walled.
Great story with predictable results. Welcome to your AI future where people give their thinking over to machines made by sociopaths.
Let me tell you why the Trojan horse worked. It is because students do not know what they do not know. My hidden text asked them to write the paper “from a Marxist perspective”. Since the events in the book had little to do with the later development of Marxism, I thought the resulting essay might raise a red flag with students, but it didn’t.
I had at least eight students come to my office to make their case against the allegations, but not a single one of them could explain to me what Marxism is, how it worked as an analytical lens or how it even made its way into their papers they claimed to have written. The most shocking part was that apparently, when ChatGPT read the prompt, it even directly asked if it should include Marxism, and they all said yes. As one student said to me, “I thought it sounded smart.”
Christ…
Yeah, those students deserve to fail.
I assume he taught what Marxism is in the class, yeah?
No, it was irrelevant to the class
No, it was irrelevant to the class
This is university, not high school or college.
They should’ve checked it themselves, realised that it was ridiculous and either removed it, or at least asked the prof if that’s supposed to be there.
Well that’s kinda fucked then.
Maybe you need his class lmao. Re read the article and have a think about what it means.
So he fails people who are Marxist because they follow directions and have actually educated themselves on something?
It’s a history class ffs. We should be teaching Marxism anyway
I legitimately think that you need to work on reading comprehension or let yourself read things without just getting too pissed based on trigger phrases you decided on to think about what you read. He asked all the students who submitted Marxist papers about their perspectives and none of them could even explain what Marxism was. I implore you to step back and think through this rationally - how many US college freshmen have strong Marxist views and are knowledgeable and passionate enough about them to write a strong essay about how an event in American history fits within a Marxist perspective? I also invite you to consider how a professor who wants to “fail people who are marxist” would recognize Marxist language and rhetoric subtly inserted into an essay? Furthermore, THE INSTRUCTIONS TO WRITE FROM A MARXIST PERSPECTIVE WERE IN WHITE TEXT THE STUDENTS COULDN’T READ UNLESS THEY COPY AND PASTED THEM INTO A DIFFERENT TEXT FIELD! I don’t understand how you could supposedly read this article multiple times and not understand this crucial detail.
If you didn’t make it to your freshmen year without learning what Marxism is, you probably shouldn’t be in Uni
Yes, most educated US Americans learn about Marxism at some point in high school
THE INSTRUCTIONS TO WRITE FROM A MARXIST PERSPECTIVE WERE IN WHITE TEXT THE STUDENTS COULDN’T READ
The fuck are you yelling about. That doesn’t make sense. Obviously it could be read from the assignment paper. Otherwise, it wouldn’t be read by the students. Which, the article made very clear, it was read by the students.
All I read there was. That they could not define Marxist perspectives correctly, which when not part of your specific field is rather challenging
A lot of this comment section does apparently lol
that was the trap.
I don’t get it. I try to see everything in life through Marxist and feminist lens.
Yes, this Trojan horse could only be done when Marxism is a foreign concept
I get it about the hidden text and the only students who used it were tricked.
But, it’s sad
Then they whine on Reddit when they can’t get a job.
". My hidden text asked them to write the paper “from a Marxist perspective”
Freshmen.
That’s a dangerous proof.
He could have said to write from a zagnoore brandle-frujt perspective. Some would have scanned the assignment, ignored the part they didn’t understand, and kept chooching right along. Many students would rather try to figure it out than sound stupid in class or risk the spotlight of social interaction.
Interrogating each of them on the material is the only safe way.
Many students would rather try to figure it out than sound stupid in class or risk the spotlight of social interaction.
This ignores the entire premise of the experiment. The 39% were not interested in learning regardless of the consequences.
Honestly interrogating each one on the material would be best but costly. Some classes have 80 students and professors have more than one class.
Also rest of the article says he asked students objecting accusations to tell what Marxism is and they admitted afterwards.
Apparently chatgpt asked “are you sure you want marxist perspective” as topic is older and not that directly related to marxism. One said they picked yes because quote on quote “it sounded smart.”
Also rest of the article says he asked students objecting accusations to tell what Marxism is and they admitted afterwards.
Yes, I read it. All of it.
He definitely caught a number of them, but he called it proof, it should NOT be treated as proof, an indicator at best. If it were proof he could just fail them all and not catch false positives.
Totally agree about the number of people to interview being expensive. But it is more adequate as proof.
What he didn’t do wasn’t wrong, but he can’t count on that to be a point to fail.
He definitely caught a number of them, but he called it proof, it should NOT be treated as proof, an indicator at best. If it were proof he could just fail them all and not catch false positives.
So you frequently wrote your papers from a Marxists perspective randomly when it had no relevance to the topic at hand? He hid the text by making it the same color as the background in the assignment so only when one copy and pasted the assignment or attached the file to their prompt would it be picked up. The only real false positives would be those staunchly Marxists students or someone using a screen reader. Which I think if you are inserting Marxism into random essays that are not relevant you probably are going to be bringing it up constantly in every setting you can and would also be able to explain it. It definitely is a reason to be failed on an assignment if he caught you with this.
The idiom you’re looking for in your last sentence is quote, unquote
Don’t do this to me.
(Jokes aside thank you)
No shame at all, just wanted to let you know!
The Trojan worked because the students who read the assignment would not have seen the reference to Marxism. Only by copy pasting the text in to another field would that show up.
You’re assuming the freshmen would recognize a reference to marxism and not ignore that part because they didn’t understand. It’s what inexperienced students do
I’m not assuming a damn thing. He literally made it impossible to see the reference to Marxism unless you copy pasted the text in to another field. Like you would do if you were feeding it to an AI. Think very small white font.
He made it hidden text in the assignment, i.e im guessing he made the text the same color as the background. Only when one copied the text and pasted it as instructions for a prompt would it be seen. Hence it was a Trojan horse on the assignment where those not using ai blatantly would not find it.
The article mentions that he gave them a chance to explain why they chose to write it from a Marxist perspective and none of the students even knew what Marxism was.
He gave students an out in the event they actually did write from a Marxist pov
