Chat GPT and Outlining

[Shepard]

I'm a history teacher that lives in constant fear that my students will use Chat GPT to write their essays and papers. There are some easy fixes... I've abandoned the conveniences of Google Classroom and reverted back to in-class essay writing since the beginning of the school year. However, for obvious reasons, I can't have them write their research papers in class. College Board has weighed in on the subject (pasted below), suggesting a series of checkpoints with students to assess the authenticity of their work. Susan Wise Bauer came to town recently and spoke about outlines, and the importance of outlines as a means of scaffolding writing. Not only do my students need to work with these, but they might make an excellent means of checking in with them to ensure writing authenticity. Is anyone else hearing about good ways to offset A.I. shenanigans? Any ideas on where I might find good outline strategies/resources? 

Thanks,

Chris 

ps- If your interested, I've added College Board's language re: A.I. below: 

Students can use generative AI tools as optional aids for exploration of potential topics of inquiry, initial searches for sources of information, confirming their understanding of a complex text, or checking their writing for grammar and tone. However, students must read primary and secondary sources directly, perform their own analysis and synthesis of evidence, and make their own choices on how to communicate effectively both in their writing and presentations. It remains the student’s responsibility to engage deeply with credible, valid sources and integrate diverse perspectives when working on the performance tasks...

 

To ensure students aren’t using generative AI to bypass work, we’ll require students to complete interim "checkpoints" with their teacher to demonstrate genuine engagement with the tasks. This builds on and formalizes existing practice for Seminar and Research students.

• In AP Research, students are already required to complete “checkpoints” in the form of in-progress meetings and work in the Process and Reflection Portfolio (PREP). No further checkpoints will be required. • In AP Seminar, teachers will assess the authenticity of student work based on checkpoints that will take the form of short conversations with students during which students make their thinking and decision-making visible (similar to an oral defense). These checkpoints will occur during the sources and research phase (IRR and IWA), and argument outline phase (IWA only). A final validation checkpoint (IRR and IWA) will require teachers to confirm the student’s final submission is, to the best of their knowledge, authentic student work.

[ieta_cassiopeia]

Welcome to the Well-Trained Mind forum! Outlines are an excellent idea, not just for avoiding AI use, but for helping build students' capabilities to write their own work.

Look for hallucinated text - facts that are wrong in ways that students of this age group are unlikely to get wrong (especially given what the students already do or should know). An example of what hallucinated writing looks like can be seen on the Interactive Fiction forum (someone asked Chat-GPT to give a walkthrough of a famous game and it "confidently" generated text full of invented rooms, items and objectives). To give a more history-related example:

If a student, responding plausibly to an essay question, is going into 5 pages of detail about the American Revolution with lots of details that make superficial sense, the essay probably shouldn't mention that Denzel Washington commanded the Continental Army. (For anyone reading this who doesn't recognise the name, Denzel is a 20th-century actor/producer/director, thus such a "fact" would be impossible).

Said mistake should not occur on page 3, after associating George Washington with that position for the previous 2 pages. (A student writing their own work is more likely to either make a typo or substitute a name that is vaguely similar to the previously-used one).

The student's work should not indicate this basic error has been printed in any credible source the student would consider valid to cite (unless this is a university class and this is an essay where low-credibility sources are permitted).

The essay really shouldn't claim that such an erroneous statement came from a scholarly book or research paper that doesn't exist (complete with reference that doesn't tally with anything findable using a search engine search or library catalogue check). Artificial intelligence is currently abysmal at citing sources that exist and match the quotes/material attributed to them. It often gets basic facts wrong about areas where it can generate plausible (not always accurate) information about more detailed exercises.

Check all graphs and charts correspond with the tables that generated them and the text they are meant to support - AI often gets the relationships between the three elements wrong.

Consider having several really clear instructions in key assignments, rather than just one vaguely-worded one. For example, "Describe and explain three causes of the Great Depression, with reference to at least two primary sources" is a better request than "Explain why the Great Depression happened". Where students can choose a thesis, similarly ask them to be specific on what the question will be. The more specific the request, the less likely it is that a large language model will succeed in a way that fools you or indeed the student. Better still, agree a definition of command words with other staff members at the school - AI can't interpret organisation-specific definitions of command words very well at this point because it depends on dictionaries and general user experience.

Creative requests (like writing a letter as if from someone involved in a historical event) are more likely to generate plausible nonsense, but also more likely to reveal that an artificial intelligence wrote it rather than the student (as it is less likely to match the student's pre-existing style).
 

If you have a large class (more than a couple of dozen students), try to set different tasks to different students. For example, you could ask half the class to explain the causes of a decision and the other half its consequences (swapping them next essay, whether sticking to the same general topic or using the next one in the syllabus). Having a third of the class do an essay about each of three consecutive presidents would also work, especially if the students will later present their findings to their classmates. The reason is that if dozens of students put in the same request, the AI can learn to make something more likely to trick the student into thinking the generated work might be accepted.

 

Also, if possible, teach at least some of this to the students so they are alert to signs of adulterated writing, and that large language models are not as competent as their websites/apps/companies present them as being. Dishonest people increasingly try to pass off AI papers as genuine research because it's easier than writing genuine research. The more people learn how to spot AI bloopers, the fewer of them will cause problems, and the easier your job will be with the people you've taught to avoid over-dependence on AI.

Edited December 26, 2023 by ieta_cassiopeia