Historically, I have used Medieval disputatio as a method of teaching my high school philosophy students to summarize and curate the best arguments from the history of philosophy and to write their own logically valid arguments in defense of a philosophical position. The method works really well, and it accomplishes the goals of two different approaches to philosophy: the historical approach and the topical approach.  Over the years I’ve been refining this approach and presenting on it to other teachers at conferences. I even helped lead an NEH workshop in the summer of 2021 on how this method can be applied to other fields in the humanities aside from philosophy.

With the advent of ChatGPT-3 and other large language models (LLM’s), I was curious how well AI’s could handle such a complex task. It turns, they can do it fairly well, at least as well as most of my high school students could. Of course, that presents a challenge to me as a teacher. Do I keep doing the assignment the same way, pleading with my students not to use these new tools and hoping their conscience and their fear of getting caught outweigh their desire to earn high marks while expending minimal effort? Instead, I decided to continue using the disputatio method to accomplish a narrower set of goals, and then I modified the assignment and the rubric to incorporate student use of ChatGPT-3.

Assignment Goals

My goals for this assignment are as follows:

1. Help students review the major questions and arguments they have encountered over the course of the year
2. Expose students to what large language models like ChatGPT-3 can and cannot do
3. Push them to discover and develop their specific human abilities, the things they can do better than the LLM

It is important to note that I’m not, with this assignment, training my students to:

1. Write skillfully on their own
2. Reconstruct arguments from their reading assignments
3. Construct their own arguments by themselves
4. Perform traditional research practices

I have other assignments for developing and assessing those skills, and I am concerned, like most teachers are, about how we will teach students those important skills in the era of large language models. This assignment is coming near the end of a year-long course in which students have read primary and secondary texts, written summaries of their reading assignments without help from ChatGPT-3, discussed ideas with their classmates, heard lectures from me and from other philosophers like Arthur Holmes and Peter Millican on YouTube, and done other projects like writing blog posts and essays, making their own podcasts, creating philosophy explainer videos, and teaching philosophy to a group of fifth grade homeschooled kids.

Task

Using ChatGPT-3Links to an external site. (to keep things fair, please do not use ChatGPT-4 or other LLM’s), write a series of five disputed questions (Articles) on one of the following topics (Questions), incorporating the major arguments from the history of philosophy up to Hume.

1. Freedom and Determinism – Libertarianism, determinism, compatibilism?
2. The Existence and Nature of God – Does God exist? Can we prove it? What are God’s attributes? What is God’s relationship to space and time?
3. Epistemology – What is knowledge? How do we acquire it? What are its limits?
4. Philosophical Anthropology – What is the Nature of Human Beings? Do we have souls or not? Can we persist beyond our deaths? Personal identity over time
5. The Nature of Reality – Material? Immaterial? Some combination?
6. The One and the Many – Are universals real or constructed? Do they exist transcendently, immanently, or in our minds?

Instructions on Process Documentation

As you construct your disputatio with help from ChatGPT-3, document your process using a copy of this table. Copy and paste it for subsequent articles. 

1. Under “Prompt,” copy and paste the prompt you submitted to ChatGPT-3.
2. Under “Goal/Rationale for Prompt” give your justification for feeding ChatGPT-3 that specific prompt.
3. Under “Output” copy and paste what ChatGPT returned in response to the prompt.
4. Under “Evaluation of Output” document your thoughts on how successfully ChatGPT performed the task, whether you need to adjust the prompt or feed ChatGPT-3 follow-up prompts to get it to refine the answer or write alternative responses. 
5. Under “Strategic Response/Action” explain what you will do next in light of your overarching goal and your evaluation of the most recent output.

Disputatio Format

Question: [your chosen topic goes here]

Article [article number goes here]: [the question you will answer goes here]

It would appear that…

Objection 1: [argument from philosopher A]

Objection 2: [argument from philosopher B]

Objection 3: [argument from philosopher C]

Objection 4 : optional [argument from philosopher D]

Objection 5: optional [argument from philosopher E]

On the contrary…

[Argument from authority goes here]

I answer that…

[Your multi-paragraph argument goes here]

Reply to Objection 1:

Reply to Objection 2:

Reply to Objection 3:

Reply to Objection 4:

Reply to Objection 5:

Reflection Questions

Additionally, you’ll write a personal reflection on your experience writing your disputatio in collaboration with ChatGPT-3. This reflection should be written without help from ChatGPT-3 and should answer the following questions:

1. Did you enjoy this assignment? Why or why not?
2. What was ChatGPT-3 able to do surprisingly well?
3. What were ChatGPT-3’s limitations?
4. What specific human capacities did you employ to contribute to the collaborative project?
5. How do you think large language models might impact your education, career, relationships, and daily life in the future?

Rubric

The rubric I’ve given my students for this assignment contains a lot more information than I will share here, but here are the highlights:

• Process documentation – 40 points
  • Full marks requires exceptional process documentation, prompt-engineering, decision-making, evaluation of outputs, refinement of outputs. Student is incredibly thorough and uses human judgement to vastly improve upon what ChatGPT-3 could do in response to several prompts written by a capable human prompt engineer.
• Finished disputatio – 40 points
  • To earn full marks, finished disputatio must be vastly superior to what a non-philosophical but capable human prompt engineer could write and on par with what a graduate student philosopher could generate on his or her own. Objections are logically valid, concise, and accurate representations of the philosophers’ positions and their reasons for those positions. Objections selected are among the strongest counter-arguments to the position ultimately taken by the author. Citation of authority clearly and powerfully counters the objections. Author’s own argument is logically valid and contains strong argumentation and evidence for each premise of the core argument. Argument displays a level of sophistication typically expected from graduate students in philosophy. Replies to objections likewise present clear and concise arguments that persuasively cast doubt upon at least one premise in the objection and/or make a careful distinction to show why the objection is only partially correct.
• Reflection questions – 15 points
• Proper disputatio formatting – 5 points