I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.

I rely on written exams that are open book, but forbid any use of computers and smartphones in class.

The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.

I just had an experience where all students were given "sample problems" to try at home and prepare for the written test. Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.

It seems like the university also has many people pushing for AI use for everything, but I teach basic stuff where the goal is to make students think on their own and digest some fundamental ideas, LLMs can produce perfect solutions, but relying on them is pointless.

The easiest path is to allow students to cheat on homework and watch them fail in the non-coursework-based exam. Your classes, then, abandon to the un-motivated; with AI enlarging the educational gap.

If you’re forced to attain a pass rate, and can’t watch so many fail, then the certificate becomes worthless in the workplace. Or just offer paid retakes and watch AI make you money.

If you insist on persisting with coursework then a short but heavily-weighted oral part seems to be the only answer, something that will diminish teaching time as a downside, as the article and comments about the flipped classroom touched on.

> For example, the evidence favors frequent low-stakes assessments with feedback (e.g., homework, quizzes) over few high-stakes ones (e.g., exams) – but AI is undermining practice in low-stakes settings and pushing us more toward exams.

Why would AI undermine quizzes? I took an intro to Japanese class where we would have a quiz at the beginning of each class which I quite enjoyed. I can’t see how AI would change the utility of that.

Heh, dumb idea of the day: do a "flipped classroom" in a different sense -- instead of a student taking a quiz or exam, you provide a clueless AI and the student has to train it on the subject at hand, and then the AI takes the test.

Lest anyone think this is a brilliant idea: an LLM would likely be much better at doing the ~~RLHF~~ RLMF, so you haven't actually gained anything. But I feel like there may still be the kernel of a good idea somewhere in here.

Perhaps your assignment is to iteratively train the AI, which is pre-prompted to seek out every reasonably possible failure mode when it generates solutions. So the learning mechanism is to train an adversarial AI on the subject matter as a way of learning it yourself. This does not solve the problem of cheating with an LLM, it's an alternative pedagogical approach.

Honest question: doesn't this highlight how mundane and rote teaching has become? It's been distilled to the point that the lowest of the lowest common denominators can pass. The kids going through it are forced to learn things they will never use, unless it's going to be used in the more advanced class of the thing they'll never use until they graduate.

My hope is that this is a wake up call to teachers / professors to find a better way to evaluate students. To determine who has actually come away with a real understanding, vs those who were running around copying the assignments off of their peers anyway. These same people are now just skipping the peers and going straight to the all knowing oracle. It's the same as it ever was, degree mills.