May 5
Math Across the Cannon: Jennifer Hill
Tue, May 5, 2026
• 6:00pm
- 7:00pm (1h) • Weitz Cinema
Speaker: Jennifer Hill, Professor of Applied Statistics at New York University Steinhardt
Title: ChatGPT is not a PhD in your pocket
Abstract: The rapid rise of Large Language Models (LLMs), colloquially referred to as artificial intelligence (AI), has raised substantial economic, technical, and social questions. Due to their size and power demands, the development and deployment of these models carry substantial economic costs and have environmental consequences. The staggering cost of LLM development and deployment is justified by the promise that LLMs will substantially augment or replace human knowledge-economy labor. At face value, these findings make the automation of vast sectors of the knowledge economy appear all but inevitable. However, a growing body of evidence suggests that the capacity LLMs have been severely overstated in part because most of the assessments suffer from "benchmark contamination" -- that is, the LLMs being tested had access to the "answer key." I will describe two studies our lab has performed to understand (1) the performance of LLMs on standard tasks in causal inference and (2) how students can be influenced by information provided by an LLM. We discuss the implications of our findings for how we should think about the role of LLMs in a data analytic workflow.
This event is sponsored by The St. Olaf Department of Mathematics, Statistics, and Computer Science; The Carleton Department of Mathematics and Statistics; the TCL Colloquium Fund; and the Michael Morrill Fund.
Event Contact: Sue Jandro