Skip to main content

Spring Deadline: Sunday, March 1 @ 11:59pm PT. Click here to apply.

Back to Research
Accepted to CHum @ COLING 2025

Pragmatic Metacognitive Prompting Improves LLM Performance on Sarcasm Detection

Joshua Lee, Wyatt Fong, Alexander Le, Sur Shah, Kevin Han, Kevin Zhu

Abstract

Sarcasm detection is a significant challenge in sentiment analysis due to the nuanced and context-dependent nature of verbiage. We introduce Pragmatic Metacognitive Prompting (PMP) to improve the performance of Large Language Models (LLMs) in sarcasm detection, which leverages principles from pragmatics and reflection helping LLMs interpret implied meanings, consider contextual cues, and reflect on discrepancies to identify sarcasm. Using state-of-the-art LLMs such as LLaMA-3-8B, GPT-4o, and Claude 3.5 Sonnet, PMP achieves state-of-the-art performance on MUStARD and SemEval2018 benchmarks.

Citation

Joshua Lee, Wyatt Fong, Alexander Le, Sur Shah, Kevin Han, Kevin Zhu. "Pragmatic Metacognitive Prompting Improves LLM Performance on Sarcasm Detection". Accepted to CHum @ COLING 2025.

Details

Conference
Accepted to CHum @ COLING 2025
Authors
6 authors

Publish Your Research

Join Algoverse and work with world-class mentors to publish at top AI conferences.

Start Your Application