While Humor Understanding has been the focus of many shared tasks, Humor Generation remains an even more challenging and largely unexplored frontier. The HumorGen task at SemEval 2026 is the first of its kind, dedicated to advancing the state-of-the-art in computational humor generation. We invite participants to develop systems capable of generating genuinely funny content under a variety of constraints.
Our goal is to push models beyond memorization and towards true humorous creativity. By using carefully designed constraints, we aim to ensure fairness in evaluation and encourage the generation of novel jokes. This task has significant implications for more engaging conversational AI, creative writing tools, and a deeper understanding of the complex nature of humor itself.
News
- July 15th, 2025: This website was created and the sample data was released.
All dates are 23:59 UTC-12h ("anywhere on Earth").
Subtasks
Please Note:
The constraints listed and the subtasks' conditions below are still subject to change.
Subtask A: Text-based Humor Generation
Given a set of text-based constraints, generate a joke. This subtask will be conducted in English, Spanish, and Chinese.
Constraints:
Each generated joke must follow a combination of three constraints, designed to make it difficult to simply retrieve existing jokes from the web. Constraints include:
- Word Inclusion: Must contain a specific word (from a list of 100 words).
- Style: Must adhere to a specific style (Dialogue, Anecdote, Q&A, or Archaic Language).
- Topic: Must be about a specific topic (Sports, Tech, Travel, Education, Work, Daily Life, or Entertainment).
Subtask B: Multimodal Humor Generation with Images
This subtask explores humor in a multimodal context, combining visual inputs with text generation. This subtask is in English only.
Image-Based Caption Generation
Given an image, generate a free-form humorous caption (max 20 words) that enhances its comedic effect.
Data and Resources
No labels
In line with the task's focus on genuine generation over memorization, and given the diversity in Humor and the hardship of evaluating jokes, we will not provide labeled data; we will provide only inputs. Participants are encouraged to use any publicly available data, pre-trained models, API, or rule-based systems.
Sample Data
To help participants understand the task format and constraints, we are providing a sample dataset. We provide 100 samples for Subtask A and a version in Spanish. For Subtask B, we provide 20 samples, which use constraints as in Subtask A, instead of the conditions currently specified for Subtask B.
Sample Data for Subtask A (JSONL) Sample Data for Subtask A in Spanish (JSONL) Sample Data for Subtask B (JSONL)Evaluation
Evaluation will be based on human preference judgments. We will use a pairwise comparison setup ("battle"), where annotators choose the funnier of two generated texts produced under the same conditions. We will use a web interface inspired by Chatbot Arena to crowdsource annotations from anybody on the Internet. The systems will be ranked using an Elo-based leaderboard.
Important Dates
- "Training" data release: September 1, 2025
- Evaluation period starts: January 10, 2026
- Evaluation period ends: January 31, 2026
- System description paper submission: February 28, 2026
- Notification of acceptance: March 31, 2026
- Camera-ready papers due: April 30, 2026
- SemEval 2026 Workshop: July 2026
All dates are 23:59 UTC-12h ("anywhere on Earth").
Organizers
Universidad de la República / Netflix
Universidad de la República
University of Edinburgh
Universidad de la República
University of Michigan
University of Michigan
Netflix
Universidad de la República
Universidad de la República
Universidad de la República
Universidad de la República
Contact
For all inquiries, please join our official Google Group or email the organizers.