CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity

Post-training typically boosts LLM quality but sacrifices output diversity and creativity, hurting both explicit creative tasks like story writing and implicit ones like reinforcement learning exploration. CreativeInstruct addresses this by teaching models to inject special markers that bias generation toward creativity while preserving post-trained quality, eliminating the need for multiple models at inference. The authors also propose a structural diversity metric using graph edit distance to capture narrative-level variation. Applications include narrative and creative writing tools that need genuine variety, and RL pipelines where creative base models serve as better starting points—demonstrated by gains on math reasoning benchmarks like AMC and MATH. Paper: https://arxiv.org/abs/2608.07460