LOGISTAR — Scheda Oggetto

Dati oggetto

URL

Apri

Titolo / Descrizione

Sito / Azienda

Estratto / Body

Large language models (LLMs) are rapidly transforming social science research by enabling the automation of labor-intensive tasks like data annotation and text analysis. However, LLM outputs vary significantly depending on the implementation choices made by researchers (e.g., model selection or prompting strategy). Such variation can introduce systematic biases and random errors, which propagate to downstream analyses and cause Type I (false positive), Type II (false negative), Type S (wrong sign for significant effect), or Type M (correct but exaggerated effect) errors. We call this phenomenon where configuration choices lead to incorrect conclusions LLM hacking.

We find that intentional LLM hacking is strikingly simple. By replicating 37 data annotation tasks from 21 published social science studies, we show that, with just a handful of prompt paraphrases, virtually anything can be presented as statistically significant.

Beyond intentional manipulation, our analysis of 13 million labels from 18 different LLMs across 2,361 realistic hypotheses shows that there is also a high risk of accidental LLM hacking, even when following standard research practices. We find incorrect conclusions in approximately 31% of hypotheses for state-of-the-art LLMs, and in half the hypotheses for smaller language models. While higher task performance and stronger general model capabilities reduce LLM hacking risk, even highly accurate models remain susceptible. The risk of LLM hacking decreases as effect sizes increase, indicating the need for more rigorous verification of LLM-based findings near significance thresholds. We analyze 21 mitigation techniques and find that human annotations provide crucial protection against false positives. Common regression estimator correction techniques can restore valid inference but trade off Type I vs. Type II errors.

Overall, our findings advocate for a fundamental shift in LLM-assisted research practices, from viewing LLMs as convenient black-box annotators to seeing them as complex instruments that require rigorous validation. Based on our findings, we publish a list of practical recommendations to prevent intentional LLM hacking and limit the risk of accidental LLM hacking.

Note

Voto oggetto (1-10)

Precisione (1-10)

Tag (5)

✎ Modifica tag 🤖 Suggerisci AI

          ai:5, llm:5, llm_internal:5, hacker:5        

ai 5 hacker 5 llm 5 llm_internal 5 wpg 5

ID: 68233 • Tipo: WPG • Reg: 2026-09-19 • Mod: 20/09/2026 23:08