Oggetti

Descrizione

dta reg (aaaa-mm-gg)

Body

Large language models (LLMs) are rapidly transforming social science research by enabling the automation of labor-intensive tasks like data annotation and text analysis. However, LLM outputs vary significantly depending on the implementation choices made by researchers (e.g., model selection or prompting strategy). Such variation can introduce systematic biases and random errors, which propagate to downstream analyses and cause Type I (false positive), Type II (false negative), Type S (wrong sign for significant effect), or Type M (correct but exaggerated effect) errors. We call this phenomenon where configuration choices lead to incorrect conclusions LLM hacking.

We find that intentional LLM hacking is strikingly simple. By replicating 37 data annotation tasks from 21 published social science studies, we show that, with just a handful of prompt paraphrases, virtually anything can be presented as statistically significant.

Beyond intentional manipulation, our analysis of 13 million labels from 18 different LLMs across 2,361 realistic hypotheses shows that there is also a high risk of accidental LLM hacking, even when following standard research practices. We find incorrect conclusions in approximately 31% of hypotheses for state-of-the-art LLMs, and in half the hypotheses for smaller language models. While higher task performance and stronger general model capabilities reduce LLM hacking risk, even highly accurate models remain susceptible. The risk of LLM hacking decreases as effect sizes increase, indicating the need for more rigorous verification of LLM-based findings near significance thresholds. We analyze 21 mitigation techniques and find that human annotations provide crucial protection against false positives. Common regression estimator correction techniques can restore valid inference but trade off Type I vs. Type II errors.

Overall, our findings advocate for a fundamental shift in LLM-assisted research practices, from viewing LLMs as convenient black-box annotators to seeing them as complex instruments that require rigorous validation. Based on our findings, we publish a list of practical recommendations to prevent intentional LLM hacking and limit the risk of accidental LLM hacking.

tipo ogg.

cat

subcat

proprietario

azienda

ute

selez1

op1

codart

codice

codclifor

motivo

url

tag

imgsrc

sunto_breve

sunto_medio

note

keyscrape

Lista tag dell'oggetto (oggettag)
cod_tag	value
ai	5
hacker	5
llm	5
llm_internal	5
wpg	5

Lista delle righe dell'oggetto (oggettir)
tip_pro	cod_pro	descrizione	val_pro	pes_pro

---.