Half of social-science studies fail replication test in years-long project

Key Points:

• The seven-year SCORE project analyzed 3,900 social-science papers and found that only about half of the studies could be successfully replicated, highlighting ongoing concerns about reproducibility in social sciences.
• The project involved 865 researchers examining papers across multiple fields, revealing that many studies lacked sufficient data or methodological detail to enable accurate replication, though some irreproducibility may stem from honest errors or novel analyses.
• SCORE assessed reproducibility (repeating analyses on the same data), robustness (alternative analyses yielding consistent conclusions), and replicability (repeating entire experiments), finding reproducibility at 53%, robustness at about 75%, and replicability at 49%.
• The findings underscore the need for greater transparency, open data sharing, and adoption of new tools like multiverse analysis to improve the reliability of scientific results.
• Experts caution against taking individual studies at face value, advocating for skepticism and reliance on multiple corroborating studies before accepting findings as robust.