Your browser doesn't support javascript.
loading
Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.
Landy, Justin F; Jia, Miaolei Liam; Ding, Isabel L; Viganola, Domenico; Tierney, Warren; Dreber, Anna; Johannesson, Magnus; Pfeiffer, Thomas; Ebersole, Charles R; Gronau, Quentin F; Ly, Alexander; van den Bergh, Don; Marsman, Maarten; Derks, Koen; Wagenmakers, Eric-Jan; Proctor, Andrew; Bartels, Daniel M; Bauman, Christopher W; Brady, William J; Cheung, Felix; Cimpian, Andrei; Dohle, Simone; Donnellan, M Brent; Hahn, Adam; Hall, Michael P; Jiménez-Leal, William; Johnson, David J; Lucas, Richard E; Monin, Benoît; Montealegre, Andres; Mullen, Elizabeth; Pang, Jun; Ray, Jennifer; Reinero, Diego A; Reynolds, Jesse; Sowden, Walter; Storage, Daniel; Su, Runkun; Tworek, Christina M; Van Bavel, Jay J; Walco, Daniel; Wills, Julian; Xu, Xiaobing; Yam, Kai Chi; Yang, Xiaoyu; Cunningham, William A; Schweinsberg, Martin; Urwitz, Molly; Uhlmann, Eric L.
Afiliación
  • Landy JF; Department of Psychology and Neuroscience, Nova Southeastern University.
  • Jia ML; Warwick Business School, University of Warwick.
  • Ding IL; Department of Marketing, National University of Singapore.
  • Viganola D; Department of Engineering, C4I and Cyber Center, George Mason University.
  • Tierney W; Kemmy Business School, University of Limerick.
  • Dreber A; Department of Economics, Stockholm School of Economics.
  • Johannesson M; Department of Economics, Stockholm School of Economics.
  • Pfeiffer T; New Zealand Institute for Advanced Study, Massey University.
  • Ebersole CR; Department of Psychology, University of Virginia.
  • Gronau QF; Faculty of Social and Behavioural Sciences, University of Amsterdam.
  • Ly A; Faculty of Social and Behavioural Sciences, University of Amsterdam.
  • van den Bergh D; Faculty of Social and Behavioural Sciences, University of Amsterdam.
  • Marsman M; Faculty of Social and Behavioural Sciences, University of Amsterdam.
  • Derks K; Nyenrode Business University.
  • Wagenmakers EJ; Faculty of Social and Behavioural Sciences, University of Amsterdam.
  • Proctor A; Department of Economics, Stockholm School of Economics.
  • Bartels DM; Department of Marketing, Booth School of Business, University of Chicago.
  • Bauman CW; Paul Merage School of Business, University of California, Irvine.
  • Brady WJ; Department of Psychology, New York University.
  • Cheung F; School of Public Health, University of Hong Kong.
  • Cimpian A; Department of Psychology, New York University.
  • Dohle S; Social Cognition Center Cologne, University of Cologne.
  • Donnellan MB; Department of Psychology, Michigan State University.
  • Hahn A; Social Cognition Center Cologne, University of Cologne.
  • Hall MP; Department of Psychology, University of Michigan.
  • Jiménez-Leal W; Department of Psychology, University of the Andes.
  • Johnson DJ; Department of Sociology, University of Maryland at College Park.
  • Lucas RE; Department of Psychology, Michigan State University.
  • Monin B; Graduate School of Business and Department of Psychology, Stanford University.
  • Montealegre A; Department of Psychology, University of the Andes.
  • Mullen E; School of Management, Lucas College, San José State University.
  • Pang J; Department of Marketing, Renmin Business School, Renmin University of China.
  • Ray J; Department of Psychology, New York University.
  • Reinero DA; Department of Psychology, New York University.
  • Reynolds J; Department of Psychology, Stanford University.
  • Sowden W; Department of Psychology, University of Michigan.
  • Storage D; Department of Psychology, University of Denver.
  • Su R; NUS Business School, National University of Singapore.
  • Tworek CM; HarrisX.
  • Van Bavel JJ; Department of Psychology, New York University.
  • Walco D; New York Yankees.
  • Wills J; Department of Psychology, New York University.
  • Xu X; Hainan University.
  • Yam KC; Department of Management and Organization, National University of Singapore.
  • Yang X; Department of Electronic Engineering, Tsinghua University.
  • Cunningham WA; Rotman School of Management, University of Toronto.
  • Schweinsberg M; Department of Organizational Behavior, European School of Management and Technology.
  • Urwitz M; Department of Economics, Stockholm School of Economics.
  • Uhlmann EL; Department of Organizational Behavior, INSEAD.
Psychol Bull ; 146(5): 451-479, 2020 05.
Article en En | MEDLINE | ID: mdl-31944796
ABSTRACT
To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from 2 separate large samples (total N > 15,000) were then randomly assigned to complete 1 version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same

hypothesis:

Materials from different teams rendered statistically significant effects in opposite directions for 4 of 5 hypotheses, with the narrowest range in estimates being d = -0.37 to + 0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for 2 hypotheses and a lack of support for 3 hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, whereas considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Asunto(s)
Texto completo
Imprimir
XML
PubMed Links
Buscar en Google

Texto completo: 1 Banco de datos: MEDLINE Asunto principal: Psicología / Proyectos de Investigación / Colaboración de las Masas Tipo de estudio: Clinical_trials / Prognostic_studies Límite: Adult / Humans Idioma: En Año: 2020 Tipo del documento: Article
Texto completo
Imprimir
XML
PubMed Links
Buscar en Google

Texto completo: 1 Banco de datos: MEDLINE Asunto principal: Psicología / Proyectos de Investigación / Colaboración de las Masas Tipo de estudio: Clinical_trials / Prognostic_studies Límite: Adult / Humans Idioma: En Año: 2020 Tipo del documento: Article