Reproducibility in Management Science

Date

2024-03-01

Author

Fišar, Miloš
Greiner, Ben
Huber, Christoph
Katok, Elena
Ozkes, Ali I.
Abraham, Diya
Adams, Gabrielle S.
Adbi, Arzi
Addoum, Jawad M.
Adena, Maja
Akella, Laxminarayana Yashaswy
Akey, Pat
Akmansoy, Olivier
Alban, Andres
Alexeev, Vitali
Alimov, Azizjon
Aman, Argun
Aouad, Ali
Appel, Gil
Arnosti, Nick
Arora, Kashish
Arpinon, Thibaut
Artinger, Florian M.
Arts, Joachim
Baardman, Lennart
Babutsidze, Zakaria
Bahrami, Golnaz
Banerjee, Somnath
Bao, Chenzhang
Bao, Te
Baron, Opher
Barriola, Xabier
Monteiro e Silva Barroso, Pedro
Baskin, Ernest
Batt, Robert J.
Batta, George
Bauer, Anahid
Bauman, Konstantin
Bazley, William
Becker-Peth, Michael
Begen, Mehmet
Begen, Nazire
Benoit, Sylvain
Berger, Loic
Berlin, Noémi
Berling, Lars Peter
Bernard, Anna
Bertomeu, Jeremy
Białkowski, Jędrzej
Bilinski, Pawel
Bischof, Jannis
Black, Jeffrey R.
Blunden, Hayley
Bongaerts, Dion
Bönisch, Felix
Bos, Marieke
Bosch-Rosa, Ciril
Bourjade, Sylvain
Boysen, Andrew
Brimhall, Craig
Brokesova, Zuzana
Brooks, J. Paul
Bruns, Stephan B.
Buckle, Georgia
Buenstorf, Guido
Burtch, Gordon
Bushong, Benjamin
Buti, Sabrina
Callery, Patrick
Canayaz, Mehmet
Cao, Jie
Cao, Wei
Cao, Xinyu
Carree, Martin
Castellani, Vincent
Cerasi, Yann Joel
Chang, Hannah H.
Chang, Jin Wook
Chang, Michelle
Chang, Yanru
Chaturvedi, Aadhaar
Chauvin, Jasmina
Chavez, Daniel E.
Chen, Christopher
Chen, Fadong
Chen, Josie I.
Chen, Peng-Chu
Chen, Roy
Chen, Wei
Chen, Wei James
Chen, Yuanyuan
Chen, Zepeng
Chen, Zhuoqiong
Chew, Lydia
Chhabra, Param Pal Singh
Chintala, Sai Chand
Choi, Ga-Young
Choi, Seungho
Choudhary, Vivek
Chow, Vincent Tsz Fai
Christensen, Katherine L.
Chung, Doug J.
Cinelli, Melissa
Cingl, Lubomír
Cire, Andre Augusto
Clark, Jeffrey
Clement, Jeffrey
Clithero, John
Cloléry, Héloïse
Clough, David R.
Clyde, Nicholas
Coali, Andrea
Comeig, Irene
Cook, Nikolai
Correia-Da-Silva, Joao
Costa, Elaine
Coutts, Alexander
Cribben, Ivor
Cuculiza, Carina
Cui, Zimeng
Cunningham, Colleen
Cziraki, Peter
Dagorn, Étienne
Dai, Rui
Dana, Jason
Danks, Nicholas Patrick
Darendeli, Alper
Dato, Simon
Davcik, Nebojsa
de Grazia, Charles
De Sousa, Jose
De Vries, Jelle
De Vries, Martijn
Deev, Oleg
DeFronzo, Ryan
Dekker, Lennart
Delarue, Arthur
Demiral, Elif E.
Demiroglu, Cem
Deore, Aishwarrya
Detzel, Andrew
Devonaev, Azamat
Bala, Archana Dhinakar
Dimant, Eugen
Dimmery, Drew
Dimmock, Stephen G.
Ding, Cheng
Ding, Likang
Ding, Tingting
Ding, Yuheng
Dong, Lu
Donohue, Karen
Drichoutis, Andreas
Du, Shaoyin
Duan, Ying
Duevski, Teodor
Duong, Huu Nhan
Ederhof, Merle
El Hajj, Hussein
Ellison, Martin
Eriksen, Jonas Nygaard
Espinosa, Miguel
Fallucchi, Francesco
Fang, Xiaohua
Fanghella, Valeria
Faralli, Matilde
Farham, Saleh
Fattinger, Felix
Feiereisen, Stephanie
Feng, Yiding
Ferracuti, Elia
Filippin, Antonio
Fillon, Adrien
Fiorin, Stefano
Fisher, Geoffrey
Fisher, Matthew
Flath, Christoph
Foerderer, Jens
Frey, Vincenz
Fuchs, Christoph
Fugger, Nicolas
Gabel, Sebastian
Gaessler, Fabian
Ganglmair, Bernhard
Gangwar, Manish
Ares, Pedro Angel Garcia
Garg, Rajiv
Gaspar, José Miguel
Gastaldi, Chiara
Gauriot, Romain
De Genaro, Alan
Geng, Yuxin
Georgalos, Konstantinos
Geraldes, Diogo
Gerhards, Leonie
Gerken, William
Gibson, Mike
Gijsbrechts, Joren
Goerg, Sebastian
Goetz, Daniel
Goldman, Jim
Gonschorek, Filip
Gonzalez-Jimenez, Victor
Goossens, Jorgo T.G.
Gordy, Michael
Gorny, Paul M.
Goswami, Indranil
Goyal, Amit
Goyenko, Ruslan
Grad, Tom
Greenblatt, Wesley
Gregor, Martin
Grieco, Daniela
Grieder, Manuel
Grossmann, Max R.P.
Grüner, Sven
Guha, Sreyaa
Guo, Audrey
Guo, Gang
Guo, Haihao
Guo, Lewen
Gutt, Dominik
Gygax, André F.
Hacamo, Isaac
Haeckl, Simone
Hagenberg, Thomas C.
Hagmann, David
Haislip, Jacob
Han, Eojin
Han, Jiatong
Harvey, Joseph Earle
Havrylchyk, Olena
Hazarika, Sonali
He, Leshui
He, Yuhang
Hedgcock, William
Heimbach, Irina
Henderson, Brian
Hendrikse, Jurian
Henry, Erin
Hepfer, Bradford
Hernan, Roberto
Herz, Holger
Heyes, Anthony
Hildebrand, Christian
Hillenbrand, Adrian
Hillert, Alexander
Hilweg, Michael
Hjalmarsson, Erik
Hoelscher, Seth
Hoffmann, Peter
Hollenbeck, Brett
Holtrop, Niels
Holzmeister, Felix
Homroy, Swarnodeep
Hossain, Mallick
Houf, Leon
Howell, Taeya
Hu, Kejia
Huang, Allen
Huang, Jing-Zhi
Huang, Lingbo
Huang, Sterling
Huber, Stefanie J.
Hudja, Stanton
Humphrey, Jacquelyn
Hünermund, Paul
Hurst, William Reuben
Hurtado, Carlos
Huynh, Kim P.
Hyndman, Kyle
Ingolfsson, Armann
Ipeirotis, Panos
Israeli, Ayelet
Ivashchenko, Alexey
Jabr, Wael
Jain, Pankaj K.
Jaramillo-Gutierrez, Ainhoa
Javadinarab, Nahid
Ji, Yonghua
Jia, Mofei
Jiang, Hansheng
Jiang, Houyuan
Jiang, Jiashuo
Tan, Jingdan
Jirásek, Michal
Julio, Brandon
Jung, Heejung H.J.
Marcel te Kaat, Daniel
Kalodimos, Jonathan
Kamstra, Mark
Kang, Hyo
Kang, Qiang
Kanimian, Salpy
Kapons, Martin M.
Karmaziene, Egle
Kausar, Asad
Kelly, Patrick J.
Kesavan, Saravanan
Khadjavi, Menusch
Khobzi, Hamid
Khubulashvili, Robizon
Kim, Alex G.
Kim, Byungyeon
Kim, Chungyool
Kim, Dong Soo
Kim, Sehoon
Kim, Seojin
Kim, Seung Hyun
Kim, Soohun
Kirneva, Margarita
Kiss, Andrea
Kluppel, Leonardo Mayer
Koçak, Özgecan
Kogler, Christoph
König-Kersting, Christian
Kopányi-Peuker, Anita
Koppel, Lina
Koppman, Sharon
Kopsacheilis, Orestis
Kornish, Laura J.
Krahn, Anne
Krčál, Ondřej
Krishnamurthy, Srinivasan
Kropp, Philipp
Kundu, Santanu
Kurschilgen, Michael
Kusterer, David J.
Kutuk, Samet
Kuzmina, Olga
Kyung, Ellie
Lacan, Camille
Lam, Adrian
Lambert, Thomas
Lanahan, Lauren
Langen, Mike
Laurentsyeva, Nadzeya
Law, Kelvin K.F.
Le, Quoc Thai
Lee, Choonsik
Lee, Daniel
Lee, Kyeong Hun
Lee, Sunkee
Lee, Yeonjoo
Lei, Murray
Lei, Zhou
Leitner, Stephan
Lepori, Gabriele Mario
Levari, David E.
Lewis, Ben William
Leyden, Benjamin T.
Li, Chenghuai
Li, Jiasun
Li, King King
Li, Linfeng
Li, Meng
Li, Shukai
Li, Shuo
Li, Ye
Li, Yushen
Liang, Chuchu
Lim, Stanley
Lin, Mingfeng
Lin, Po-Hsuan
Lin, Yunduan
Linardi, Sera
Lincoln, William
Lindenmayr, Michaela
Linnenluecke, Martina
Listo, Ariel
Litjens, Robin
Liu, Chengwei
Liu, Dingyue
Liu, Fang
Liu, Haibo
Liu, Haiyang
Liu, Jiaxin
Liu, Kaiqi
Liu, Nan
Liu, Sheng
Liu, Xiaojin
Livneh, Neta
Lluent, Tatiana
Loehndorf, Nils
Lof, Matthijs
Loheac, Youenn
Lohmann, Paul
Lopez, Luis Arturo
Lorko, Matej
Lotti, Francesca
Lu, Joy
Lu, Xinyu
Luffarelli, Jonathan
Luhan, Wolfgang J.
Luong, Hoang
Lyu, Guodong
Ma, Liang
Madio, Leonardo
Maeckle, Kai
Mahmoudzadeh, Mahdi
Maillé, Patrick
Mak, Vincent
Malézieux, Antoine
Mankad, Shawn
Mantilla, César
Mantin, Benny
Mantovani, Marco
Marchesini, Giacomo
Marcucci, Juri
Fages, Diego Marino
Masiliunas, Aidas
Massoni, Sébastien
Matias, Nunez
Matthys, Thomas
Mattsson, Martin
Maurer, Thomas Andreas
Maus, Patrick
Kütük, Merve Mavuş
Max, Malte M.
Meinerding, Christoph
Meister, Matt
Meitong, Dong
Melero, Eduardo
Mendes, Diogo
Menzer, Tyler
Merkle, Christoph
Merrick, Jason
Meyer, Steffen
Miklánek, Tomáš
Mill, Wladislaw
Minner, Stefan
Mirzayev, Emil
Mittlaender, Sergio
Møller, Stig Vinther
Molnar, Andras
Moore, David
Mortal, Sandra
Moscariello, Giovanni
Mou, Yuting
Mu, Jifeng
Mueller, Clemens
Mukherjee, Anirban
Mustafazade, Sara
Muthuraman, Kumar
Nakkas, Alper
Naughton, Jim
Ng, Hunter Boon Hian
Nguyen, Lily
Nguyen, Mike
Nguyen, Ngoc Phuong Anh
Nguyen, Thi Thuy Tien
Nguyen-Chyung, Amy
Nicolaou, Nicos
Nolte, Sven
Non, Arjan
Ødegaard, Bernt Arne
Ofek-Shanny, Yuval
Oh, Chang Hoon
Olivola, Christopher Yves
Omer, Thomas C.
Orland, Andreas
Otto, Tizian
Ouyang, Manlu
Ozyilmaz, Hakan
Pairolero, Nicholas A.
Palan, Stefan
Pandit, Navya
Papies, Dominik
Park, Jiyong
Park, Tae-Youn
Parker, Chris
Patel, Vinay
Pawlina, Grzegorz
Nestour, Elise Payzan-Le
Pearce, Graeme
Peeters, Thomas
Peliova, Jana
Peng, Zhuozhen
Pérignon, Christophe
Peter, Noemi
Peukert, Christian
Phan, Hieu
Philipp-Muller, Aviva
Phua, Kenny
Pierson, Matthew
Plíhal, Tomáš
Ploner, Matteo
Porcher, Simon
Pourieux, Matthieu
Preuss, Susanne
Procházka, Jakub
Pu, Shaolin
Puklavec, Žiga
Qin, Hanzhang
Qiu, Tian
Qiu, Xincheng
Rahal, Rima-Maria
Rahimian, Amin
Rajabzadeh, Mohammadreza
Randall, Oliver
Ray, Soumya
Rehbein, Oliver
Reichenecker, Jurij-Andrei
Reinholtz, Nicholas
Reiss, J. Philipp
Renne, Jean-Paul
Reza, Sadat
Richardson, Paul
Riddiough, Steven
Rieger, Marc Oliver
Righi, Cesare
Rilke, Rainer Michael
Riutort, Julio
Robotti, Cesare
Römer, Nathalie
Rose, Julia
Rose, Michael
Rosmer, Paul
Rossi, Federico
Rostami, Borzou
Roszbach, Kasper
Rotaru, Kristian
Roth, Yefim
Rotolo, Daniele
Rott, Christina
Routledge, Bryan
Rubineau, Brian
Rusch, Hannes
Ryzhov, Ilya O.
Saffi, Pedro
Saglam, Mehmet
Samahita, Margaret
Sarantopoulos, Panagiotis
Sarhangian, Vahid
Savaşaneril Tüfekci, Seçil
Scheule, Harald
Schleritzko, Maximilian
Schnidman, Max
Schoch, Daniela Stephanie
Schröder, Marina
Schütte, Erik Christian Montes
Schwartz, Daniel
Schwerter, Frederik
Seamans, Robert
Seifert, Matthias
Servranckx, Tom
Sethuraman, Nagarajan
Sevcenko, Victoria
Shah, Divyesh Rajendra
Shah, Rachna
Sharma, Kartikey
Sharma, Padma
Sheneman, Amy
Shi, Yunting
Shuai, Ling
Siegenthaler, Simon
Silberholz, John
Silva, Rui
Silz-Carson, Katherine
Simon, Felipe
Singal, Raghav
Sinha, Nitish Ranjan
Skouras, Spyros
Smerdon, David
Smolka, Katrin
Soetevent, Adriaan
Sojli, Elvira
Sokolov, Konstantin
Somasundaram, Jeeva
Son, Yoonseock
Song, Ju Myung
Soni, Vikas
Sonsino, Doron
Souther, Matthew
Spaenjers, Christophe
Spann, Martin
Spiliotopoulou, Eirini
Starck, Jeffrey
Starkweather, Austin
Steele, Dayton
Stefan, Matthias
Stehr, Frauke
Steiner, Eva
Stich, Lucas
Stoeckl, Thomas
Stoop, Jan
Ströhlein, Karoline
Stüber, Robert
Sturgess, Jason
Su, Yuhan
Su, Yuxin
Suchon, Rémi
Sui, Mengtian
Sülz, Sandra
Sung, Elie
Szymanowska, Marta
Tabacco, Giovanni Alberto
Tannenbaum, David
Tereyagoglu, Necati
Tergiman, Chloe
Testoni, Marco
Thakor, Richard
Tham, Wing Wah
Thelaus, Samuel
Thielen, Simon
Tong, Lu
Tonguc, Ozlem
Tonin, Mirco
Toraman, Sinem Yagmur
Tortoriello, Marco
Tracy, J. Dustin
Tremewan, James
Tripathi, Muktak K.
Tumer-Alkan, Gunseli
Turcic, Danko
Turocy, Theodore
Tyagi, Hanu
Udenio, Maximiliano
Ulku, Sezer
Ungeheuer, Michael
Utke, Steven
Uzmanoglu, Cihan
Vacca, Matteo
Valta, Philip
Van Der Borgh, Michel
Van Der Geest, Jesse
Van Steenvoort, Milan
Van Veldhuizen, Roel
Vana, Prasad
Vanhoucke, Mario
Vanneste, Bart
Vecci, Joseph
Venkataraman, Sriram
Veronesi, Marcella
Vicente, Sergio
Villa, Sebastian
Martin, Marta Villamor
Vincent, Lynne
Vladasel, Theodor
Voigt, Stefan
Vosgerau, Joachim
Vossler, Christian A.
Vossmeyer, Angela
Wagner, Hannes F.
Waguespack, David M.
Walker, Edward
Walker, Matthew
Walzl, Markus
Wan, Zhixi
Wang, Charles C.Y.
Wang, Joseph Tao-Yi
Wang, Kanix
Wang, Victor Xiaoqi
Wang, Xiaohong
Wang, Yiwei
Warnes, Xavier S.
Wasserka-Zhurakhovska, Lilia
Wei, Wei
Weiergraeber, Stefan
Weiss, Patrick
Weng, Jingjing
Weng, Wei-Chien
Weston, James
White, Joshua Tyler
Wibral, Matthias
Williams, Jared
Wilms, Ole
Wirl, Franz
Wolanski, Adrian
Wong, M.H. Franco
Woods, Daniel John
Wu, Biyu
Wu, Yiran
Wu, Ziye
Wuttke, David
Xia, Yuze
Xie, Jingui
Xie, Wen
Xu, Feiyu
Xu, Luze
Xu, Sikun
Xu, Simon
Xu, Yilong
Xue, Rui
Yalcinkaya, Beril
Yang, Ruijing
Yang, Yadi
Yao, Huang
Yao, Shiqing
Ke, Yaojun
Yapar, Ozge
Yelagin, Eduard
Yeung, Ira
Yilmaz, Erdem Dogukan
Yilmaz, Levent
Yoo, Woongsun
Yoon, Simon
Youn, Sora
Young, Alex
Yu, Jin
Yu, Jungju
Yu, Junhao Vincent
Yu, Lizi
Yuan, Huaiping
Yuan, Yuan
Yue, Lei
Zednik, Anita
Zeinali, Yasser
Zhai, Shenghui
Zhan, Xintong
Zhang, Aiqi
Zhang, Chengyu
Zhang, Huanan
Zhang, Huanren
Zhang, Hulai
Zhang, Jack H.
Zhang, Le
Zhang, Quan
Zhang, Renyu
Zhang, Ruishen
Zhang, Shu
Zhang, Sili
Zhang, Walter W.
Zhang, Zhiqi
Zhao, Jiayu
Zhao, Xiaofei
Zhao, Zhongyu
Zheng, Jiakun
Zheng, Yaping
Zheng, Zhanzhi
Zhou, Aner
Zhu, Hongyi
Zhu, Jason
Zhu, Yayongrong
Zihlmann, Christian
Zoican, Marius
Zultan, Ro'i
Zuo, Zhuan

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

81
views

0
downloads

Cite This

With the help of more than 700 reviewers, we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hardware and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29% of articles, at least part of the data set was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. These figures represent a significant increase compared with the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, of which 55% could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in data set accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, and software and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies and suggest potential avenues for enhancing their effectiveness.

Subject Keywords

crowd science, replication, reproducibility

URI

https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85187694519&origin=inward
https://hdl.handle.net/11511/118629

Collections

Department of Industrial Engineering, Article