Web-based experimental economics software: How do they compare to desirable features?

Research output: Contribution to journalReview article

Abstract

Web-based experiments that cut across the lab vs. field distinction are increasingly popular with economists. However, non-standardized software features and services hinder comparability and replication. This study reviews a wide selection of experimental economics software packages and evaluates them against criteria based on the logistics and operational requirements of economic experiments. We find that oTree and SoPHIE rank highest across criteria, but Veconlab and classEx might be suitable for those with a dominant need for a large library of ready-made experiments. We find a portability gap: no presently available software allows portability of experiments across platforms because of technical complexity and the challenging coordination needs of experimental economists. As a result, experiments may be replicated only on the same platform or with the same software, but general replicability is slow and costly. This constrains the development of experimental economics as a replicable science.

Original languageEnglish
Pages (from-to)138-160
Number of pages23
JournalJournal of Behavioral and Experimental Finance
Volume23
DOIs
Publication statusPublished - 1 Sep 2019

Fingerprint

Web-based
Experiment
Software
Experimental economics
Economists
Portability
Economic experiments
Replication
Logistics

Cite this

@article{da239c7d83854cbb8365c0410c78713a,
title = "Web-based experimental economics software: How do they compare to desirable features?",
abstract = "Web-based experiments that cut across the lab vs. field distinction are increasingly popular with economists. However, non-standardized software features and services hinder comparability and replication. This study reviews a wide selection of experimental economics software packages and evaluates them against criteria based on the logistics and operational requirements of economic experiments. We find that oTree and SoPHIE rank highest across criteria, but Veconlab and classEx might be suitable for those with a dominant need for a large library of ready-made experiments. We find a portability gap: no presently available software allows portability of experiments across platforms because of technical complexity and the challenging coordination needs of experimental economists. As a result, experiments may be replicated only on the same platform or with the same software, but general replicability is slow and costly. This constrains the development of experimental economics as a replicable science.",
keywords = "Economic experiments, Experimental economics, Online experiments, Software, Web-based, Web-based experiments",
author = "Chan, {Shu Wing} and Steven Schilizzi and Iftekhar, {Md Sayed} and {Da Silva Rosa}, Raymond",
year = "2019",
month = "9",
day = "1",
doi = "10.1016/j.jbef.2019.04.007",
language = "English",
volume = "23",
pages = "138--160",
journal = "Journal of Behavioral and Experimental Finance",
issn = "2214-6350",
publisher = "Elsevier",

}

TY - JOUR

T1 - Web-based experimental economics software

T2 - How do they compare to desirable features?

AU - Chan, Shu Wing

AU - Schilizzi, Steven

AU - Iftekhar, Md Sayed

AU - Da Silva Rosa, Raymond

PY - 2019/9/1

Y1 - 2019/9/1

N2 - Web-based experiments that cut across the lab vs. field distinction are increasingly popular with economists. However, non-standardized software features and services hinder comparability and replication. This study reviews a wide selection of experimental economics software packages and evaluates them against criteria based on the logistics and operational requirements of economic experiments. We find that oTree and SoPHIE rank highest across criteria, but Veconlab and classEx might be suitable for those with a dominant need for a large library of ready-made experiments. We find a portability gap: no presently available software allows portability of experiments across platforms because of technical complexity and the challenging coordination needs of experimental economists. As a result, experiments may be replicated only on the same platform or with the same software, but general replicability is slow and costly. This constrains the development of experimental economics as a replicable science.

AB - Web-based experiments that cut across the lab vs. field distinction are increasingly popular with economists. However, non-standardized software features and services hinder comparability and replication. This study reviews a wide selection of experimental economics software packages and evaluates them against criteria based on the logistics and operational requirements of economic experiments. We find that oTree and SoPHIE rank highest across criteria, but Veconlab and classEx might be suitable for those with a dominant need for a large library of ready-made experiments. We find a portability gap: no presently available software allows portability of experiments across platforms because of technical complexity and the challenging coordination needs of experimental economists. As a result, experiments may be replicated only on the same platform or with the same software, but general replicability is slow and costly. This constrains the development of experimental economics as a replicable science.

KW - Economic experiments

KW - Experimental economics

KW - Online experiments

KW - Software

KW - Web-based

KW - Web-based experiments

UR - http://www.scopus.com/inward/record.url?scp=85068099886&partnerID=8YFLogxK

U2 - 10.1016/j.jbef.2019.04.007

DO - 10.1016/j.jbef.2019.04.007

M3 - Review article

VL - 23

SP - 138

EP - 160

JO - Journal of Behavioral and Experimental Finance

JF - Journal of Behavioral and Experimental Finance

SN - 2214-6350

ER -