FAQ

What is Empirica? Why is it different?

Empirica is a free, open-source, virtual lab platform for developing and conducting synchronous and interactive human-participant experiments in the web.

What makes Empirica different is that it tries to find the sweet spot between flexibility and simplicity.

No way to hide this, it does require minimal programming knowledge (which can be outsourced). Therefore, it is not as easy as point and click experiment system (such as Qualtrics). Nor is it as powerful as programming a fully custom program from scratch (which is obviously a lot of work).

Instead it aims for the middle ground, taking care of all the low-level complicated and repetitive tasks and offers a simple yet powerful programmatic interface.

It also offers different levels of complexity depending on your requirements. For simple experiments, not much needs to be done to get going. For more complex situations, there are many hooks and knobs to shape the system to your liking.

Our intent could be defined by: easy to do simple things and possible to do anything.

How does Empirica compare to XYZ?

It requires minimal Javascript programing skills at this moment (adapters for other languages are possible).

Compared to many platforms also requiring programing, it is not as firmly opinionated. You do start off with a healthy set of conventions to just get things done. But once your needs go beyond the default design, you are free to reprogram many things, such as how assignment works, how stage timing works, how lobbies work, etc.

Much of the “game” logic is exposed and can be modified. The immutable internals of Empirica are relatively small, dealing with all the basic needs of such as system (re/connections, synchronization, access control, persistence, etc.) How an experiment actually proceeds can be customized at will.

Empirica, since v2.0, has been redesigned to be API first, meaning everything goes through our GraphQL API “kernel” called Tajriba. This allows the different parts of Empirica to be customized. For example, you can interfaces with other systems (e.g. recruitment platforms such as Prolific), you can implement Empirica components in other languages (e.g. implement callbacks in Python), use other platforms (e.g. native mobile), etc.

Where can I find help developing my experiment?

Contact us at hello@empirica.ly, or join us on Slack, and we will do our best to put you in contact with developers that have experience with Empirica.

Where can I find help?

Check out our documentation at docs.empirica.ly, join us on Slack, or file a bug in Github.

How can I help?

Thank you very much for asking. This is an open-source project, so any help is welcome. You can help others in our Slack or help us with code. For code contributions, head over to https://github.com/empiricaly/empirica to learn more about contribution guidelines.

Who is behind Empirica?

Empirica is open source project with many contributors. The project was created (and is currently managed) by Abdullah Almaatouq and Nicolas Paton. We are also advised and supported by a strong team of academics and science enthusiasts including Iyad Rahwan, Duncan J. Watts, and Alex ‘Sandy’ Pentland. We are proud to collaborate with a diverse set of partners in academia and funding agencies, including the Alfred P. Sloan Foundation, SAGE Publishing, and the Computational Social Science Lab at Penn.

How can I cite Empirica?

If you use Empirica please cite the following paper.

Almaatouq, Abdullah, Joshua Becker, James P. Houghton, Nicolas Paton, Duncan J. Watts, and Mark E. Whiting. "Empirica: a virtual lab for high-throughput macro-level experiments." Behavior Research Methods (2021): 1-14. https://doi.org/10.3758/s13428-020-01535-9