Tools

You can find here some in-house tools and links to external resources that might be useful in designing your study. In the long term we will be constructing a library of off-the-shelf customizable tools that can be used for data collection, measuring study endpoints and more.

Randomization

We've made a simple tool called Weighted Links which allows you to create a link that randomly redirects people to different URLs. This could be used for example to randomly assign participants to different study arms ('intervention 1', 'intervention 2', 'controls'), with different instructions for intervention and control groups. In this example with 3 groups you would assign a 1:1:1 weighting if the aim was to have an equal number of participants in each group, or 1:1:2 if you wanted double the number of controls. You can try out the tool at https://demos.science/links

Measure outcomes

Test human reaction times: https://www.humanbenchmark.com/tests/reactiontime

Sample size calculations

As a general rule of thumb -- the bigger your trial the more accurate the results, and in Demos we encourage you to aim high! This and other simple rules of thumb about the minimum number of participants required for your studies are available from the Abdul Latif Jameel Poverty Action Lab (J-PAL), a global research center focused on reducing poverty by running randomized trials to inform policy making (which recently won them the Noble Prize in economics!).

For a more precise calculation of the required sample size there are a variety of online tools. We particularly like this one from UCSF, as it allows you to easily ask either the number of participants required to detect a per-specified effect size you think would be meaningful, or to start with the minimal number of participants you expect to to calculate the effect size you will be powered to detect; this might be most relevant to Demos studies where we aim for large scale studies.

Calculating the required sample size for study designs that include multiple intervention arms can be done with an online tool developed by researchers at the Universities of Newcastle and Cambridge.

And no reason to be daunted if this is all new to you -- as we build up the Demos community we hope to attract experienced researchers to join or advise study teams, and for now the central Demos team has expert statisticians, web-developers and data managers and scientists on hand to support you!