Demos.science Education

Demos.science Education is an experiential learning platform that aims to enhance our understanding of how randomized trials work, giving us a better idea of how evidence-based answers should be obtained.

Watch video

Pilot Study – Does Classical or Techno Music Enhance Concentration?

This is a pilot study to showcase Demos.science Education as an experiential learning platform that engages and educates the public on randomized trials. It is not intended to be a rigorous study on the impact of music on concentration.

If you choose to participate in the pilot study, you will have an opportunity to learn how a randomized trial works, experience what it means to participate in a randomized study, and acquire a better understanding of key terms and concepts used in the experimental research process.

The pilot study is open to persons 18 years of age and older. Participating in the experiment will require approximately 5 minutes of your time, and you can jump straight in here, or spend a few minutes reading through this page first, familiarizing yourself with how randomized trials work and why they are so important.

Take part in the study

Understanding Randomized Trials

We will run through what a well-run trial should look like, drawing on the CONSORT Statement, which sets out how randomized trials should be reported in medical research – but don't worry, this is a much abbreviated and simplified walkthrough of the main ideas. To illustrate the different points, we will use the pilot study on the effects of different music types on concentration, working our way from early conceptualization to reporting the final conclusions. You will also have the opportunity to take part as a study participant so you can familiarize yourself with the study process from different standpoints.

Background and rationale for the trial

As a starting point, we need to explain the background and rationale for the trial, and its general outline.

The context for this study is the COVID-19 pandemic, which resulted in many of us being asked to work from home, where we faced many more distractions affecting our productivity. What tools can we use to enhance our productivity and concentration? Music has been touted to improve concentration and focus. But before we rush out to buy the latest active noise-cancelling headphones, how can we find out if listening to music actually does enhance concentration and focus? And if so, what type of music might be most helpful?

What does the existing evidence suggest?

The first thing we need to do is determine if previous studies have already provided an evidence-based answer to our question. A research study should aim to generate new knowledge, building on existing evidence. This is particularly important in studies that might pose risks to the participants. Conducting a study when previous studies have already conclusively answered the research question may be unnecessary and unethical.

Here, we find that previous studies have examined the effects of background music in the workplace, but these have often included very specific groups of individuals, like young males, or members of certain professions. Some studies have shown that music preference, age and gender can change how music affects concentration. After reviewing other studies, we might conclude that there is a need for a further study that includes a broader representation of working adults, and more evidence is needed to find out if different music types affect concentration differently. So we can narrow our research question down to:

Which music types are best at improving concentration?

Formulate the hypothesis

The research question describes the overall question the study is addressing. For randomized trials (and other experimental studies) it is often more useful to have a specific and testable hypothesis – a statement setting out what the experiment will either confirm or refute. In this case, we are unsure which music genre will enhance concentration compared to not listening to music. So, we will frame our ‘non-directional’ hypothesis as:

Listening to different music genres will affect concentration differently.

Learn more: what makes a hypothesis?

To keep things simple, we will limit our investigations to two music genres - classical and techno.

Trial Design

We now need to select our study design and devise a testing method. The study design and testing method should enable us to prove or disprove the hypothesis: listening to different types of music affects concentration differently.

To allow us to test our hypothesis, we will need to compare participants’ performance across three study groups:

Participants listening to classical music
Participants listening to techno music
Participants listening to ‘white noise’ – the control group

But how are participants allocated to each group? Letting participants choose what music they want to listen to and comparing their performance in the attention task can completely undermine the validity of the study’s conclusions! A much better design is a randomized trial, where participants are allocated to different study groups at random.

Learn more: Why randomized trials are a superior study design

Learn more: Not all scientific studies are created equal

In this instance our study design will be a ‘multi-arm randomized controlled trial’. (It’s ‘multi-armed’ as we have two intervention groups listening to different music types and a ‘control’ group listening to white noise.)

Let’s imagine for a moment that classical music lovers are on average more focused by nature, and perform better on a concentration tests regardless of what they’re listening to. They might all decide to join the classical music group, and then go on to perform well on the test, which they would have anyway being naturally more focused! We might then falsely conclude that their better performance was a consequence of listening to classical music. This is known as ‘confounding’ and it is the cause of many false research findings, and many misleading headlines in the media.

Another topical example, is the need for randomized trials to help us identify the best treatments for COVID-19. Many of the headlines and vociferous arguments for or against possible treatments have been based on non-randomized studies (or just anecdotes) looking at the treatments patients received and whether they recovered to conclude as to whether different treatments are effective. Some of these claims were based on data from just a handful of patients, which is obviously problematic, but even those based on very large datasets suffer from a critical bias – there are likely to be systematic underlying differences between patients that were chosen to receive one treatment as opposed to another. If, as is often the case, an untested drug is given primarily to patients who are so severely ill that their physicians are willing to try anything in a desperate attempt to help them, it makes no sense to compare their recovery rates with patients who were less severely ill and therefore more likely to receive other standard treatments. By assigning patients at random into different treatment groups we can gain high quality evidence as to whether the treatments themselves result in different recovery rates, rather than any underlying risk factors.

By ‘randomizing’ the allocation of participants into the study groups, we can avoid confounding problems, assigning individuals to different groups at random (e.g. flipping a coin). By doing this, and recruiting enough participants, groups will end up with approximately equal numbers of individuals of all different musical preferences, concentration abilities, and other characteristics like age and gender. If one group does in fact go on to perform better than others we can be more confident that this is caused by the music they listened to, rather than other confounding factors.

Not yet convinced? Here’s another great summary of why randomized trials are so important in battling bad science.

Participants

We now need to decide what the eligibility criteria are – who will be included in the study? Importantly, our eligibility criteria should aim to ensure that the study participants reflect the population where we expect the intervention will be used. If our study participants are different to the population of interest, our results might not be generalizable to this population. For example, if all our study participants are classically trained musicians (perhaps because we recruited all our participants from a music school), our findings from the trial might not apply to non-musicians. Another major concern is that we exclude people who might be at higher risk if they participate in the study. So, we might need to devise exclusion criteria to avoid exposing them to unnecessary risk. In this pilot study, we are interested in the impact of music on concentration for adults working remotely. Our population of interest is working adults. So, our inclusion criteria will be:

Adults aged 18 years or older
Working from home on desk-based tasks
With access to a internet-connected computer or mobile device
Who have consented to participate in the study

Interventions

We now describe the trial interventions. In medical research this might involve a new treatment or vaccine. A detailed description of the intervention is critical: it might be the case that a drug doesn’t work at a specific dose in one study, but would work very well at a higher dose in another study. By stating exactly what the intervention is we can also ensure reproducibility in research – the ability to repeat the experiment and obtain similar findings.

In this trial, we have two intervention groups and a control group:

Group A will listen to a selected piece of classical music
Group B will listen to a selected piece of techno music
Group C will listen to white noise (the control group)

Play a sample of the classical music

Play a sample of the techno music

Play a sample of the white noise

Outcomes

We now need a tool that will allow us to measure concentration – an attention task. Here, we use a standardized test, known as the Sustained Attention to Response Test (SART). The SART is a computer-generated test that measures individual performance in attention tasks. Participants must tap the spacebar every time a number flashes on the screen, but refrain from tapping when a specific number - "3" - appears. For this study we developed an open-source three-minute version of the test.

Sample size

Generally speaking – the more participants we have in a study the higher the probability of our experiment proving (or disproving) the hypothesis correctly. On the other hand there are good reasons to keep the number of participants to a minimum, such as the logistical requirements and high costs for running studies. Importantly, if there are any risks for participants it would be unethical to recruit more participants than needed to generate robust findings. So to ensure our study yields a ‘scientifically valid’ or ‘statistically significant’ result, we need to calculate the minimum number of participants needed – our sample size – for each experimental group in our study.

In our pilot study, we estimate that we will need 202 participants in total.

Learn more: calculate a sample size?

Learn more: Sample size calculation for the pilot study

Randomization

As mentioned, randomization of participants into study groups is critical to avoiding bias and confounding in the findings. How to ensure true randomization is actually much more challenging than you might expect! Recruiting into different groups on alternative days for instance might seem like a simple approach but can open the door to confounding factors or even ‘gaming’ the system (participants choosing to enroll on days when they will be allocated to a preferred treatment).

In our study we’re making use of an open source randomization tool we developed, whereby participants are redirected to one of three different webpages where they will listen to the different music genres (or white noise) while completing the SART.

Blinding

Ideally, the trial would be designed so that neither participants nor researchers have any knowledge of which study group participants are placed in until the trial itself is completed. The ‘unblinding’ occurs only when the data have been analyzed.

Learn more: what is blinding?

In our pilot study it is impossible to blind the participants, as they will know which group they are in. Because there are no researchers involved in their recruitment or assessment to be blinded either, the study is considered ‘unblinded’

We are however keeping the results partially concealed until we recruit enough participants to meet the sample size. Why? Because if participants see that, for example, those listening to techno are performing much better than those listening to classical music, this could affect their own performance in the experiment.

Statistical Methods

We must have a clear plan for analyzing the data we collect. A trial should have a single ‘primary’ outcome. Before we begin the experiment, we should determine which statistical test we will use to decide if the hypothesis is correct. If there are multiple outcomes and we use different or unsuitable statistical tests to compare them, we might find differences between study groups just by chance. If we report these "findings" people might believe the intervention has effects that it doesn't actually have.

In our study we will compare the average (or ‘mean’) SART error rates between the three study groups. The full statistical plan is available (not for the faint-hearted!)

Informed Consent

Before enrolling participants in a study, the researchers must obtain the free, voluntary and informed consent of each prospective participant.

Learn more: what is informed consent?

In the pilot study, participants will have an opportunity to experience a simplified informed consent process. Participants will be presented with an overview of the study that sets out key points at a quick glance. Participants will then be provided with a link to a downloadable detailed Study Information Sheet. Participants will be asked to provide free, voluntary and informed consent by clicking on three consent boxes, confirming: (1) they understand what it means to participate in the study, and voluntarily agree to participate; (2) they understand and acknowledge the risks involved in participating in the study; (3) they are 18 years of age or older.

Obtain ethical approval

Prior to recruiting participants and conducting the study, we need to obtain ethical and regulatory approval from local authorities. For a study to be viewed as ethical, it must abide by four guiding principles:

Research integrity – does the research question have merit? will the study design yield a valid scientific result?
Justice – are participants being treated fairly?
Beneficence – do the benefits of the study outweigh its risks?
Respect – are we treating people with respect and dignity?

The ethical review process will vary depending on the nature of the experiment, the participants involved and the context in which the study takes place.

This study complies with ethical standards and codes of conduct of the University of Oxford. This pilot study is conducted for educational purposes to teach participants about randomized trials, with minimal risk to participants and no directly identifiable information being collected. After consulting an Ethics Committee at the University of Oxford it was agreed that no review would be necessary.

Learn more: what is ethical approval?

Run the experiment and collect data!

You can now take part as a participant in the randomized trial. This is completely voluntary and the data collected is confidential.

Take part in the study

Analyze data and report findings

After running the experiment, we now need to compare the data between the intervention groups and the control group. Go ahead and see the live results of the study so far!

See pilot study results and analysis

Discussion – study limitation, generalizability and conclusions

With our results in hand, we can now go back to our research question and hypothesis, and discuss how we might interpret the findings. What were the strengths and weaknesses (or limitations) of our study design? How generalizable do we think these results are to a real life, non-experimental setting? And what about generalizing to populations that differ from those in our study, such as children? How do our results compare with other studies? We might also suggest new research questions. Can we consider whether personal music preference will affect the extent to which music improves concentration? Is listening to music more beneficial in some types of tasks than others?

Researchers writing up their conclusions will often be very cautious in stating the limitations of their studies, avoiding bold statements such as ‘Our study definitively proves that techno music improves concentration!’. This cautiousness is all too often abandoned in popular media when they report on research findings.

Thank you for joining us in the Demos.science Education Pilot Study! We hope this experience has been informative as to how randomized trials are conducted and how they can generate evidence-based answers to everyday research questions. Once we have enough participants we will run the full (unblinded) analysis and write up the conclusions. If you would like to be kept informed of this and other Demos.science news please sign up on our website.

Your feedback will be very helpful for us in deciding whether to continue developing this educational component of the Demos.science platform.

Take part: Complete exit survey

Share this page:

This project was supported by a Wellcome Trust funded Public Engagement Bursary at the Mahidol-Oxford Tropical Medicine Research Unit (MORU).