The Education Endowment Foundation (EEF) collaborated with Tes Magazine to release the “Tes explains” series in the UK.
The series includes short articles that explain key education research terminology and provide guidance on identifying robust evidence sources.
E4L has curated a selection of these articles in order to help Australian educators use education research and critically assess the evidence underlying a given approach. These curated articles are useful for educators who wish to strengthen their understanding of research methods, processes and outcomes to support their professional judgment and decision making.
This page provides a brief overview of each term, and links to the correlating article where you can find more detail.
When considering the findings of a piece of research, it’s important to think about the design of the study, and whether this makes the research capable of answering the question that you are interested in; and any problems with the study that might represent “threats” to the validity of the findings.
When trying to judge a study, you are ultimately trying to assess whether it has been well designed, well conducted and well reported. It should be based on a clear research question, and it should use a sound research methodology, with limitations fully acknowledged. The data should be collected and analysed carefully, and all the analyses should be reported transparently.
E4L uses a padlock rating for evaluated projects to indicate the strength and security of the research.
A randomised controlled trial (RCT) is an experiment in which candidates are randomly assigned to one of two groups. The experimental group receives the intervention being tested while the other receives an alternative, or no intervention at all (for the time period under study). The two groups are then compared to see if there are any differences in outcome.
RCTs are often considered to be the gold standard for evaluating the effectiveness of interventions because they help to ensure that results are not due to chance. RCTs are the best method we currently have for trying to understand causality.
However, RCTs may not always be the best approach – it depends on the research question.
E4L has completed three randomised controlled trials.
Learn more about whether everything should be evaluated through RCTs.
A literature review offers a critical evaluation of academic writing on a specific subject. It explains the strengths and weaknesses of the arguments presented within the research literature.
The term “literature review” encompasses many types of research that can differ greatly in how much they tell us (e.g. a systematic review to a think piece). Sometimes, how decisions are made about what research to include are not clear.
Action research is a practice-based research method, designed to bring about change in a context, like a classroom. It includes identifying a problem or question and developing ways to address it.
When a problem or question is identified, an initiative is developed and implemented to address it. Following implementation, the impact is assessed and reflected upon. This reflection may lead to amendments to the initiative, before repeating the process.
Translating findings from research studies to new contexts can be complex, and even inadvisable in some cases. There are considerations to assess whether an intervention or approach that has proven effective in one study will be suitable and have a similar impact in a new setting.
In general, research that is based on well-established principles and theories is more likely to be applicable to new contexts, because it is less likely to be influenced by the specific context in which it was conducted. It’s also important to consider the similarity between the old and new contexts.
Learn more about how far we can apply research to a new context.
A lethal mutation occurs when evidence-informed practice is modified beyond recognition from the original practice.
When adapting evidence-informed practices to fit within their contexts, educators may unintentionally develop counterproductive strategies.
Researcher bias occurs when a researcher’s own beliefs or expectations influence the outcome of their research leading to inaccurate or misleading results.
However, there are many ways to minimise researcher bias and safeguards that can be built into the research process.
When conducting trials or experiments, researchers first give a hypothesis of what they expect to find. When the outcome does not support that hypothesis, it is called a null result.
Null results can include results where there is a secure estimate that is close to zero impact, or results whose statistical uncertainty means that, while there may be a positive or negative impact recorded, on average, there is not enough security to confirm this.
Matched studies are a type of research that aims to understand impact by comparing individuals who receive an approach with individuals who do not.
In a matched study, participants are not randomised – instead, an intervention group and a control group are selected and compared based on having similar characteristics. In education settings, participants may be matched based on variables such as age, school and teacher.
Matched studies are considered quasi-experiments because they use what is known as quasi-experimental design (QED) rather than randomisation.
When describing the results of research, “significance” does not mean that the result is noteworthy or important. Instead, significance (or statistical significance) has a technical meaning that is used to describe the statistical uncertainty of a result.
A randomised controlled trial (RCT) is an experiment in which candidates are randomly assigned to one of two groups. The experimental group receives the intervention being tested while the other receives an alternative, or no intervention at all (for the time period under study). The two groups are then compared to see if there are any differences in outcome.
E4L has completed three RCTs.
In a systematic review, researchers summarise all the research on a particular topic that meets pre-defined eligibility criteria, in order to answer a specific research question. A systematic review can also be combined with meta-analysis.
This type of research review gives an objective overview of the evidence on a particular topic or question.
E4L has done a number of systematic reviews, which include student health and wellbeing and supporting rich conversations in early childhood education. Systematic reviews also underpin the approaches in the ECE Toolkit and the Teaching & Learning Toolkit.
In a meta-analysis, researchers combine the results of multiple studies to calculate an overall effect. This involves more than just pooling data from studies together – the process uses established statistical methods to account for differences in sample size, variability and findings.
Combining the results from several studies in this way can be helpful when it comes to investigating the average effectiveness of a particular teaching and learning approach. If you’re using meta-analyses to support decision making, it’s important to look beyond the average and consider the diversity of results.
Learn more about meta-analyses.
A cohort study is a type of longitudinal study that follows participants who share a common characteristic over a period of time, often for many years.
Growing Up in Australia: The Longitudinal Study of Australian Children is an Australian example of a cohort study.
Learn more about cohort studies.
Peer review is used in academic research as a form of quality control. Research papers are sent to other experts in the field, who scrutinise the work and ensure that the findings are original, valid and high quality.
An effect size is a simple way of presenting the difference between two groups of pupils, usually one that has received a particular teaching and learning approach and another that hasn’t. When something has had an impact on learning, the effect can be measured and expressed as a number. Effect sizes are standardised measures that translate impacts from different outcomes into a numerical value that can allow for impacts to be compared with each other or combined in a meta-analysis.
In our Toolkits and randomised control trials, E4L converts effect sizes to months of progress, to help schools interpret the potential impact that a particular approach could have (noting that there are nuances that sit behind a standardised effect).
When researchers analyse evidence, they look at outcome measures. For primary research, this will be raw data collected from a new trial, whereas in a systematic review, this will be outcome data from pre-existing studies.
In both cases, the researchers will work to translate these outcome measures to interpret the data and draw conclusions, making caveats where there are important limitations or discussing whether the results corroborate previous research.
E4L research trials and systematic reviews involve pre-specification – which is when researchers publish a protocol in advance of any data analysis to explain how the data will be analysed ahead of time. This helps to mitigate the risk of prioritising only positive results in the analysis.
Learn more about how researchers analyse data.
Research studies are carried out for various reasons. Some are conducted in an academic setting, either for publication in an academic journal or as part of a PhD. Research can also be commissioned by third parties, such as governments, a company, a charity or other non-governmental organisation.
The way research is commissioned is likely to be influenced by the aims and budget of the organisation.
Learn more about how research is commissioned.
All research has a cost, and most studies can only be carried out with the support of grants from funding councils, government bodies, private companies or non-profit organisations.
This doesn’t mean that the results of those studies cannot be trusted. However, it is crucial to consider the independence and transparency of research – and the interests of the sponsors – when interpreting results. These considerations include whether the study methods are pre-specified, all results are published, an independent peer review process is used and or data is made available for replication.
E4L is an independent not-for-profit and all E4L research is completed by independent research teams from E4L’s evaluation panel, according to E4L’s values and informed by E4L’s and the EEF’s evaluation and peer review processes.
Learn more about research and sponsors.
Research gets published via several different routes. One of the most common is through an academic journal, which involves authors submitting their manuscript to an editor, who then determines whether the article will be reviewed by the editorial board or external reviewers. There are time and accessibility challenges in relation to academic journal publications.
Research funders, think-tanks and non-profits, such as E4L, will often commission and publish findings outside of academia. Often these are free to access and written in more accessible language.
Learn more about how research is published.