The STEAM-Active (Project Number: 2021-1-ES01-KA220-HED-000032107) project is funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Education and Culture Executive Agency (EACEA). Neither the European Union nor EACEA can be held responsible for them.

Evidence of STEM enactment effectiveness in Asian student learning outcomes

Partners' Institution
University of Perugia
Year of publication
Educational stage
Secondary Level, University Level
Journal name
International Journal of STEM Education
Thematic Area
STEAM intervention (teaching strategies, evaluation...)
The study is a systematic review and a quantitative meta-analysis of publications about STEM enactments in Asia. Even though eastern and western countries have differences in teaching and learning methods, both regions have similarities in terms of challenges faced (low interest in the STEM field and increasing competitiveness of workplaces). Thus, the focus of the paper - which concerns the effectiveness of STEM teaching in Asia -, could also provide some generalized information. Specifically, it investigates how the learning outcomes (achievements, higher-order thinking skills, and motivation) are influenced by STEM enactments. The key questions are articulated in 3 points:
1) What is the scenario of STEM enactments in Asian countries?
2) Do the STEM enactments influence learning outcomes (achievements, HOTS, and motivation)?
3) Under what circumstances and for what learning outcomes are STEM enactments more effective?
As the first question isn’t relevant in the European’s STEM scenario it won’t be discussed further. Instead, the answers to the other two could provide some information in our project.
The study is very heavy on the “quantitative” part and gives answers to these questions only through a statistical analysis of a selection of papers, which were collected through screening of different databases and coded into categories. 54 studies were used for the analysis, which were obtained through a 4-stage screening using different criteria. The papers remaining have the minimum quantity of data required for calculating an effect size (value measuring the strength of the relationship between an independent variable and a dependent variable). In order to answer to the second research question, the papers are divided into categories on the basis of the learning outcomes covered in that study. To address the third research question, they code three moderator variables (learning model, learning orientation, duration of instruction). Once all the data is coded, the analysis is performed, and results are obtained and discussed.
Relevance for Complex Systems Knowledge
The study doesn’t report an intervention but it’s a statistical analysis on multiple STEM enactment studies. The focus of the first part of the paper is on the 3 learning outcomes:
- Academic learning achievements, which are defined as students’ score from either the mean of pretest/posttest or only the mean of the posttest score.
- Higher-order thinking skills (HOTS), which are students’ thinking ranked above level three according to Bloom’s taxonomy. The HOTS studies concern test with scores assigned to activities such as creating, evaluating, and analyzing.
- Motivation, defined as the process in which the learner attention becomes focused on meeting their educational objectives. Students’ motivation is usually measured in the studies through questionnaires.
Papers collected are divided in 3 categories, each addressing one of these learning outcomes. The data analysis performed uses a random effect model to calculate a summary effect size through the Cohen’s coefficient “d”. The effect size measures the magnitude of the effect of the independent variable (STEM education) on the dependent variable (learning outcome). It is:
- Very small if d < 0.2
- Small if 0.2 ≤ d < 0.5
- Medium if 0.5 ≤ d < 0.8
- Large if d ≥ 0.8
The summary results are that the overall effect size of STEM enactment on learning outcomes is 0.69, which is a medium effect. In details, the effect size on students academic learning is 0.64, on HOTS is 1.02, and on motivation is 0.49.
In order to answer to the third research question (i.e., if there are specific factors behind the effectiveness for each learning outcome) the study introduces moderator variables. For each category, every study is classified in terms of:
- Approach or learning model, meaning if the study is based on a methodological approach or learning model or not. The 2 options are present/absent.
- Learning orientation, which was divided in “culture-centric” or “universal oriented” if the study follows the unique characteristics of Asian students or refers to a more “open and free” teaching methodology, respectively.
- Duration of instruction, which refers to the duration of the STEM enactment.
The results highlight that, for learning achievements, the presence of an approach or learning model contributes to the effectiveness of STEM enactments. Moreover, culture-centric enactments have a higher effect compared to universal oriented ones. Lastly, longer durations of instruction have better effects.
For HOTS the results are more heterogeneous. The duration of instructions follows the same results of the previous case, but it seems that there’s no significant differences on the effect of the other moderator variables. With or without an approach or learning model, and whether learning orientation tends to be cultural centric or universal oriented, the HOTS of students have relatively the same effectiveness.
For what concerns motivation, the three moderator variables seem to have no effect.
The overall results point out that the STEM enactments are more successful if accompanied by an approach or learning model, like a project-based learning or a 6E learning model (Engage, explore, explain, engineer, enrich, evaluate). STEM enactments are also more effective if culture-centric, local culture can be in the form of the main lesson’s topic or as an enrichment material. For what concerns the duration of the STEM lessons, the study suggests more than 2 hours, spread over two or more class periods, will help with learning achievements and high order thinking skills.
Point of Strength
The quantitative analysis used to evaluate the effectiveness of STEM interventions in this study could be used to give a data-backed evaluation to our project. The study also gives recommendation and practical implications that can be utilized.
STEM education, STEM effectiveness, Asia, learning outcome
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

The STEAM-Active project | Copyright 2022