Relevance for Complex Systems Knowledge
The study doesn’t report an intervention but it’s a statistical analysis on multiple STEM enactment studies. The focus of the first part of the paper is on the 3 learning outcomes:
- Academic learning achievements, which are defined as students’ score from either the mean of pretest/posttest or only the mean of the posttest score.
- Higher-order thinking skills (HOTS), which are students’ thinking ranked above level three according to Bloom’s taxonomy. The HOTS studies concern test with scores assigned to activities such as creating, evaluating, and analyzing.
- Motivation, defined as the process in which the learner attention becomes focused on meeting their educational objectives. Students’ motivation is usually measured in the studies through questionnaires.
Papers collected are divided in 3 categories, each addressing one of these learning outcomes. The data analysis performed uses a random effect model to calculate a summary effect size through the Cohen’s coefficient “d”. The effect size measures the magnitude of the effect of the independent variable (STEM education) on the dependent variable (learning outcome). It is:
- Very small if d < 0.2
- Small if 0.2 ≤ d < 0.5
- Medium if 0.5 ≤ d < 0.8
- Large if d ≥ 0.8
The summary results are that the overall effect size of STEM enactment on learning outcomes is 0.69, which is a medium effect. In details, the effect size on students academic learning is 0.64, on HOTS is 1.02, and on motivation is 0.49.
In order to answer to the third research question (i.e., if there are specific factors behind the effectiveness for each learning outcome) the study introduces moderator variables. For each category, every study is classified in terms of:
- Approach or learning model, meaning if the study is based on a methodological approach or learning model or not. The 2 options are present/absent.
- Learning orientation, which was divided in “culture-centric” or “universal oriented” if the study follows the unique characteristics of Asian students or refers to a more “open and free” teaching methodology, respectively.
- Duration of instruction, which refers to the duration of the STEM enactment.
The results highlight that, for learning achievements, the presence of an approach or learning model contributes to the effectiveness of STEM enactments. Moreover, culture-centric enactments have a higher effect compared to universal oriented ones. Lastly, longer durations of instruction have better effects.
For HOTS the results are more heterogeneous. The duration of instructions follows the same results of the previous case, but it seems that there’s no significant differences on the effect of the other moderator variables. With or without an approach or learning model, and whether learning orientation tends to be cultural centric or universal oriented, the HOTS of students have relatively the same effectiveness.
For what concerns motivation, the three moderator variables seem to have no effect.
The overall results point out that the STEM enactments are more successful if accompanied by an approach or learning model, like a project-based learning or a 6E learning model (Engage, explore, explain, engineer, enrich, evaluate). STEM enactments are also more effective if culture-centric, local culture can be in the form of the main lesson’s topic or as an enrichment material. For what concerns the duration of the STEM lessons, the study suggests more than 2 hours, spread over two or more class periods, will help with learning achievements and high order thinking skills.
