Extending sequential pattern analyses for understanding the arrangement of learning events: maximum gaps, conditional associations, and permutation tests

Zhang, Yingbin

Permalink

https://hdl.handle.net/2142/117574 Copy

Description

Title

Extending sequential pattern analyses for understanding the arrangement of learning events: maximum gaps, conditional associations, and permutation tests

Author(s)

Zhang, Yingbin

Issue Date

2022-11-29

Director of Research (if dissertation) or Advisor (if thesis)

• Paquette, Luc
• Anderson, Carolyn Jane

Doctoral Committee Chair(s)

Paquette, Luc

Committee Member(s)

• Lane, H Chad
• Bosch, Philip Nigel

Department of Study

Curriculum and Instruction

Discipline

Curriculum and Instruction

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• learning analytics
• educational data mining
• sequence analysis
• temporal analysis
• learning process
• sequential pattern mining
• lag-sequential analysis
• sequential association

Abstract

Sequential pattern analyses are a powerful tool for capturing and understanding the relative arrangement of learning events, but current analysis methods suffer from issues that prevent educational researchers from taking full advantage of them. This dissertation contains three studies, each addressing one issue in current methods. The first two studies are concerned with the sequential association in patterns consisting of two events, such as the likelihood that students read relevant material after viewing questions answered incorrectly. Existing methods for estimating the sequential association entails a fixed gap between the events. Sequential associations where one event directly and indirectly follows another are estimated separately, even if both cases may theoretically represent the same sequential association. The challenge of estimating sequential associations with a maximum gap rather than a fixed gap is how to count pattern occurrences. Study one proposed four counting methods: table sum, event window, event window with adjustment for unequal conditional probabilities of events (denoted as P(B)), and event window with adjustment for non-random events. Simulations indicated that adjusting unequal conditional probabilities of events had a lower type I error rate and bias than the others, while table sum showed better interpretability than the others. Table sum and adjusting unequal conditional probabilities of events had higher power than the benchmark, but only when the true sequential association occurred at both gaps one and two. An empirical data analysis replicated the simulation result and found that table sum and adjusting unequal conditional probabilities of events might be useful in examining how the sequential association between learning events is related to learning gains. Current educational studies have ignored conditional sequential associations between learning events. However, conditional sequential associations may contribute to a more comprehensive understanding of students’ learning process because they consider the impact of the context (i.e., preceding events) on the sequential association. Such conceptual understanding may have practical implications for adaptive scaffolding. Study two investigated whether conditional and marginal sequential associations differed in learning processes. It found that conditional and marginal sequential associations between affective states were not equal because (1) the conditional strengths of a sequential association might be heterogeneous (e.g., the sequential association from frustration to boredom was weaker when the affective state before frustration was also frustration than when the prior state was not frustration), and (2) even when the conditional strengths were homogeneous, they were still unequal to the marginal strength. Sequential pattern mining often returns excessive patterns of learning events, and many may be noise. This issue exacerbates the researchers’ burden in interpreting the patterns to obtain actionable insights into the learning process and the risk of an incorrect understanding. Study three proposed a permutation test for detecting noisy sequential patterns among the results produced by traditional sequential pattern mining algorithms. Simulations indicated that the permutation test had a low false discovery rate and high power in most conditions. The results suggest that the permutation test distinguishes a noisy pattern and a reliable pattern. An empirical data analysis found that the result of the permutation test in one dataset was generalizable to another dataset from a different student sample and a different learning topic. Each study has limitations and can be improved in future studies. Nevertheless, the proposed methods and findings of these studies may assist researchers in applying sequential pattern analyses to understand the relative arrangement of learning events in time. Based on the results of these and prior studies, the dissertation proposes a guideline to help researchers to consider practical questions about using sequential pattern analyses in education: (1) whether sequential pattern analyses fit the research purpose; (2) if so, which methods are the best for answering the research questions; (3) how to interpret sequential patterns.

Graduation Semester

2022-12

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Yingbin Zhang

Owning Collections