Causal machine learning identifies tactical levers in football defenses

A causal transformer made predictions of critical heat flux more consistent in a test set of experiments involving water-cooled reactors with uneven heating along their length. The model kept the usual method of starting with a prediction for even heating and then correcting it. However, instead of using a fixed formula based on past data, it learned the correction by studying the order of heating conditions along the channel, from upstream to downstream. Across 109 test experiments, the corrected model had an average prediction-to-measurement ratio of 1.028 and a standard deviation of 0.115. A neural network using a traditional correction factor had a mean ratio of 1.063 and a standard deviation of 0.143. The transformer's mean absolute percentage error was 0.094, about 23% lower than the traditional method's value of 0.122. These results came from computer modeling using existing experimental databases, not from a new series of reactor tests. The authors warned that the model performed less well on some heating patterns it had not seen during training.

Critical heat flux is an important heat-related limit in water-cooled nuclear reactors. If heat transfer suddenly becomes less effective, the temperature of the material covering the fuel can rise quickly and may damage it. When heating varies along a channel, the safety margin at a given point depends on more than pressure, the amount of water flowing through the channel, its shape, or the mixture's equilibrium quality. It also depends on where heat was added earlier along the channel. Earlier heating changes how the mixture of liquid and vapor flows and how its temperature and other properties develop. Engineers commonly start with a critical heat-flux value calculated for even heating and then adjust it to account for how heating varies along the channel. The study kept this basic approach. It used a previously trained neural network, which was kept unchanged and had learned from a large database of even-heating experiments, as the starting model. A smaller transformer was trained only to estimate the extra correction needed for uneven heating.

The researchers used a database prepared for benchmarking from measurements of critical heat flux under uneven heating. These measurements had been collected in a report from the Korea Atomic Energy Research Institute. The database contained 888 experiments covering four types of heating patterns: heating strongest near the inlet, strongest in the middle, strongest near the outlet, and patterns with a sharp, narrow spike in heating. Before training the models, the researchers removed 39 cases in which both liquid and vapor were present at the inlet, because the unchanged neural network for even heating gave unreliable predictions for them. They also removed 123 cases that fell outside the neural network's available ranges for channel diameter, mass flux, or equilibrium quality. The remaining 726 experiments were randomly divided by experiment into training, validation, and test sets, containing 70%, 15%, and 15% of the data, respectively. These exclusions define the conditions in which the model can be used; they are not built-in limits of the transformer design itself. The researchers also ran a separate test in which one entire family of heating patterns was left out of both training and validation to see how the model handled patterns it had never learned from.

At each position along the channel, the transformer received a sequence entry containing the normalized position, the local equilibrium quality, and the normalized shape of the heating profile. It also received the average heat flux and mass flux. Mass flux describes how much mass flows through a given area per unit of time. Because the test channels had a constant area and steady flow, mass flux stayed the same along each channel, so the model treated it as a global operating condition. The model had one encoder layer, a 32-dimensional embedding, two attention heads, a 64-unit feed-forward layer, GELU activation, and a dropout rate of 0.10. It did not use a separate position code based on sequence order, because the actual position along the channel was already included in the input. A causal attention mask stopped information from downstream positions from affecting corrections at upstream positions. This meant the correction could use information from the current position and earlier positions, but not from the future. After the attention step, a two-layer feed-forward component predicted an unrestricted value, which was converted into a positive correction factor using a bounded exponential transformation.

The model was trained to predict the minimum departure from nucleate boiling ratio for each experiment. This ratio describes the smallest local thermal safety margin associated with the critical heat-flux limit. The training objective penalized the squared difference between this minimum ratio and one, while also discouraging correction factors from moving too far away from one along the sequence. This approach differs from training on individual local critical heat-flux values, because the experiments provide one critical condition for the whole experiment rather than separate target values at every position along the channel. The researchers said they did not test other training objectives, such as losses based on logarithmic percentage error. Across all positions in the main test set, the predicted correction factors ranged from 0.354 to 2.948. At the positions the transformer identified as limiting, they ranged from 0.814 to 2.493. These values stayed well within the numerical bounds. The authors stressed that the wide bounds were designed to prevent extreme values during training, not to represent physical limits on critical heat flux.

In the main test using a random split, the unchanged neural network for even heating greatly overpredicted the limiting critical heat flux when no correction for uneven heating was applied. Its average prediction-to-measurement ratio was 1.247, its standard deviation was 0.388, and its mean absolute percentage error was 0.313. Applying the traditional correction factor lowered the standard deviation to 0.143 and the mean absolute percentage error to 0.122. The Tong F-factor correction also improved the uncorrected neural network's predictions, but it was less accurate on this test set, with a standard deviation of 0.246 and a mean absolute percentage error of 0.178. Predictions from look-up tables varied even more. However, these predictions were available for only 108 of the 109 test cases, because one case with heating strongest in the middle had a limiting position outside the usable equilibrium-quality range. When the transformer and traditional correction were compared on the same 108 cases, the transformer still had lower prediction scatter and error. This showed that the result did not depend on leaving out the case that could not be evaluated using the look-up tables. The researchers also used paired bootstrap comparisons with 200,000 resamples taken at the experiment level. The 95% confidence intervals for the differences in error and scatter were below zero, supporting better results from the transformer. However, the researchers did not adjust for multiple comparisons.

The results depended on the shape of the heating profile. The transformer produced less prediction scatter and lower mean absolute percentage error for cases where heating was strongest near the inlet or in the middle. These cases made up most of the test set. Among the 15 cases where heating was strongest near the outlet, the transformer's standard deviation was similar to that of the traditional correction, while its error and average prediction ratio were slightly better. The main test set contained only one case with a sharp, narrow spike in heating, so this result was not enough to draw a firm conclusion about that type of profile. The test that withheld an entire profile family showed the model's main weakness. When the middle-peaked family was left out, the transformer still had lower scatter and error. But when the inlet-peaked or outlet-peaked family was left out, the transformer performed worse than the traditional correction. For the unseen inlet-peaked family, the transformer's standard deviation and mean absolute percentage error were 0.332 and 0.349, compared with 0.235 and 0.200 for the traditional correction. For the unseen outlet-peaked family, the corresponding values were 0.250 and 0.224 for the transformer and 0.182 and 0.152 for the traditional method. The model's ability to handle a completely new type of heating profile was therefore inconsistent.

Tests that removed individual inputs showed that the transformer needed several kinds of information, not just one measure of heating. Removing the overall mass flux increased prediction scatter from 0.115 to 0.150, showing that the total mass flow through the channel provided important information for scaling the prediction. Removing the normalized heating profile increased scatter to 0.144, making the model similar to the traditional correction on this measure. Removing the history of equilibrium quality produced the highest mean absolute percentage error among these tests, at 0.115, and caused the average prediction-to-measurement ratio to fall below one. The complete model therefore gave the best overall balance of prediction scatter, bias, and absolute error in the reported test set. The model could also work with different numbers of positions along a channel without being retrained. A model trained using 200 positions was tested using 40, 100, 200, and 400 positions. Its prediction-to-measurement standard deviation stayed between 0.114 and 0.115. The researchers did not test other ways of providing overall operating conditions to the model, so they could not show that their chosen approach was the best one.

Tests of reliability also showed that results varied depending on the model's random starting values during training. Across four training runs using different random seeds, the transformer had a lower mean absolute percentage error than the traditional correction every time. It reduced prediction scatter in three of the four runs. In one run, however, its standard deviation was 0.164, higher than the traditional correction's 0.143. Even in that run, its mean absolute percentage error was lower, at 0.115. The researchers said that more work would be needed to understand or reduce this variation before the model could be used in engineering practice. They suggested combining predictions from several trained models as one possible solution, but they did not test it. When the researchers examined attention patterns at selected limiting positions, they found that the model generally spread its attention across several upstream positions that it was allowed to use, rather than relying on a single position. The attention weights changed with the shape of the heating profile. For some profiles, the model focused more on earlier regions with high heating. In a case where heating was strongest near the outlet, it focused more on the high-heating region farther downstream. The researchers treated these visual patterns as a way to describe how information moved through the model. They did not consider them direct proof of physical cause and effect or a complete explanation of any prediction.

Overall, the study shows that a small causal transformer can learn useful corrections that depend on the sequence of heating conditions along a channel, while keeping the familiar engineering method of correcting a prediction for even heating. Its clearest reported benefit was lower prediction error and less scatter in the main benchmark. Tests that removed inputs, changed the number of positions, used different random starting values, and compared paired bootstrap samples provided further evidence for its performance. However, the study also had clear limits. The database of uneven-heating experiments was small and unevenly distributed across profile types. Results on entirely unseen heating profiles were inconsistent. The comparison also used only one unchanged neural network as its starting predictor for even heating. The researchers did not compare other baseline prediction models or establish a higher allowable design safety margin, because they had not developed the statistical treatment of prediction errors and other uncertainties needed to set such a limit. The authors described their method as a data-driven correction-factor model, not as a model that explains the physical mechanisms behind critical heat flux.

Subject of Research: Using machine learning to correct critical heat-flux predictions when heating varies along a reactor channel

Article Title: A transformer-based correction factor for critical heat flux prediction under non-uniform axial heating

Article References: Bourisaw, R., & Iskhakov, A. S. (2026). A transformer-based correction factor for critical heat flux prediction under non-uniform axial heating. Frontiers in Mechanical Engineering, 12, Article 1980739. https://doi.org/10.3389/fmech.2026.1980739

Image Credits: AI Generated

DOI: 10.3389/fmech.2026.1980739

Keywords: critical heat flux, uneven heating along a channel, causal transformer, neural networks, nuclear reactors, liquid-vapor flow, minimum DNBR, model reliability, correction factor, transformer-based, correction, factor