Bilinear Convolution Decomposition for Causal RL Interpretability

Alice Rigg; Narmeen Oozeer; Sinem Erisken

REVIEW 4 major objections 6 minor 6 references

Bilinear Convolution Decomposition for Causal RL Interpretability

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing ReLU with bilinear convolutions yields an RL agent whose maze-solving mechanisms can be decomposed into eigenfilters and causally validated.

desk verdict The algebra is correct and the protocol is promising, but the ablation section contradicts itself on the key k=1 result, so the causal claims are not yet backed by the experiments. read the letter →

arxiv 2412.00944 v1 pith:3O665CCD submitted 2024-12-01 cs.LG cs.AI

Narmeen Oozeer , Sinem Erisken , Alice Rigg This is my paper · ORCID

classification cs.LGcs.AI

keywords bilinearconvolutionsreinforcementlearninginterpretabilityeigenfilterdecompositionconceptprobescausalvalidationBimpalaProcGenmazesolving

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that swapping the ReLU nonlinearity in convolutional networks for a bilinear gating operation --- replacing Conv2D+ReLU with $(x \ast U) \odot (x \ast V)$ --- gives reinforcement learning agents that perform on par with a standard IMPALA baseline on ProcGen maze and related tasks, while making the network's internals analytically tractable. Because the bilinear layer's local input-output map is a quadratic form, it can be decomposed into an orthonormal basis of eigenfilters with signed eigenvalues. The paper then shows how to combine this decomposition with a linear concept probe: SVD of the probe weights separates channel from spatial dimensions, and the resulting singular channels pick out eigenfilters whose activations implement the probed concept. Applied to a maze-solving agent, the method finds a positive/negative eigenfilter pair that acts as a cheese detector, and ablations show that keeping only the top action eigenvector preserves maze-solving success. The upshot is a path from correlational probing to causal, mechanism-level validation in an RL agent.

What carries the argument

The load-bearing object is the symmetric bilinear matrix $B_{\mathrm{sym}}$ of shape $[nK^2, nK^2]$ for each output channel, obtained by summing outer products $U_j^\top V_k$ over channel pairs and symmetrizing; it turns the gated convolution into one quadratic form. From it, the protocol builds eigenfilters: for an output direction $u$ (a probe-derived singular vector or an action direction), $Q_u = u B_{\mathrm{sym}}$ is a symmetric matrix whose eigendecomposition $Q_u = F^\top \Lambda F$ gives an orthonormal basis of filters $f^i$, each with eigenvalue $\lambda^i$, and the layer's response decomposes as $\sum_i \lambda^i (f^i \ast X)^2$. SVD of a probe's weight matrix reshaped to $[C, wh]$ separates channel-space singular vectors $s_j$ from spatial components, so the probe's total score is a signed, weighted sum of eigenfilter activations. These eigenfilters are the units that ablation studies remove or retain, which is what makes the interpretability claims causal rather than correlational.

What would settle it

Open the trained Bimpala model and check whether its BConv2D layers have nonzero bias parameters. If they do, compare each layer's actual output with the bias-free quadratic form $x^\top B_{\mathrm{sym}} x$ on held-out maze patches; if the difference changes which eigenfilters rank highest or alters the ablation outcomes, the decomposition is incomplete. Retraining with explicit zero biases and rerunning the ablations would settle it.

Watch

Extended reading notes

Core claim

The central claim is that a bilinear convolution layer, defined by $\mathrm{BConv2D}(x,U,V) = (x \ast U) \odot (x \ast V)$, has a closed-form decomposition: for each output channel, the layer's action on any local patch is exactly $x^\top B_{\mathrm{sym}} x$, where $B_{\mathrm{sym}}$ is the symmetric matrix built from the U and V kernels. Eigendecomposing the quadratic form $Q_u = u B_{\mathrm{sym}}$ for a chosen output direction $u$ yields $nK^2$ eigenfilters $f^i$ with real eigenvalues $\lambda^i$, and the layer's contribution to $u$ is $\sum_i \lambda^i (f^i \ast X)^2$. Combined with an SVD of a linear probe's weights into channel singular vectors $s_j$, the probe's score becomes $\sum_{j,i} (s_j \lambda^i_{u_j})(f^i_{u_j} \ast X)^2$, so each eigenfilter's relevance is quantified by a signed product of a singular value and an eigenvalue. The paper demonstrates this machinery on a Bimpala agent trained in ProcGen Maze: probes on residual-block activations reach roughly 99--100\% F1 for cheese position, the top singular channel explains 30\% of probe variance, the top positive and negative eigenfilters form a cheese detector, and ablating all but one action-direction eigenvector preserves maze-solving.

Load-bearing premise

The load-bearing premise is that each trained bilinear convolution layer is fully described by the quadratic form $x^\top B_{\mathrm{sym}} x$ built from the U and V kernels alone, with no bias term and no other nonlinearity; the paper omits biases 'for brevity' in its equations, so if the actual trained layers contain non-negligible biases, the eigenfilter spectra and ablations describe only part of the layer's true input-output map.

Editorial extensions

If this is right

A bilinear RL policy can match a standard ReLU IMPALA policy on ProcGen environments, so analytic interpretability does not come at an obvious performance cost.
Any linear concept probe on a bilinear layer can be rewritten as a signed weighted sum of eigenfilter activations, turning a correlational probe into a list of concrete, ablatable computational units.
Keeping only the top action-direction eigenvector in the final layer is enough to preserve near-perfect maze-solving, implying the policy's decision-relevant computation is concentrated in a low-rank subspace.
In the standard channel basis the eigenfilter spectrum has only two nonzero eigenvalues, while the probe-aligned singular basis has a nondegenerate spectrum, so the probe direction is what makes the mechanism visible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the trained convolution layers turn out to include non-negligible bias parameters, the decomposition should be re-derived with a bias term; otherwise the eigenfilter that gets ablated is not the layer's true computation.
The same SVD-plus-eigendecomposition recipe should transfer to any bilinear layer stack, suggesting a general workflow: train with bilinear gates, probe a concept, decompose the preceding layer toward the probe's top channel, and ablate the ranked eigenfilters to test causality.
The finding that the top negative eigenfilter downweights non-cheese patterns without erasing cheese activation suggests a subtraction mechanism; a testable extension is to check whether this paired positive/negative structure appears for other concepts and environments.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know about arXiv:2412.00944. First, the mathematical core is real: the symmetrized block-matrix decomposition of a bilinear convolution into eigenfilters is a correct and useful extension of Pearce et al.'s bilinear MLP work, and the SVD trick for separating channel and spatial axes is a clever addition. Second, the experimental support for the causal claims is not yet there — Section 4.4 literally contradicts itself about the k=1 ablation, saying in one sentence that the agent fails and in the next that it solves the maze with extra steps. Until that is sorted, treat the sufficiency results with caution.

What is genuinely new: the Bsym construction, the SVD-based channel/spatial separation, and the protocol that connects probe directions to filter decompositions and then tests them by ablation. The equivalence of Bsym and B on all inputs is proved, and the spectral decomposition is parameter-free, so the algebra is solid. The paper also deserves credit for being honest about its preliminary nature and for including a clear limitations section.

Now the soft spots, in proportion. The bias omission is a real gap, not a nitpick: BConv2D is defined without bias, but the paper says biases are omitted 'for brevity' elsewhere, so it's unclear whether the actual trained layers have them. If they do, the quadratic form misses a linear term and the eigenfilter analysis does not describe the layers' true input-output map. The training comparison in Figure 4 has no error bars, which for RL is a significant omission. And the ablation section is internally inconsistent: after saying 'After knocking out the entire layer, maze solving ability is preserved,' it later says with k=1 'all convolutions are ablated' (which would be k=0) and then gives two contradictory outcomes for the same intervention. The stress-test note is right about this; it is a real problem for the causal-validation narrative.

One clarification: the 'top action vector' ablation in Figure 8 targets a different object (the final FC layer) than the eigenfilter ablations in Figures 9-10 (BConv layers). The paper should keep those claims separate. The circularity concern about probes is mild: the ablations are behavioral, so they do test causal relevance, though the selection of components is probe-driven.

Who should read this: people working on mechanistic interpretability of RL vision models, and anyone interested in bilinear layers as a transparency tool. The algebra alone is worth taking seriously. As a referee, I would engage: the method is new and the math checks out, but the experimental reporting needs revision. I would not accept the causal claims in the current draft.

Referee Report

4 major / 6 minor

Summary. The paper proposes replacing ReLU convolutions with bilinear convolutions, BConv2D(x, U, V) = (x ∗ U) ⊙ (x ∗ V), and bilinear fully-connected layers, yielding a variant of IMPALA called Bimpala. It claims that Bimpala matches or occasionally outperforms a simplified ReLU IMPALA on ProcGen environments (Maze, Heist, Plunder, Dodgeball). The central interpretability contribution is a weight-based decomposition: for each output channel, the bilinear convolution is rewritten as a symmetric quadratic form Bsym of dimension nK², whose eigendecomposition yields "eigenfilters" that can be visualized and ablated. The paper then proposes a protocol that trains a linear probe for a concept (cheese location), applies SVD to the probe weights, uses the top singular vectors as output directions for the preceding BConv layer, and decomposes the layer along those directions. Experiments report probe F1 scores, eigenfilter spectra, and ablation results intended to show that a small number of eigenfilters suffices to preserve maze-solving behavior, with a single action eigenvector claimed sufficient in Section 4.3.4.

Significance. Conditional on the empirical claims being correct, the paper offers a concrete template for mechanism-level, interventional interpretability of an RL agent: the decomposition is a closed-form function of the trained weights, and the ablations directly modify the network rather than only analyzing activations. The linear algebra in Section 3.1 is explicit and checkable, and the symmetrization argument that Bsym agrees with B on all inputs appears sound. The paper also honestly states limitations, including the difficulty of data-independent unit interpretation and the omission of batch norm, dropout, and pooling. However, the significance is currently weakened by three load-bearing issues: the training comparison lacks error bars or seed counts, the ablation section contains a direct self-contradiction about the k=1 condition, and the decomposition omits bias terms without evidence that biases are negligible in the trained Bimpala model. These issues must be resolved before the causal claims can be evaluated.

major comments (4)

[Section 4.4] The final paragraph of Section 4.4 contains two adjacent sentences that directly contradict each other: "With k = 1, all convolutions are ablated, and the agent fails to solve the maze (as it reaches the max steps)" and "With k = 1 eigenfilters, we find that the agent does solve the maze, but it takes relatively more time compared to average as it makes some wrong moves." Both sentences refer to the same k=1 condition, so they cannot both be correct. Additionally, the first sentence mislabels k=1 as "all convolutions are ablated," which would be the k=0 condition. This contradiction is load-bearing because the paper's headline sufficiency claim (Section 4.3.4 and Figure 8, "keeping just 1 eigenvector for each output action is enough to preserve near 100% success rate") is contradicted by the statement in Section 4.4 that 2 eigenfilters per channel are needed (Figure 10). Please clarify the exact intervention at k=1, report the success-rate curves for k=0,1,2, and reconcile the discrepancy between Figure 8 and Figure 10.
[Sections 2.1 and 3.1] The decomposition constructs Bsym exclusively from the U and V kernels, with no bias term, and BConv2D in Eq. (2) is defined without bias. However, Section 2.1 states that biases are omitted only "for brevity" in the fully-connected case, which implies that the actual Bimpala model used in Section 4 includes bias parameters. Standard convolutional layers in RL architectures indeed include biases. If biases are present and non-negligible, the layer's input-output map is (x ∗ U + b_U) ⊙ (x ∗ V + b_V), which is not equal to the quadratic form x^T Bsym x. Consequently, the eigenfilter spectra (Figure 6), the probe decomposition (Section 4.2), and the ablations (Section 4.4) would not describe the true computation of the layer. Please either confirm that the trained Bimpala BConv layers have no bias parameters, or extend the decomposition to include the affine terms and demonstrate that the conclusions are unchanged.
[Section 4.1.1 and Figure 4] The claim that "Bimpala matches and occasionally outperforms IMPALA" is a central motivation for the architecture, but Figure 4 shows learning curves without error bars, confidence intervals, or a reported number of seeds. Without this information, the "comparable performance" claim could be within run-to-run noise. Please report mean and standard deviation (or interquartile range) over at least 3–5 independent seeds for each environment, and state the number of seeds used in the figure and in the main text.
[Section 4.2 protocol] The proposed protocol is partly self-referential: a linear probe is trained on the activations, its SVD directions are used to select the output directions for the BConv decomposition, and then the probe's F1 score (Table 1) and the resulting eigenfilter activations are used as evidence that the decomposition captures the concept. Since the probe is fit to the same activations it later selects, high probe accuracy is to be expected. The independent evidence must come from the behavioral ablations (maze-solving success), not from probe accuracy alone. Please report the network's behavior on a held-out set of mazes that were not used for probe training or eigenfilter selection, and explicitly state how the probe and the decomposition are evaluated out-of-sample.

minor comments (6)

[Section 2.2] The notation is confusing when K is used both for the kernel width and as a summation index in Eq. (3); please use K1 and K2 or explicit bounds such as -ℓ to ℓ for clarity.
[Section 3.1] The proof that Bsym agrees with B on all inputs references "the respective red and blue terms," but the colors are not visible in a monochrome printout; please recast this explanation without color references.
[Section 3.1] The notation Xj is used for both the input patch and its flattened vector; please introduce separate symbols (e.g., X̌_j for the flattened patch) to avoid ambiguity.
[Figure 4] The figure would benefit from explicit axis labels and a legend that identifies ReLU vs. Bilinear curves and the number of seeds; currently the caption is the only guide.
[Section 4.3.1 and Table 1] The F1 score of 2.73% for the Value FC layer is surprisingly low; please discuss why the value head does not preserve cheese-position information, or check whether this entry is an error.
[Section 4.3.4 and Figure 7] The text says the action spectrum has "one very large positive eigenvalue," but the figure does not show the negative eigenvalues or the full eigenvalue distribution; please clarify the range of the y-axis and whether negative eigenvalues are present.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the bilinear-convolution decomposition is a self-contained algebraic identity, and the empirical performance and ablation claims are not forced by construction.

full rationale

The central derivation (Section 3.1) starts from the BConv2D definition in Eq. (2) and constructs Bsym from the U and V kernels. The equality between the gated convolution output and X^T Bsym X is proven by expansion and does not presuppose the eigenfilter result. The eigendecomposition in Section 3.2 is a direct application of the spectral theorem to the symmetric matrix Qu, so it is a parameter-free mathematical consequence rather than a fitted prediction. The probe protocol in Section 4.2 uses the SVD of a trained linear probe to choose output directions, and the resulting Qprobe formula is an exact algebraic rewriting of the probe's dependence on the preceding BConv layer. This is a re-expression rather than a circular validation, although the paper's 'causal validation' language is stronger than the intervention evidence it provides. The only self-citation, Pearce et al. (2024), shares author Alice Rigg but is motivational: Section 3 re-derives the decomposition instead of importing it as an unexamined premise. The Bimpala-versus-IMPALA comparison and the behavior-level ablations in Sections 4.1.1, 4.3.4, and 4.4 are empirical results that could have failed and are independent of the decomposition. The internal contradiction in Section 4.4 and the omission of bias terms in the decomposition are correctness risks, not circularity, so they do not raise the circularity score.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the bias-free quadratic form representation of BConv2D, the validity of using a trained probe's SVD directions as interpretable axes, and standard linear algebra. No new physical entities are introduced. The single free parameter is the analysis hyperparameter m, not a fitted constant.

free parameters (1)

Number of singular components m for probe decomposition = 16 (to explain >=90% variance)
Protocol step 2 selects m from the singular value distribution; it shapes the decomposition but is not a constant fit to make the main result work.

assumptions (4)

domain assumption Bilinear convolution layers have zero bias; the quadratic form x^T Bsym x exactly represents the BConv2D output.
Section 3.1 derives Bsym from U and V only; Section 2.1 says 'omitting biases for brevity', indicating biases were omitted from the analysis but may exist in the model.
domain assumption A linear probe trained on activations provides a valid output direction for the preceding layer; the probe's top singular vector is a meaningful axis of computation.
The protocol in Section 4.2 assumes the probe weight vector, after SVD, gives channel-space directions that correspond to how the layer writes to the concept. This is a modeling assumption, not proven by the paper.
standard math The spectral theorem applies: Qu is real symmetric and diagonalizable with orthonormal eigenvectors.
Bsym is constructed to be symmetric in Section 3.1, so eigendecomposition is valid; this is standard linear algebra.
domain assumption The agents trained on the 'easy' ProcGen distribution are representative of general RL performance for this architecture.
Section 4.1.1 trains only on easy distribution, so the claim of comparable performance is restricted to that setting.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Bilinear Convolution Decomposition for Causal RL Interpretability." pith.science (2026). https://pith.science/paper/3O665CCD

@misc{pith2026241200944,
  author       = {Pith},
  title        = {Pith review of: Bilinear Convolution Decomposition for Causal RL Interpretability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3O665CCD}},
  note         = {Machine review of arXiv:2412.00944}
}

read the original abstract

Efforts to interpret reinforcement learning (RL) models often rely on high-level techniques such as attribution or probing, which provide only correlational insights and coarse causal control. This work proposes replacing nonlinearities in convolutional neural networks (ConvNets) with bilinear variants, to produce a class of models for which these limitations can be addressed. We show bilinear model variants perform comparably in model-free reinforcement learning settings, and give a side by side comparison on ProcGen environments. Bilinear layers' analytic structure enables weight-based decomposition. Previous work has shown bilinearity enables quantifying functional importance through eigendecomposition, to identify interpretable low rank structure. We show how to adapt the decomposition to convolution layers by applying singular value decomposition to vectors of interest, to separate the channel and spatial dimensions. Finally, we propose a methodology for causally validating concept-based probes, and illustrate its utility by studying a maze-solving agent's ability to track a cheese object.

Figures

Figures reproduced from arXiv: 2412.00944 by the authors.

**Figure 1.** Bimpala: we modified a simplified IMPALA architecture (black) by replacing the operation ReLU(Conv2D) (Equation 2) with BConv2D which consists of gating 2 Conv2D blocks. We also swap Relu(FC) with FCBilinear (Equation 1). (red) 3.1. Tensor Decomposition of Bilinear Convolutional Tensors Let us consider the output of a convolution layer at location (α, β) for the i-th output channel: u(α, β, i) = Xn j X K k1 X K k2 U… view at source ↗

**Figure 2.** Visualization of the quadratic form derivation for gated bilinear convolutions. The diagram illustrates the transformation from spatial convolution operations to a bilinear matrix form in three stages: (1) The upper path shows the computation of spatial convolutions U (i) with input Xj , producing terms aj . (2) The lower path similarly computes convolutions V (i) with input Xk, producing terms bk. (3) The right sid… view at source ↗

**Figure 3.** Activations for the top positive (left) and negative (right) eigenfilters in the second BConv layer, for the cheese probe’s top singular channel. Activations for a maze with cheese (top) vs without cheese (bottom). Middle plots show the difference between the activations with and without cheese. While the positive filter activates on non-cheese patterns, the negative filter downweighs non-cheese patterns without era… view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Performance comparison between ReLU and Bilinear architectures across four reinforcement learning environments. Each row represents a different environment (Maze, Heist, Plunder, Dodgeball), while columns show different evaluation metrics (expected return, average ent…

**Figure 5.** Figure 5: Left: Probe singular values. Right: Fraction of variance explained [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Left: Eigenfilter spectrum towards top singular channel. Right: Eigenfilter spectrum in the standard channel basis. Singular spectrum is nondegenerate, whereas the channel basis spectra have just two eigenvalues, indicating it is not an informative basis. 4.3.4. ACTION…

**Figure 7.** Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

**Figure 10.** Figure 10: Ablating all but the top k eigenfilters for all output channels in the standard basis, across all layers. The top positive and negative eigenfilters must work together, but it doesn’t tell us anything more granular. 5. Discussion Summary We introduce an approach to in…

**Figure 8.** Figure 8: Keeping just 1eigenvector for each output action is enough to preserve near 100% success rate in solving mazes [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

**Figure 9.** Figure 9: Last Conv Layer: Ablating eigenvectors lead to the agent taking more steps to solve the maze. Top 2 eigenvectors per channel is sufficient for the agent to recover its performance [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

6 extracted references · 1 canonical work pages

[1]

Leveraging procedural generation to benchmark reinforcement learning, 2020

Cobbe, K., Hesse, C., Hilton, J., and Schulman, J. Leveraging procedural generation to benchmark reinforcement learning, 2020. URL https://arxiv.org/abs/1912.01588

arXiv 2020
[2]

Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures, 2018

Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., Legg, S., and Kavukcuoglu, K. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures, 2018. URL https://arxiv.org/abs/1802.01561

arXiv 2018
[3]

T., Dooms, T., Rigg, A., Oramas, J

Pearce, M. T., Dooms, T., Rigg, A., Oramas, J. M., and Sharkey, L. Bilinear mlps enable weight-based mechanistic interpretability, 2024. URL https://arxiv.org/abs/2410.08417

arXiv 2024
[4]

A technical note on bilinear layers for interpretability, 2023

Sharkey, L. A technical note on bilinear layers for interpretability, 2023

work page 2023
[5]

Glu variants improve transformer, 2020

Shazeer, N. Glu variants improve transformer, 2020. URL https://arxiv.org/abs/2002.05202

arXiv 2020
[6]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Leveraging procedural generation to benchmark reinforcement learning, 2020

Cobbe, K., Hesse, C., Hilton, J., and Schulman, J. Leveraging procedural generation to benchmark reinforcement learning, 2020. URL https://arxiv.org/abs/1912.01588

arXiv 2020

[2] [2]

Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures, 2018

Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., Legg, S., and Kavukcuoglu, K. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures, 2018. URL https://arxiv.org/abs/1802.01561

arXiv 2018

[3] [3]

T., Dooms, T., Rigg, A., Oramas, J

Pearce, M. T., Dooms, T., Rigg, A., Oramas, J. M., and Sharkey, L. Bilinear mlps enable weight-based mechanistic interpretability, 2024. URL https://arxiv.org/abs/2410.08417

arXiv 2024

[4] [4]

A technical note on bilinear layers for interpretability, 2023

Sharkey, L. A technical note on bilinear layers for interpretability, 2023

work page 2023

[5] [5]

Glu variants improve transformer, 2020

Shazeer, N. Glu variants improve transformer, 2020. URL https://arxiv.org/abs/2002.05202

arXiv 2020

[6] [6]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...