REVIEW 3 major objections 5 minor 43 references
Retrieval of Surface Solar Radiation through Implicit Albedo Recovery from Temporal Context
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A satellite solar-radiation model that never sees surface albedo can match the accuracy of an albedo-informed model once it is given a 40-hour window of past observations, because the temporal context lets it reconstruct the clear-sky…
desk verdict Temporal context genuinely closes most of the albedo gap in SSR retrieval, but the paper's equivalence claim rests on single-run RMSE gaps that could be noise. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is TSViT-r, a dual-encoder vision transformer that factorizes attention into a temporal encoder followed by a spatial encoder. Each image patch is tokenized along the time axis, the temporal transformer compresses the multi-step spectral history of a pixel into a class token, the spatial transformer then relates those tokens across the image, and a regression head outputs SSR per pixel. This architecture is what lets a window of past SEVIRI observations act as an implicit compositing buffer, replacing the explicit clear-sky reflectance statistics and albedo maps used by Heliosat-style methods.
What would settle it
Train the same model on context windows whose past frames are randomly permuted: if shuffling preserves the RMSE gain, the model is relying on aggregate statistics rather than temporal recovery of clear-sky reflectance. Conversely, remove all clear-sky frames from the context window; if the convergence to albedo-informed accuracy disappears, the implicit-albedo explanation is confirmed.
Extended reading notes
Core claim
The central claim is that temporal context acts as a soft memory for surface conditions: a model fed a sequence of past satellite images can internally reconstruct the clear-sky background reflectance that physics-based algorithms estimate through explicit albedo compositing. The paper shows this by training TSViT-r with and without albedo as an input across context lengths from T=1 to T=120. Without albedo, RMSE falls steadily as context grows and plateaus near 72 W/m2 at T=40; with albedo, RMSE is roughly 71 W/m2 and nearly flat across context sizes. The convergence is strongest for bright snow-covered surfaces above 2200 m and for dark, overcast lowlands, and the spatial map of residual differences between the two models fades as context lengthens. The authors conclude that the model implicitly recovers background reflectance through the stability of extreme ground albedo values relative to dynamic atmospheric conditions.
Load-bearing premise
The evaluation assumes that HelioMont's no-horizon SSR product, which excludes terrain shadowing and reflections, is the right target to emulate, and that SwissMetNet ground measurements, which include those effects, are a valid reference for that target; if this mismatch is large, the reported high-elevation improvement could reflect the model compensating for inconsistent labels rather than genuine physical skill.
Editorial extensions
If this is right
- Temporal context can substitute for explicit albedo input in SSR retrieval, with a 40-step window closing most of the accuracy gap (RMSE about 72 W/m2 versus 71 W/m2 over the full test area).
- The substitution effect is strongest where background reflectance is dynamic: bright snow-covered surfaces above 2200 m and dark overcast lowlands, exactly the cases where monthly reflectance percentiles fail.
- HeMu matches or beats HelioMont against 87 SwissMetNet ground stations, with RMSE of 134.0 W/m2 for instantaneous, 52.9 W/m2 for daily, and 22.5 W/m2 for monthly estimates, while running about 10 times faster.
- The attention-based TSViT-r outperforms the convolutional ConvResNet baseline in every sky-clearness and albedo condition, and remains robust when input features are removed or permuted.
- Operational retrieval in mountainous or snow-affected regions could drop hand-crafted albedo maps, cloud masks, and other engineered features without sacrificing accuracy.
Reading between the lines
- Editorial inference: if the implicit-albedo mechanism is real, the temporal class token should encode an albedo-like quantity; probing it with a simple linear readout against measured surface albedo would make the mechanism directly testable and could support transfer to other sensors.
- Editorial inference: the model's success likely depends on clear-sky frames appearing inside the context window; training on windows containing only overcast frames, or with shuffled temporal order, would distinguish true compositing of clear-sky reflectance from mere averaging of the window.
- Editorial inference: because the HelioMont target excludes horizon effects while the SwissMetNet ground reference includes them, part of the reported high-elevation improvement may reflect the model compensating for a target-versus-reference inconsistency; re-training on a horizon-inclusive target would isolate genuine physical skill.
- Editorial inference: the same architecture should be testable in other snow-prone regions or with other geostationary sensors, since the paper validates the convergence only over Switzerland.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TSViT-r (HeMu), a Temporo-Spatial Vision Transformer that emulates the HelioMont surface solar radiation retrieval algorithm from sequences of SEVIRI satellite images plus static topography and solar geometry features. The central claim is that increasing the temporal context length (up to T=120) lets an albedo-free model implicitly recover background reflectance, matching the RMSE of a model that receives explicit surface albedo as input, with the largest gains in bright, snow-covered, high-elevation areas. The authors also report a convolutional baseline, feature ablation and permutation importance experiments, and validation against 87 SwissMetNet ground stations at instantaneous, daily, and monthly aggregations, together with a computational speedup benchmark.
Significance. If the central convergence claim holds, the result is practically and scientifically meaningful: it would show that learned temporal compositing can replace hand-crafted albedo and cloud-mask inputs, simplifying operational SSR retrieval in complex terrain. The study has concrete strengths: a clean behavioral experiment (albedo on/off with context length sweep), a credible attention-based architecture, publicly available code and data, and a validation chain from the emulated product to independent ground stations. The main weakness is that the central claim is supported by single training runs without uncertainty quantification, so the reported RMSE gaps between configurations are not yet shown to be larger than run-to-run noise; the ground-station validation also mixes a target product that excludes terrain shadowing with station measurements that include it.
major comments (3)
- [§4.1, Table 4.1] The central claim that temporal context substitutes for explicit albedo is not established because each configuration is reported for a single training run. At T=40 the albedo-free RMSE is 72.6 versus 71.0 W/m2 for the albedo-informed model, and at T=120 it is 73.3 versus 70.6; meanwhile the albedo-free sequence is non-monotonic after T=30 (73.7, 72.6, 73.9, 73.3 for T=30, 40, 80, 120). This variation is of the same order as the alleged convergence gap. Please provide multiple seeds with confidence intervals, or a significance test, for at least the key contrast (T=40 and T=120, with and without albedo), and report the distribution of RMSE across runs so the reader can judge whether the residual gap is distinguishable from noise.
- [§3.3 vs §4.2, Table 4.2] The claim that HeMu improves over HelioMont at ground stations is affected by a target/reference mismatch. Section 3.3 states that the target product excludes terrain shadowing and reflections, while station measurements of GHI include horizon and terrain effects. The reported HeMu improvement at 2000–4000 m (instantaneous RMSE 154.5 vs 173.0 W/m2) may partly reflect the model compensating for this inconsistency rather than genuine physical skill. Please quantify the mismatch, for example by evaluating against a HelioMont product that includes horizon effects, or by explicitly discussing the magnitude of the excluded terrain contributions and how they scale with elevation in the study region.
- [§4.1, Table 4.1] The text says that beyond T=40 the albedo-free estimates 'plateau at RMSE ≈70 W/m2', but Table 4.1 reports 73.9 W/m2 at T=80 and 73.3 W/m2 at T=120, i.e. 3–4 W/m2 above that value. This discrepancy overstates the degree of convergence between the albedo-free and albedo-informed curves. Please correct the text to match the reported numbers, or justify an alternative summary statistic that supports the '≈70' statement.
minor comments (5)
- [§3.3, Table 3.1] The latitude range is inconsistent: the text states [45.75°N-47.88°N] while Table 3.1 lists 45.75°-47.75° for both target and features; please align them.
- [§3.3, Table 3.1] The SEVIRI band labeled 'Infrared 0.16µm (IR016)' should be 1.6 µm; the wavelength notation appears to be off by a factor of ten.
- [§3.2, §3.5, Appendix E, Appendix F] Several typos occur throughout: 'devided' for 'divided' (Figure 3.1), 'the the previous experiments' for 'the previous experiments' (Section 3.5), 'repsectively' for 'respectively' (Appendix B), 'archtecture' for 'architecture' (Appendix F), and 'instantenous' for 'instantaneous' (Section 4.2).
- [§4.1, Figure 4.1] The subplot caption uses 'k∗ t' while the text uses k∗ T for the clear-sky index; please use a single notation for the time step and the clear-sky index to avoid confusion.
- [Table 4.1] The baseline ConvResNet is reported only for context size 1, and the column header layout makes this easy to miss; please state explicitly in the table or caption that the baseline model does not use temporal context.
Circularity Check
No significant circularity: the temporal-context result is an empirical behavioral comparison, not an identity or a renamed fit.
full rationale
The central claim is that a temporal-context-aware TSViT-r model can match the performance of an albedo-informed model without receiving surface albedo as input. This is tested by training regression models on a fixed target (HelioMont instantaneous SSR) and varying only whether the input set contains HelioMont's internal albedo and how long the input temporal window is. No parameter is fitted to a subset and then reported as a prediction of a closely related quantity; the RMSE values in Table 4.1 and Figure 4.1 are held-out test-set outcomes. The target is not defined in terms of the model's predictions, and the temporal context is not derived from the target. The albedo input used in the comparison is a feature, not the target, and the albedo-free model's outputs are not constructed from it; the convergence at T=40 is an empirical outcome rather than a mathematical identity. The only self-citation in the paper ([28], co-authored by M. Lehning) supports an ancillary statement about anisotropic surface reflectance and is not load-bearing. The acknowledged limitation that HeMu is an emulator of HelioMont and inherits its systematic errors (Section 5.3) is a correctness/generalization concern, not circularity. Likewise, possible run-to-run variability and the target-versus-ground-station reference mismatch are robustness concerns and do not make the derivation circular. The derivation chain is self-contained against an external benchmark (HelioMont target and SwissMetNet station data), so the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- Temporal context length T =
40 time steps
- Patch size =
3x3x1
- Latent dimension d =
128
- Attention window =
48x48
assumptions (4)
- domain assumption SEVIRI satellite observations and HelioMont SSR targets are spatiotemporally aligned to the 1.7 km grid without significant error.
- domain assumption Solar angles, fcorr, and topography features computed for a single year are valid for all other years.
- domain assumption SwissMetNet 10-minute averaged ground measurements are an appropriate reference for instantaneous 1.7 km pixel SSR estimates.
- domain assumption HelioMont's SSR product is a suitable training target, and its internal albedo is a fair proxy for surface albedo when used as an input feature.
Cite this review
Pith. "Pith review of Retrieval of Surface Solar Radiation through Implicit Albedo Recovery from Temporal Context." pith.science (2026). https://pith.science/paper/TALRZSS6
@misc{pith2026250610174,
author = {Pith},
title = {Pith review of: Retrieval of Surface Solar Radiation through Implicit Albedo Recovery from Temporal Context},
year = {2026},
howpublished = {\url{https://pith.science/paper/TALRZSS6}},
note = {Machine review of arXiv:2506.10174}
}
read the original abstract
Accurate retrieval of surface solar radiation (SSR) from satellite imagery critically depends on estimating the background reflectance that a spaceborne sensor would observe under clear-sky conditions. Deviations from this baseline can then be used to detect cloud presence and guide radiative transfer models in inferring atmospheric attenuation. Operational retrieval algorithms typically approximate background reflectance using monthly statistics, assuming surface properties vary slowly relative to atmospheric conditions. However, this approach fails in mountainous regions where intermittent snow cover and changing snow surfaces are frequent. We propose an attention-based emulator for SSR retrieval that implicitly learns to infer clear-sky surface reflectance from raw satellite image sequences. Built on the Temporo-Spatial Vision Transformer, our approach eliminates the need for hand-crafted features such as explicit albedo maps or cloud masks. The emulator is trained on instantaneous SSR estimates from the HelioMont algorithm over Switzerland, a region characterized by complex terrain and dynamic snow cover. Inputs include multi-spectral SEVIRI imagery from the Meteosat Second Generation platform, augmented with static topographic features and solar geometry. The target variable is HelioMont's SSR, computed as the sum of its direct and diffuse horizontal irradiance components, given at a spatial resolution of 1.7 km. We show that, when provided a sufficiently long temporal context, the model matches the performances of albedo-informed models, highlighting the model's ability to internally learn and exploit latent surface reflectance dynamics. Our geospatial analysis shows this effect is most powerful in mountainous regions and improves generalization in both simple and complex topographic settings. Code and datasets are publicly available at https://github.com/frischwood/HeMu-dev.git
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
IEA. Renewables 2024. Technical report, IEA, Paris, 2024. Licence: CC BY 4.0
work page 2024
-
[2]
Quentin Paletta, Guillermo Terrén-Serrano, Yuhao Nie, Binghui Li, Jacob Bieker, Wenqi Zhang, Laurent Dubus, Soumyabrata Dev, and Cong Feng. Advances in solar forecasting: Computer vision with deep learning.Advances in Applied Energy, 11:100150, September 2023
work page 2023
-
[3]
Neha Sehrawat, Sahil Vashisht, and Amritpal Singh. Solar irradiance forecasting models using machine learning techniques and digital twin: A case study with comparison.International Journal of Intelligent Networks, 4:90–102, 2023
work page 2023
-
[4]
Sameer Al-Dahidi, Mohammad Alrbai, Hussein Alahmer, Bilal Rinchi, and Ali Alahmer. Enhancing solar photo- voltaic energy production prediction using diverse machine learning models tuned with the chimp optimization algorithm.Scientific Reports, 14(1):18583, August 2024
work page 2024
-
[5]
A. Carpentieri, D. Folini, J. Leinonen, and A. Meyer. Extending intraday solar forecast horizons with deep generative models.Applied Energy, 377:124186, January 2025
work page 2025
-
[6]
Alberto Carpentieri, Jussi Leinonen, Jeff Adie, Boris Bonev, Doris Folini, and Farah Hariri. Data-driven Surface Solar Irradiance Estimation using Neural Operators at Global Scale, November 2024. arXiv:2411.08843 [physics]
arXiv 2024
-
[7]
Richard Perez, Robert Seals, and Antoine Zelenka. Comparing satellite remote sensing and ground network measurements for the production of site/time specific irradiance data.Solar Energy, 60(2):89–96, February 1997
work page 1997
-
[8]
P . Ineichen and R. Perez. Derivation of Cloud Index from Geostationary Satellites and Application to the Production of Solar Irradiance and Daylight Illuminance Data.Theoretical and Applied Climatology, 64(1-2):119–130, October 1999
work page 1999
Show all 43 references
-
[9]
Estimating surface solar irradiance from satellites: Past, present, and future perspectives.Remote Sensing of Environment, 233:111371, November 2019
Guanghui Huang, Zhanqing Li, Xin Li, Shunlin Liang, Kun Yang, Dongdong Wang, and Yi Zhang. Estimating surface solar irradiance from satellites: Past, present, and future perspectives.Remote Sensing of Environment, 233:111371, November 2019
2019
-
[10]
Schroedter-Homscheidt, F
M. Schroedter-Homscheidt, F . Azam, J. Betcke, N. Hanrieder, M. Lefèvre, L. Saboret, and Y. -M. Saint-Drenan. Surface solar irradiation retrieval from MSG/SEVIRI based on APOLLO Next Generation and HELIOSAT-4 methods. Meteorologische Zeitschrift, 31(6):455–476, December 2022
2022
-
[11]
SARAH-3 – satellite-based climate data records of surface solar radiation.Earth System Science Data, 16(11):5243–5265, November 2024
Uwe Pfeifroth, Jaqueline Drücke, Steffen Kothe, Jörg Trentmann, Marc Schröder, and Rainer Hollmann. SARAH-3 – satellite-based climate data records of surface solar radiation.Earth System Science Data, 16(11):5243–5265, November 2024
2024
-
[12]
Castelli, R
M. Castelli, R. Stöckli, D. Zardi, A. Tetzlaff, J.E. Wagner, G. Belluardo, M. Zebisch, and M. Petitta. The HelioMont method for assessing solar irradiance over complex terrain: Validation and improvements.Remote Sensing of Environment, 152:603–613, September 2014
2014
-
[13]
Cano, J.M
D. Cano, J.M. Monget, M. Albuisson, H. Guillard, N. Regas, and L. Wald. A method for the determination of the global solar radiation from meteorological satellite data.Solar Energy, 37(1):31–39, 1986
1986
-
[14]
The libRadtran software package for radiative transfer calculations (version 2.0.1).Geoscientific Model Development, 9(5):1647–1672, May 2016
Claudia Emde, Robert Buras-Schnell, Arve Kylling, Bernhard Mayer, Josef Gasteiger, Ulrich Hamann, Jonas Kylling, Bettina Richter, Christian Pause, Timothy Dowling, and Luca Bugliaro. The libRadtran software package for radiative transfer calculations (version 2.0.1).Geoscienti...
2016
-
[15]
Lefèvre, A
M. Lefèvre, A. Oumbe, P . Blanc, B. Espinar, B. Gschwind, Z. Qu, L. Wald, M. Schroedter-Homscheidt, C. Hoyer- Klick, A. Arola, A. Benedetti, J. W . Kaiser, and J.-J. Morcrette. McClear: a new model estimating downwelling solar radiation at ground level in clear-sky conditions....
2013
-
[16]
Carpentieri, D
A. Carpentieri, D. Folini, M. Wild, L. Vuilleumier, and A. Meyer. Satellite-derived solar radiation for intra-hour and intra-day applications: Biases and uncertainties by season and altitude.Solar Energy, 255:274–284, May 2023
2023
-
[17]
A deep learning algorithm to estimate hourly global solar radiation from geostationary satellite data.Renewable and Sustainable Energy Reviews, 114:109327, October 2019
Hou Jiang, Ning Lu, Jun Qin, Wenjun Tang, and Ling Yao. A deep learning algorithm to estimate hourly global solar radiation from geostationary satellite data.Renewable and Sustainable Energy Reviews, 114:109327, October 2019. 15
2019
-
[18]
Predicting surface solar radiation using a hybrid radiative Transfer–Machine learning model.Renewable and Sustainable Energy Reviews, 173:113105, March 2023
Yunbo Lu, Lunche Wang, Canming Zhu, Ling Zou, Ming Zhang, Lan Feng, and Qian Cao. Predicting surface solar radiation using a hybrid radiative Transfer–Machine learning model.Renewable and Sustainable Energy Reviews, 173:113105, March 2023
2023
-
[19]
A state of art review on estimation of solar radiation with various models.Heliyon, 9(2):e13167, February 2023
Ali Etem Gurel, Umit Agbulut, Huseyin Bakır, Alper Ergun, and Gokhan Yıldız. A state of art review on estimation of solar radiation with various models.Heliyon, 9(2):e13167, February 2023
2023
-
[20]
K. R. Schuurman and A. Meyer. Surface solar radiation: AI satellite retrieval can outperform Heliosat and general- izes well to other climate zones, September 2024. arXiv:2409.16316 [physics]
2024
-
[21]
Retrieval of surface solar irradiance from satellite imagery using machine learning: pitfalls and perspectives.Atmospheric Measurement Techniques, 16(18):4165–4181, September 2023
Hadrien Verbois, Yves-Marie Saint-Drenan, Vadim Becquet, Benoit Gschwind, and Philippe Blanc. Retrieval of surface solar irradiance from satellite imagery using machine learning: pitfalls and perspectives.Atmospheric Measurement Techniques, 16(18):4165–4181, September 2023
2023
-
[22]
ViTs for SITS: Vision Transformers for Satellite Image Time Series, April 2023
Michail Tarasiou, Erik Chavez, and Stefanos Zafeiriou. ViTs for SITS: Vision Transformers for Satellite Image Time Series, April 2023. arXiv:2301.04944 [cs]
2023 arXiv
-
[23]
Solar energy assessment using remote sensing technologies.Remote Sensing of Environ- ment, 86(3):423–432, August 2003
Annette Hammer, Detlev Heinemann, Carsten Hoyer, Rolf Kuhlemann, Elke Lorenz, Richard Müller, and Hans Georg Beyer. Solar energy assessment using remote sensing technologies.Remote Sensing of Environ- ment, 86(3):423–432, August 2003
2003
-
[24]
Bird and Carol Riordan
Richard E. Bird and Carol Riordan. Simple Solar Spectral Model for Direct and Diffuse Irradiance on Horizontal and Tilted Planes at the Earth’ s Surface for Cloudless Atmospheres.Journal of Climate and Applied Meteorology, 25(1):87–97, January 1986
1986
-
[25]
T . Muneer. Solar radiation model for Europe.Building Services Engineering Research and Technology, 11(4):153–163, November 1990
1990
-
[26]
John E. Hay. Calculating solar radiation for inclined surfaces: Practical approaches.Renewable Energy, 3(4):373– 380, June 1993
1993
-
[27]
Bin Tan, Robert Wolfe, Jeffrey Masek, Feng Gao, and Eric F . Vermote. An illumination correction algorithm on Landsat-TM data. In2010 IEEE International Geoscience and Remote Sensing Symposium, pages 1964–1967, Honolulu, HI, USA, July 2010. IEEE
1964
-
[28]
von Rütte, A
F . von Rütte, A. Kahl, J. Rohrer, and M. Lehning. How Forward-Scattering Snow and Terrain Change the Alpine Radiation Balance With Application to Solar Panels.Journal of Geophysical Research: Atmospheres, 126(15), 2021
2021
-
[29]
Khlopenkov and Alexander P
Konstantin V . Khlopenkov and Alexander P . Trishchenko. SPARC: New Cloud, Snow, and Cloud Shadow Detection Scheme for Historical 1-km AVHHR Data over Canada.Journal of Atmospheric and Oceanic Technology, 24(3):322– 343, March 2007
2007
-
[30]
Shuttle Radar Topography Mission (SRTM) Global, 2013
OpenTopography. Shuttle Radar Topography Mission (SRTM) Global, 2013
2013
-
[31]
The HelioMont Surface Solar Radiation Processing (2022 Version)
Reto Stöckli. The HelioMont Surface Solar Radiation Processing (2022 Version). Scientific 93, MeteoSwiss, December 2022
2022
-
[32]
MSG Project, ESA Directorate of Earth Observation, ESTEC, Noordwijk, The Netherlands
W Schumann, H Stark, K McMullan, D Aminou, and H-J Luhmann. MSG Project, ESA Directorate of Earth Observation, ESTEC, Noordwijk, The Netherlands. 2002
2002
-
[33]
Five new algorithms for the computation of sun position from 2010 to 2110.Solar Energy, 86(5):1323–1337, May 2012
Roberto Grena. Five new algorithms for the computation of sun position from 2010 to 2110.Solar Energy, 86(5):1323–1337, May 2012
2010
-
[34]
Müller and Dieter Scherer
Mathias D. Müller and Dieter Scherer. A Grid- and Subgrid-Scale Radiation Parameterization of Topographic Effects for Mesoscale Weather Forecast Models.Monthly Weather Review, 133(6):1431–1442, June 2005
2005
-
[35]
NASADEM Global 1 arc-second Digital Elevation Model, 2021
OpenTopography. NASADEM Global 1 arc-second Digital Elevation Model, 2021
2021
-
[36]
Steger, Benjamin Steger, and Christoph Schär
Christian R. Steger, Benjamin Steger, and Christoph Schär. HORAYZON v1.2: an efficient and flexible ray-tracing algorithm to compute horizon and sky view factor.Geoscientific Model Development, 15(17):6817–6840, September 2022
2022
-
[37]
Automatic measurement network - MeteoSwiss. 16
-
[38]
pytroll/satpy: Version 0.54.0 (2025/01/20), January 2025
Martin Raspaud, David Hoese, Panu Lahtinen, Gerrit Holl, Simon Proud, Stephan Finkensieper, Andrea Meraner, Adam Dybbroe, Johan Strandgren, yukaribbba, Joleen Feltz, Sauli Joro, BENR0, Xin Zhang, Pierre de Buyl, Gionata Ghiggi, William Roberts, Youva, Lars Ørum Rasmussen, Oliv...
2025
-
[39]
All Models are Wrong, but Many are Useful: Learning a Variable’ s Importance by Studying an Entire Class of Prediction Models Simultaneously
Aaron Fisher, Cynthia Rudin, and Francesca Dominici. All Models are Wrong, but Many are Useful: Learning a Variable’ s Importance by Studying an Entire Class of Prediction Models Simultaneously. 2021
2021
-
[40]
Decoupled Weight Decay Regularization, 2017
Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization, 2017. Version Number: 3
2017
-
[41]
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, June 2021
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at...
2021 arXiv
-
[42]
Gomez, Lukasz Kaiser, and Illia Polosukhin
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention Is All You Need, August 2023. arXiv:1706.03762 [cs]
2023 arXiv
-
[43]
correction factors for downwelling direct shortwave radiation
Alex Graves. Generating Sequences With Recurrent Neural Networks, June 2014. arXiv:1308.0850 [cs]. Appendix A Vision Transformers (ViTs) A Vision Transformer (ViT) [41] is an adaptation of the Transformer architecture [42] to image data. It treats fixed-size image patches as t...
2014 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.