eight_tick_forces_D3

"we experimentally realize an array of optical tweezers trapping over 6,100 neutral atoms in around 12,000 sites, simultaneously surpassing state-of-the-art performance for several metrics that underpin the success of the platform. Specifically, while scaling to such a large number of atoms, we demonstrate a coherence time of 12.6(1) seconds, a record for hyperfine qubits in an optical tweezer array."

"The Compressive Transformer uses the same attention mechanism over its set of memories and compressed memories, learning to query both its short-term granular memory and longer-term coarse memory."

"XLNet integrates ideas from Transformer-XL... into pretraining"

"a generic, efficient framework for deep learning on dynamic graphs represented as sequences of timed events"

"Trained with 24K interleaved image-text contexts, it can seamlessly extend to 96K long contexts via RoPE extrapolation."

"Based on the observation of multi-periodicity in time series, we ravel out the complex temporal variations into the multiple intraperiod- and interperiod-variations. To tackle the limitations of 1D time series in representation capability, we extend the analysis of temporal variations into the 2D space by transforming the 1D time series into a set of 2D tensors based on multiple periods."

"By traversing along four scanning routes, SS2D bridges the gap between the ordered nature of 1D selective scan and the non-sequential structure of 2D vision data, which facilitates the collection of contextual information from various sources and perspectives."

"Inspired by the State Space Sequence Models (SSMs), a new family of deep sequence models known for their strong capability in handling long sequences, we design a hybrid CNN-SSM block that integrates the local feature extraction power of convolutional layers with the abilities of SSMs for capturing the long-range dependency."

"We build on the design of the S4 layer and introduce a new state space layer, the S5 layer. Whereas an S4 layer uses many independent single-input, single-output SSMs, the S5 layer uses one multi-input, multi-output SSM."

"we leverage recent advancements in scaling context window size, particularly Blockwise RingAttention (Liu et al., 2024; Liu and Abbeel, 2023), a technique that scales context size without approximations or overheads, enabling efficient training on long sequences... progressively increase the effective context length of the model across 5 stages: 32K, 128K, 256K, 512K, and 1M"

"Chunked-prefills allows constructing multiple decode-maximal batches from a single prefill request, maximizing coverage of decodes that can piggyback."

"The final mass and final spin as inferred from the low- and high-frequency parts of the waveform are consistent with each other."

"ADR automatically generates a distribution over randomized environments of ever-increasing difficulty. Control policies and vision state estimators trained with ADR exhibit vastly improved sim2real transfer."

"We introduce the gated delta rule and develop a parallel training algorithm optimized for modern hardware... preserves the benefits of chunkwise parallelism"

"HEALPix – the Hierarchical Equal Area iso-Latitude Pixelization – is a versatile data structure... Originally developed to address the data processing and analysis needs... In this paper we consider the requirements and constraints to be met in order to implement a sufficient framework for the efficient discretization and fast analysis/synthesis of functions defined on the sphere"

"DeepSpeed-Ulysses at its core partitions input data along the sequence dimension and employs an efficient all-to-all collective communication for attention computation. Theoretical communication analysis shows that whereas other methods incur communication overhead as sequence length increases, DeepSpeed-Ulysses maintains constant communication volume when sequence length and compute devices are increased proportionally."

"We show that this method trains a 1.3 billion parameter model on input sequences of length 1024 that extrapolates to input sequences of length 2048, achieving the same perplexity as a sinusoidal position embedding model trained on inputs of length 2048 but training 11% faster and using 11% less memory."

"We demonstrate that such a giant model can efficiently be trained on 2048 TPU v3 accelerators in 4 days"

"We introduce sparse factorizations of the attention matrix which reduce this to O(n√n)... We call networks with these changes Sparse Transformers, and show they can model sequences tens of thousands of timesteps long using hundreds of layers... setting a new state of the art for density modeling of Enwik8, CIFAR-10, and ImageNet-64."

"the proposed RoPE encodes the absolute position with a rotation matrix and meanwhile incorporates the explicit relative position dependency in self-attention formulation"

"The Qwen3 MoE models have 128 total experts with 8 activated experts per token."

"3π Steradian Survey covers the sky north of Dec=−30 in five filters (grizy_P1)"

"this solution can be thought of as a pre-geometric de Sitter spacetime, which also provides a novel solution for the problem of the Big Bang singularity"

"g_eff = g√N drives system into ultra-strong coupling; provides ceiling N_max on Tavis-Cummings validity"