Pith. sign in

REVIEW 9 cited by

Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.12387 v1 pith:6LF7IY6F submitted 2024-04-18 cs.CL cs.CV

Reka Team: Aitor Ormazabal , Che Zheng , Cyprien de Masson d'Autume , Dani Yogatama , Deyu Fu , Donovan Ong , Eric Chen , Eugenie Lamprecht
show 17 more authors
Hai Pham Isaac Ong Kaloyan Aleksiev Lei Li Matthew Henderson Max Bain Mikel Artetxe Nishant Relan Piotr Padlewski Qi Liu Ren Chen Samuel Phua Yazheng Yang Yi Tay Yuqi Wang Zhongkai Zhu Zhihui Xie
This is my paper · ORCID
classification cs.CLcs.CV
keywords modelsrekacorebenchmarksedgeevaluationflashhuman
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

We introduce Reka Core, Flash, and Edge, a series of powerful multimodal language models trained from scratch by Reka. Reka models are able to process and reason with text, images, video, and audio inputs. This technical report discusses details of training some of these models and provides comprehensive evaluation results. We show that Reka Edge and Reka Flash are not only state-of-the-art but also outperform many much larger models, delivering outsized values for their respective compute class. Meanwhile, our most capable and largest model, Reka Core, approaches the best frontier models on both automatic evaluations and blind human evaluations. On image question answering benchmarks (e.g. MMMU, VQAv2), Core performs competitively to GPT4-V. Meanwhile, on multimodal chat, Core ranks as the second most preferred model under a blind third-party human evaluation setup, outperforming other models such as Claude 3 Opus. On text benchmarks, Core not only performs competitively to other frontier models on a set of well-established benchmarks (e.g. MMLU, GSM8K) but also outperforms GPT4-0613 on human evaluation. On video question answering (Perception-Test), Core outperforms Gemini Ultra. Models are shipped in production at http://chat.reka.ai . A showcase of non cherry picked qualitative examples can also be found at http://showcase.reka.ai .

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?

    cs.CV 2024-12 conditional novelty 7.0 of 10

    A new audio-visual benchmark shows current multimodal LLMs perform barely above random guessing, with audio perception errors as the dominant failure mode.

  2. MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A new multimodal benchmark of 3,614 circuit QA pairs shows that large language models perform worst on back-end layout and computation tasks, and that current models generally underperform on circuit design questions.

  3. ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models

    cs.CV 2025-02 conditional novelty 6.0 of 10

    ZeroBench is a hand-built 100-question visual reasoning benchmark, adversarially filtered so every evaluated frontier LMM scored 0% at release.

  4. AGAV-Rater: Adapting Large Multimodal Model for AI-Generated Audio-Visual Quality Assessment

    cs.MM 2025-01 conditional novelty 6.0 of 10

    AGAV-Rater, an LMM fine-tuned in two stages, achieves state-of-the-art quality scores for AI-generated audio-visual content, text-to-audio, and text-to-music.

  5. CL-Attack: Textual Backdoor Attacks via Cross-Lingual Triggers

    cs.CR 2024-12 conditional novelty 6.0 of 10

    A cross-lingual paragraph structure, a fixed language-order sequence of segments, can serve as a stealthy backdoor trigger in fine-tuned LLMs, achieving high attack success at 3-5% poisoning.

  6. From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalities

    cs.AI 2024-12 conditional novelty 6.0 of 10

    A survey that organizes Omni-MLLMs into four architectural components and a taxonomy of encoding, alignment, and generation methods.

  7. Perception Test 2024: Challenge Summary and a Novel Hour-Long VideoQA Benchmark

    cs.CV 2024-11 conditional novelty 6.0 of 10

    The paper presents a new hour-long video question-answering benchmark with 70 long-context questions, plus challenge leaderboard results showing large gaps between models and humans.

  8. LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratch

    cs.LG 2025-01 conditional novelty 5.0 of 10

    K2 Diamond is a fully open 65B-parameter LLM that reaches Llama 2 70B-level performance on standard benchmarks.

  9. Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures

    cs.LG 2025-06 conditional novelty 3.0 of 10

    A review of Relational Deep Learning that formalizes relational entity graphs, catalogs datasets and methods, and advocates for unified GNN and foundation model research directions.

Pith tools