Pith. sign in

REVIEW 3 cited by

Advancing Neural Encoding of Portuguese with Transformer Albertina PT-*

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.06721 v2 pith:MENTTJRE submitted 2023-05-11 cs.CL

Advancing Neural Encoding of Portuguese with Transformer Albertina PT-*

Jo\~ao Rodrigues , Lu\'is Gomes , Jo\~ao Silva , Ant\'onio Branco , Rodrigo Santos , Henrique Lopes Cardoso , Tom\'as Os\'orio This is my paper
classification cs.CL
keywords portuguesealbertinapt-brlanguagept-ptsetsdataencoding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

To advance the neural encoding of Portuguese (PT), and a fortiori the technological preparation of this language for the digital age, we developed a Transformer-based foundation model that sets a new state of the art in this respect for two of its variants, namely European Portuguese from Portugal (PT-PT) and American Portuguese from Brazil (PT-BR). To develop this encoder, which we named Albertina PT-*, a strong model was used as a starting point, DeBERTa, and its pre-training was done over data sets of Portuguese, namely over data sets we gathered for PT-PT and PT-BR, and over the brWaC corpus for PT-BR. The performance of Albertina and competing models was assessed by evaluating them on prominent downstream language processing tasks adapted for Portuguese. Both Albertina PT-PT and PT-BR versions are distributed free of charge and under the most permissive license possible and can be run on consumer-grade hardware, thus seeking to contribute to the advancement of research and innovation in language technology for Portuguese.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. NorBERTo: A ModernBERT Model Trained for Portuguese with 331 Billion Tokens Corpus

    cs.CL 2026-04 unverdicted novelty 7.0

    NorBERTo, a ModernBERT encoder trained on the largest open Portuguese corpus of 331B tokens, reports top encoder results on several PLUE and ASSIN 2 tasks.

  2. LegalBench-BR: A Benchmark for Evaluating Large Language Models on Brazilian Legal Decision Classification

    cs.CL 2026-04 conditional novelty 7.0

    Fine-tuned BERTimbau-LoRA achieves 87.6% accuracy and 0.87 macro-F1 on LegalBench-BR, outperforming commercial LLMs by 22-28 points and eliminating their systematic bias toward civil law on Brazilian legal classification.

  3. NorBERTo: A ModernBERT Model Trained for Portuguese with 331 Billion Tokens Corpus

    cs.CL 2026-04 conditional novelty 6.0

    NorBERTo, a ModernBERT-style Portuguese encoder trained from scratch on the 331B-token Aurora-PT corpus, posts top scores on PLUE and ASSIN 2 entailment, but lower scores on semantic similarity.