| []() | | 60.12 | 54.22 | 57.21 | OpenAI/o3-mini | |
| []() | | 52.21 | 45.58 | 60.29 | Riple/Saanvi-v0.1 | |
| Beyond Reptile: Meta-Learned Dot-Product Maximization between Gradients for Improved Single-Task Regularization | | 40.6 | 21.0 | 37.0 | Pegasus+DotProd | |
| Better Fine-Tuning by Reducing Representational Collapse | ✓ Link | 40.45 | 20.69 | 36.56 | BART-RXF | 2020-08-06 |
| Muppet: Massive Multi-task Representations with Pre-Finetuning | ✓ Link | 40.4 | 20.54 | 36.21 | MUPPET BART Large | 2021-01-26 |
| OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework | ✓ Link | 39.81 | 20.66 | 37.11 | OFA | 2022-02-07 |
| Rethinking Perturbations in Encoder-Decoders for Fast Training | ✓ Link | 39.81 | 20.40 | 36.93 | Transformer+Rep(Uni) | 2021-04-05 |
| Rethinking Perturbations in Encoder-Decoders for Fast Training | ✓ Link | 39.66 | 20.45 | 36.59 | Transformer+Wdrop | 2021-04-05 |
| ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training | ✓ Link | 39.51 | 20.42 | 36.69 | ProphetNet | 2020-01-13 |
| ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation | ✓ Link | 39.46 | 20.34 | 36.74 | ERNIE-GENLARGE (large-scale text corpora) | 2020-01-26 |
| PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation | ✓ Link | 39.45 | 20.37 | 36.75 | PALM | 2020-04-14 |
| A New Approach to Overgenerating and Scoring Abstractive Summaries | ✓ Link | 39.27 | 20.40 | 37.75 | Best Summary Length | 2021-04-05 |
| ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation | ✓ Link | 39.25 | 20.25 | 36.53 | ERNIE-GENLARGE | 2020-01-26 |
| Controlling the Amount of Verbatim Copying in Abstractive Summarization | ✓ Link | 39.19 | 20.38 | 36.69 | ControlCopying + BPNorm | 2019-11-23 |
| PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization | ✓ Link | 39.12 | 19.86 | 36.24 | PEGASUS | 2019-12-18 |
| BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization | ✓ Link | 39.11 | 19.78 | 36.87 | BiSET | 2019-06-12 |
| Controlling the Amount of Verbatim Copying in Abstractive Summarization | ✓ Link | 39.08 | 20.47 | 36.69 | ControlCopying + SBWR | 2019-11-23 |
| Unified Language Model Pre-training for Natural Language Understanding and Generation | ✓ Link | 38.90 | 20.05 | 36.00 | UniLM | 2019-05-08 |
| ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation | ✓ Link | 38.83 | 20.04 | 36.20 | ERNIE-GENBASE | 2020-01-26 |
| MASS: Masked Sequence to Sequence Pre-training for Language Generation | ✓ Link | 38.73 | 19.71 | 35.96 | MASS | 2019-05-07 |
| Mask Attention Networks: Rethinking and Strengthen Transformer | ✓ Link | 38.28 | 19.46 | 35.46 | Mask Attention Network | 2021-03-25 |
| Concept Pointer Network for Abstractive Summarization | ✓ Link | 38.02 | 16.97 | 35.43 | Concept pointer+RL | 2019-10-18 |
| Attention Is All You Need | ✓ Link | 37.57 | 18.90 | 34.69 | Transformer | 2017-06-12 |
| Faithful to the Original: Fact Aware Neural Abstractive Summarization | | 37.27 | 17.65 | 34.24 | FTSum_g | 2017-11-13 |
| Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization | | 37.04 | 19.03 | 34.46 | Re^3 Sum | 2018-07-01 |
| Entity Commonsense Representation for Neural Abstractive Summarization | ✓ Link | 37.04 | 16.66 | 34.93 | Seq2seq + E2T_cnn | 2018-06-14 |
| Concept Pointer Network for Abstractive Summarization | ✓ Link | 37.01 | 17.1 | 34.87 | Concept pointer+DS | 2019-10-18 |
| A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization | | 36.92 | 18.29 | 34.58 | Reinforced-Topic-ConvS2S | 2018-05-09 |
| Joint Parsing and Generation for Abstractive Summarization | ✓ Link | 36.61 | 18.85 | 34.33 | JointParsing | 2019-11-23 |
| Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond | ✓ Link | 36.4 | 17.7 | 33.71 | words-lvt5k-1sent | 2016-02-19 |
| Global Encoding for Abstractive Summarization | ✓ Link | 36.3 | 18.0 | 33.8 | CGU | 2018-05-10 |
| Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization | | 36.30 | 17.31 | 33.88 | EndDec+WFE | 2016-12-31 |
| Deep Recurrent Generative Decoder for Abstractive Text Summarization | ✓ Link | 36.27 | 17.57 | 33.62 | DRGD | 2017-08-02 |
| Selective Encoding for Abstractive Sentence Summarization | ✓ Link | 36.15 | 17.54 | 33.63 | SEASS | 2017-04-24 |
| Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation | | 35.98 | 17.76 | 33.63 | Pointer + Coverage + EntailmentGen + QuestionGen | 2018-05-28 |
| Structure-Infused Copy Mechanisms for Abstractive Summarization | ✓ Link | 35.47 | 17.66 | 33.52 | Struct+2Way+Word | 2018-06-14 |
| Ensure the Correctness of the Summary: Incorporate Entailment Knowledge into Abstractive Sentence Summarization | | 35.33 | 17.27 | 33.19 | Seq2seq + selective + MTL + ERAM | 2018-08-01 |
| Abstractive Sentence Summarization with Attentive Recurrent Neural Networks | | 33.78 | 15.97 | 31.15 | RAS-Elman | 2016-06-01 |
| A Neural Attention Model for Abstractive Sentence Summarization | ✓ Link | 31 | | | Abs+ | 2015-09-02 |
| A Neural Attention Model for Abstractive Sentence Summarization | ✓ Link | 30.88 | | | Abs | 2015-09-02 |
| Simple Unsupervised Summarization by Contextual Matching | ✓ Link | 26.48 | 10.05 | 24.41 | Contextual Match | 2019-07-31 |