Shallow Prefill, Deep Decoding: Efficient Long-Context Inference via Layer-Asymmetric KV Visibility
Open paper
Benchmark
Model
Rank
Results
question-answering-on-natural-questions
OffShelf-FT-SPEED+BoS-24
–
EM: 46.4
Rank counts only results with a code link.