Summary
Researchers from Meta, University of Southern California, Carnegie Mellon University, and University of California San Diego recently open-sourced MEGALODON, a large language model (LLM) with an unlimited context length. MEGalODON has linear computational complexity and outperforms a similarly-sized Llama 2 model on a range of benchmarks.