A Survey on Large Language Models with some Insights on their Capabilities and Limitations

https://arxiv.org/abs/2501.04040 The paper explores the foundations, capabilities, and limitations of Large Language Models (LLMs). It examines various training methodologies (unsupervised, supervised, semi-supervised), data preprocessing techniques, and model adaptation strategies like instruction and alignment tuning. The analysis includes a review of prominent LLMs (BERT, T5, GPT series, LLaMA) and their architectures, highlighting emergent abilities such as in-context learning and chain-of-thought reasoning. Furthermore, the paper investigates LLM applications in diverse fields, such as healthcare and finance, and discusses challenges related to scaling, efficiency, and ethical considerations. Finally, it explores advanced techniques for improving LLM performance, including parameter-efficient fine-tuning and memory-efficient adaptation methods.