Large Language Models (LLMs) like GPT-3 showcase seemingly emergent abilities as they scale, prompting intrigue about their inherent capabilities. Researchers have delved into these abilities to distinguish what's intrinsic to the models from what's influenced by external evaluation techniques.
Their findings reveal that much of the LLMs' prowess is attributed to "in-context learning," where models adapt based on provided examples. When this factor is controlled, LLMs display limited unpredictably emergent skills, suggesting that their impressive behaviors might be more about leveraging context than innate intelligence. This study underscores the importance of rigorous testing in understanding AI capabilities, emphasizing that LLMs may not be as enigmatic as they appear.