Two months after wondering whether LLMs have hit a plateau, the answer appears to be a particular "no." Google’s Gemini 2.0 LLM and Veo 2 video model is spectacular, OpenAI previewed a succesful o3 model, and Chinese startup free deepseek unveiled a frontier mannequin that price lower than $6M to prepare from scratch. 1. Pretraining: 1.8T tokens (87% supply code, 10% code-related English (GitHub markdown and Stack Exchange), and 3% code-unrelated Chinese). Codellama is a mannequin made for gene...