AI Generated Image
OpenAI and Broadcom announce chip designed for LLM inference at scale
Ars Technica • • business
Key Points:
- OpenAI and Broadcom have jointly developed a new chip called Jalapeño, designed specifically for large language model (LLM) inference in data centers, marking the first generation in a long-term chip refinement project.
- The Jalapeño ASIC was created from scratch based on detailed insights from OpenAI researchers and aligns with OpenAI’s roadmap for future models, with the design and production completed in nine months.
- Early testing suggests Jalapeño delivers significantly better performance per watt compared to current state-of-the-art inference systems, though detailed performance data will be released in the coming months.
- OpenAI aims to reduce reliance on external companies like Nvidia by owning the full technology stack behind its models, potentially improving performance and efficiency through vertical integration.
- Both companies plan to deploy Jalapeño chips in data centers by the end of 2024, addressing the global compute crunch and increasing capacity for AI workloads.
