Learning to use local AI is exciting, overwhelming, and frustrating

Key Points:

• The author is beginning to use AI regularly by running powerful local models on high-end computers, motivated by privacy concerns and a desire to avoid cloud services from major providers.
• Using an M5 Ultra Mac Studio with 256GB of unified memory, they installed the open-source Hermes Agent, running large models like Qwen 3.8 Flash Next (125-billion parameters), to explore practical AI tasks.
• Initial uses include creating a daily briefing from email and calendar data, reorganizing a large Steam game library via the Steam API, and performing sensitive data analysis locally to maintain privacy.
• The author is also working on automating laptop benchmark testing with Hermes, though this remains a work in progress, emphasizing that local AI tools are helpful but not yet flawless or magical assistants.
• Overall, the author views local AI as a practical tool rather than a digital companion and plans to continue experimenting cautiously to discover its real-world utility.