Collov Labs Unveils AI Agent That Sees and Acts Through Smart Glasses
Collov Labs introduced NewEyes AI, a visual personal agent that combines smartphone cameras and smart glasses to understand and execute tasks based on what the user sees. The system uses on-device perception and cloud reasoning to convert visual context into actionable steps. It is designed to work hands-free with compatible smart glasses.
NewEyes AI pairs compatible smart glasses with a smartphone to create a hands-free visual agent. The system splits processing across three layers: on-device perception handles visual frames on the phone, a local runtime manages agent logic, and cloud reasoning tackles complex tasks. A context compaction mechanism filters redundant visual data to keep the agent's working memory efficient.
The company's research pedigree includes affiliations with Stanford, UC Berkeley, and Yale, with published work at IJCAI, AAAI, ICLR, and ICML. Collov Labs has spent over two years deploying specialized visual AI models in production settings. Demonstrated workflows include interpreting parking signs for calendar reminders, converting photographed wireframes into website drafts, and cataloging pantry items for shopping lists.
This technology could shift how people interact with everyday environments, turning routine visual observations into automated actions. Users may gain convenience in tasks like parking, shopping, and planning, but reliance on AI interpretation could introduce errors in judgment-sensitive situations. Privacy concerns may arise as cameras continuously capture surroundings, potentially affecting bystanders. The hybrid edge-cloud design could influence how wearable AI develops, though broad adoption likely depends on smart glasses affordability and user trust in the system's accuracy.