ByteDance's Doubao Phone Assistant has officially launched its consumer version, bringing a suite of AI-powered features designed for daily use, with a focus on stability and interactive experience, according to IThome (IThome). The assistant supports multiple wake-up methods, including voice and a dedicated AI key with fingerprint authentication, allowing users to invoke tasks without repeated unlocks.
Screen Q&A and Local Data Search
One of the headline additions is screen Q&A, which lets the assistant understand and act on the current screen content without requiring screenshots or app switching. For example, pointing the camera at a room and asking for a cabinet under 120 cm and under 1,000 yuan (about $141) would prompt the assistant to search e-commerce platforms for matching products. The assistant also includes a local data search tool, enabling users to query their photo albums, SMS, and notes—such as asking when their Hong Kong-Macau travel permit expires and getting an answer from relevant on-device information.
The recording experience has been upgraded with integration with Feishu Miaojian, allowing users to ask follow-up questions like 'product suggestions from yesterday's user interview.' Users can save current screen content via voice input, a three-finger swipe, or pressing the AI key and volume key simultaneously, creating a locally searchable memory.
Beta 'Operate Phone' Feature and SAEP Protocol
The consumer version opens the 'operate phone' feature in beta, aiming to build an open and standardized AI app ecosystem. To this end, Doubao Phone Assistant introduces the Screen Automation Operation Declaration Protocol (SAEP), with a 30-day public comment period. Under this protocol, third-party apps can declare their operational boundaries, explicitly allowing or denying the AI assistant's screen automation within their interfaces. Doubao will respect developers' choices and will not automate operations in apps that explicitly refuse.
Data collection and processing follow principles of 'reasonable necessity' and 'user control,' with an Agent protection system that manages operational scope in layers and levels. The first consumer version will ship on the Nubia NaviX Ultra, which is now open for pre-orders and goes on sale on September 16.
