New Wave of AI-first Devices (Part 1: AI x Smartphone)
The AI gadget era as declared by The Verge officially began when Humane's Ai Pin started shipping on Apr. 11. Consumer hardware serves as the interface for end users to interact with AI capabilities and, therefore a crucial component in delivering the value of AI. There are at least three swimlanes for AI-first hardware (excluding IoT devices for future discussion:-):
Super devices: These are highly advanced and versatile devices like smartphones.
Companions of super devices: These include earbuds, watches, rings, and smart glasses that complement super devices.
Specialized gadgets and wearables: These are new or less familiar form factors that take on specialized functions.
While the paths to success for each of them are unique, the primary goal of hardware should still be to make it easy and seamless for users to interact with AI. This series of discussions will be divided into three parts to explore how AI converges with smartphones, companions, and new form factors, primarily from the perspectives of user experience and messaging.
Success factors for AI-first devices
Before diving into each hardware swimlane, a general audit of success factors for AI-first devices can serve as a high-level checklist.
Seamless integration with everyday life. The new wave of AI-powered devices aims to blend seamlessly into our daily routines, using familiar objects to introduce advanced technology without overwhelming users.
User-centric design. These devices prioritize the user experience by minimizing the learning curve and focusing on adding value through intuitive design and functionality, ensuring that the technology serves the user, not the other way around.
Balancing innovation with familiarity. While pushing the boundaries of what's possible with AI, designers are careful to anchor new devices and allow users not to burn on both ends of the candle. The benefits of innovation are accessible to everyone, not just tech enthusiasts.
Prioritizing privacy. As AI devices often collect and process a lot more personal data, failing to address privacy concerns can be a significant pitfall. Transparent privacy policies and robust data protection measures are essential to gain and maintain user trust.
Messaging that inspires and delivers. Tie the product to a larger aspiration that resonates with the audience, in a genuine way (a.k.a. claims backed up by a product that truly delivers on its promise at different stages).
User experience
Smartphones, which have been evolving for over two decades, are what I refer to as "super devices." They can perform a wide range of tasks anytime, anywhere. Through applications or websites accessed via a browser, one can always access information or solve problems.
The confluence of AI and smartphones started much earlier - the voice assistants like “Okay Google” and “Hey Siri”, the image recognition and Magic Eraser in Google Photos, autocorrect & autosuggestions of Google Search, and many other AI features migrated from PC to mobile. Smartphone is indeed the most ready platform to accommodate and deliver new AI capabilities. Some major user experience considerations include:
Built-in action-flow & work-flow: In today's digital landscape, finding an app to address a specific question is relatively easy. Completing a task, however, frequently necessitates juggling multiple apps, for example making a fact-checked social post with an image. Advanced AI should have a proper platform to embed the action flow and automatically call useful applications to accomplish tasks. Such action chains need to be built into smartphones and manifest as a rather straightforward, clean, and guided user interface.
Conversational: GenAI's strength is in understanding natural language, making a conversation-based interface the best choice for intuitive interaction and multitasking. As of now, the audio feature of some chatbots like ChatGPT remains limited, as users can’t interrupt its talking by just chiming in when needed.
Context-aware: GenAI should leverage phone data (location, calendar, contacts, etc.) to provide proactive assistance and personalize responses, with user privacy as a priority.
Multimodal: Combine conversational AI with visual elements (images, videos, text) for enhanced information processing and content generation.
Proactive yet unobtrusive: GenAI should offer suggestions and aid in tasks when it recognizes the need, but remain unobtrusive and respectful of user attention.
Trustworthy: Design should emphasize transparency about AI's role, how it uses data, and potential limitations to build user trust.
Feedback loop: The UX must support constant learning and easy feedback mechanisms for the user to help the AI improve its responses and capabilities over time.
To deliver all these experiences to users, app developers have a lot on their plates, and so do operating system providers like Google and Apple. They are the only ones who can support the cross-platform experiences demanded by the convergence of AI capabilities. I anticipate substantial enhancements in the Android OS and iOS in upcoming iterations.
Case study: AI features in Galaxy S24
Samsung unveiled the Galaxy AI earlier this year and it has been applied to its family of devices, including phones, tablets, and earbuds. I tried all the features released on Galaxy S24 and here are some of my thoughts.
Circle to Search
Circle to Search in Galaxy S24 is brought by Google, as part of its new ways to search in 2024 evolved from Google Lens. It is a simpler way to get the information with fewer steps. However, the feature is still in development and has some limitations.
Users can use voice to ask follow-up questions, but the answers are not always available and useful (as you can see from the clip below). Nor can it offer the experience of a conversation that frees users' hands and eyes, like chatting with a helpful assistant.
An ideal user experience would be to search, chat, organize/store information, or/and take actions. This is an essential feature that can be moved to other devices that have vision capability, for example, smart glasses.
Browsing Assistance
Browsing Assistance comes with summary and translation features, designed specifically for the Samsung Internet app. This means you can't expect to use it anywhere else. While it's good to experiment with it in a scenario where users might need it a lot, it doesn't help in many more situations where both features are needed. I believe that such essential features should not be limited to specific apps but supported at the device level.
Live Translate
Live Translate in Samsung’s native phone app is a very helpful feature, allowing users to get real-time translations of foreign languages. But this only solves one use case: communicating on calls. There are numerous instances where we chat with people speaking in different languages in person or through other social apps.
Other AI features on Galaxy S24, such as Transcript Assist, Note Assist, and Chat Assist, face a similar issue of limiting use cases to only one app. It will be challenging to convince users to stick to them when platform-level functionalities are available.
One area in which I wish to see innovations is optimizing the setting up of these new AI-powered features. For those on the Samsung Galaxy S24 series, users must go through the setup manually side by side with text or video instructions. I envision a voice conversational option in which I only need to “tell” the phone what I want to do, confirm my choices, and give my consent. Isn’t that the most basic and useful action chain to build in the phone?