Portable Somatic Wearable: AI-Assisted Reflection Through Gaze-Derived Somatic Markers

AI companions are increasingly used to support reflection, emotional processing, and self-understanding through dialogue. Yet most AI-mediated reflection systems remain primarily linguistic: they attend to what people say, while overlooking how experience is often first registered through the body. Long before experience becomes language or interpretation, the body reacts—a held breath, a moment of stillness, a glance averted or sustained. Such pre-cognitive signals orient attention, signal resistance or care, and guide social interaction. Drawing from Damasio's somatic marker hypothesis, bodily states can be understood as somatic markers: affective cues that precede conscious reasoning and shape perception, attention, and decision-making [2].

Despite the centrality of these embodied cues, interactive systems for reflection typically privilege explicit narration, inviting participants to interpret their experiences only after they have been articulated in words. Even when bodily signals are incorporated, they are frequently treated as metrics for classification, reduced to predefined emotional categories that flatten the ambiguity and temporal richness of somatic experience. We argue that embodied signals should instead be treated as meaningful interaction material: not as diagnostics, but as cues for attunement, pacing, and care.

In this paper, we present the Portable Somatic Wearable, an interactive demo that enables conference attendees to engage in a short guided AI voice reflection while gaze-derived somatic markers shape the conversation in real time. The system captures micro-behaviors such as hesitation, fixation dynamics, and blink rhythms, translating them into somatic markers that modulate conversational pacing and supportive prompts. By operationalizing ocular dynamics as expressive input, the pod expands AI-assisted reflection beyond language and explores how conversational agents can respond to embodied withdrawal, uncertainty, and temporal pacing.

AI-mediated reflection and mental health. Recent systems explore conversational agents for journaling, guided reflection, and mental health support, leveraging large language models to scaffold self-disclosure and meaning-making [1, 5, 8, 12]. These tools often foreground coherence and narrative articulation, emphasizing interpretability and verbal self-report as primary interaction channels. While valuable, such approaches can implicitly privilege cognition over pre-linguistic affect, placing the burden of emotional description on the participant. At the same time, the rapid incorporation of LLMs across HCI has raised renewed questions about what interaction modalities are centered and what forms of lived experience remain uncaptured [9].

Affective computing from gaze and ocular dynamics. Affective computing research has shown that ocular micro-behaviors such as pupil diameter, blink frequency, and fixation duration can predict valence and arousal [10] and reflect attention and cognitive state [6]. Multimodal systems combining ocular dynamics with biosignals (e.g., EEG) further improve affect inference [14]. However, across many systems, somatic data is treated primarily as a classification problem—mapped onto discrete emotional labels or scores. This collapses the ambiguity inherent in bodily expression and shifts attention from lived experience to semantic certainty.

Immersive pods and introspective interfaces. A growing body of work investigates immersive enclosures, VR-adjacent installations, and intimate interaction settings that reduce external distraction and amplify presence. Recent systems explore contemplative XR environments and inward-facing mixed reality platforms designed explicitly for self-directed reflection [11, 13]. Social VR studies further examine how guided meditation can be scaffolded through shared immersive settings, including the practices and tradeoffs involved in collective inward experience [7]. Our work builds on this lineage by combining an enclosed reflective environment with real-time gaze analysis, using pre-cognitive somatic cues not simply for sensing, but as interaction material that modulates the pacing and felt responsiveness of AI-guided conversation.

Building on research in immersive introspective interfaces, our work explores how AI-guided reflection can incorporate embodied and pre-cognitive cues rather than relying solely on linguistic narration. This demo is motivated by observations from our previous installation Gaze to the Stars (GTS) [4], a public participatory installation that invited participants to share personal stories through an intimate guided interaction experience. GTS examined how personal affect could be externalized and shared at civic scale by embedding anonymized narrative summaries into close-up recordings of participants’ eyes.

Left: Isometric line drawing of the Portable Somatic Wearable, labeled with five numbered parts — Headphones, Headphone Mount, Depth Adjustment Arm, Lateral Adjustment Arm, Camera Module (XIAO ESP32-S3 Sense) — and three lettered subcomponents: XIAO ESP32, Expansion Board, OV2640 camera. Overall dimensions are 31cm wide and 14cm tall. Upper right: Zoomed eye-tracking capture showing pupil detection overlays with gaze vectors and live metrics. Lower right: Web dashboard showing a Hesitation Score of 1.98 labeled Normal, gaze position coordinates, fixation duration, pupil and blink metrics, and a Hesitation Over Time line chart.

At the center of GTS was an installation-scale sensory pod designed to reduce distraction and support vulnerability through enclosure and a controlled audiovisual atmosphere (see Figure ). Inside the pod, participants engaged in a short guided voice conversation with an AI facilitator while a close-up eye-facing camera recorded high-resolution eye video alongside the conversation transcript. This installation pod enabled consistent data capture across 200 participants and supported the broader GTS pipeline in which stories were anonymized, summarized, and later encoded into the eye video for public projection and interactive visualization [3].

However, GTS revealed two limitations that motivated this work. First, because AI-mediated reflection is rapidly becoming mainstream, access, reach, and repeatability matter: the original pod required installation-scale infrastructure and specialized fabrication, limiting who could experience the system and where it could be deployed. Second, while the original pipeline captured and visualized eye-based expression as part of the artistic output, the reflective interaction itself remained primarily language-driven: the AI responded to participants’ spoken narratives ignoring their real-time somatic micro-behaviors.

The updated wearable interaction (see Figure  3) flow integrates real-time gaze sensing to support somatic-responsive storytelling. As users engage with the wearable, the system continuously captures subtle eye-based behaviors, like gaze stability, pupil changes, and blink frequency, to assess moments of hesitation or internal tension.

At the start of each session, the system calibrates the user's baseline for gaze and pupil metrics. From there, it interprets hesitation levels on a low–medium–high scale using a composite of gaze fixation, pupil dilation, and blink activity. Rather than interrupting or advancing the conversation linearly, the system slows down or mirrors emotional moments when hesitation increases, keeping the storytelling experience aligned with the participant's embodied cues.

This gaze-driven somatic data does not directly instruct the language model. Instead, it subtly modulates the AI facilitator's pacing and tone. When low hesitation is detected, the AI prompts narrative continuation. In medium hesitation states, it holds space with open-ended or non-directive prompts. High hesitation triggers reflective affirmations that pause the narrative flow entirely. This shift allows the system to treat embodied hesitation not as silence to be filled, but as meaningful expression to be respected.

A three-part flow diagram. Step 1, Real Time Gaze Sensing: three dark UI panels display Fixation Duration (0.07s), Pupil and Blink metrics (Pupil 58.1px, EAR 0.358), and a Gaze Trajectory plot, all feeding from a camera wearable worn by a person visible in a central photo. Step 2, Hesitation Tracker: a line chart shows Hesitation Score over time, with two score states — 1.98 Normal and 5.03 High Hesitation — highlighted. Step 3, Prompt Mode Selection: three color-coded mode cards: Mirroring (pink, reflective statements), Holding (yellow, open-ended non-directive prompts), and Reflective (green, structured forward-moving prompts). Dashed arrows connect hesitation level output to prompt mode selection.

The updated wearable interaction(see Figure  3) flow integrates real-time gaze sensing to support somatic-responsive storytelling. As users engage with the wearable, the system continuously captures subtle eye-based behaviors, like gaze stability, pupil changes, and blink frequency, to assess moments of hesitation or internal tension.

At the start of each session, the system calibrates the user's baseline for gaze and pupil metrics. From there, it interprets hesitation levels on a low–medium–high scale using a composite of gaze fixation, pupil dilation, and blink activity. Rather than interrupting or advancing the conversation linearly, the system slows down or mirrors emotional moments when hesitation increases, keeping the storytelling experience aligned with the participant's embodied cues.

This gaze-driven somatic data does not directly instruct the language model. Instead, it subtly modulates the AI facilitator's pacing and tone. When low hesitation is detected, the AI prompts narrative continuation. In medium hesitation states, it holds space with open-ended or non-directive prompts. High hesitation triggers reflective affirmations that pause the narrative flow entirely. This shift allows the system to treat embodied hesitation not as silence to be filled, but as meaningful expression to be respected.

We conducted a within-subjects user study (N=7) to evaluate how gaze-based somatic sensing shaped the storytelling experience. Each participant engaged in two sessions: one with a standard text-based AI facilitator, and another using the somatic-aware pipeline. In both, participants responded to prompts designed around core affective categories, such as anger, fear, sadness, or joy.

Results showed that somatic behaviors reliably reflected emotional tone. For example, higher hesitation scores—linked to increased pupil change, longer fixations, and altered blink rhythms—were consistently observed during negative, high-arousal prompts (e.g., anxiety, anger), compared to more relaxed or joyful states(see Figure  4). These scores varied not only across conditions but also moment-to-moment within a narrative, validating the system's capacity to detect nuanced emotional shifts.

Time series analysis and cross-affect comparison of hesitation detection scores. (Top) Representative participant (ID 1) showing temporal dynamics of hesitation across two affective conditions: happiness (Condition A, mean hesitation = 1.92) and anxiety (Condition B, mean hesitation = 3.14). Threshold line (θ_H = 3.0) indicates significant hesitation moments. (Bottom) Group-level comparison showing mean hesitation score, pupil diameter, and fixation ratio. Affects ordered by ascending hesitation score demonstrate systematic somatic differentiation: relaxation (1.62) < happiness (1.92) < fear (2.88) < anxiety (3.14) < anger (3.24) < desire (3.92).

A grouped bar chart comparing mean Likert ratings (1–7 scale) across six measures between Condition A Original (gray) and Condition B Somatic (blue), with error bars showing standard deviation. A dashed line marks the neutral midpoint at 4.0. Condition B scores higher than Condition A on Mental State (4.7 vs 4.3), Focus (5.1 vs 4.9), Sharing Comfort (5.4 vs 4.6), and matches on Pause Comfort (5.4 vs 5.4). Condition A scores higher on Emotional Intelligence (5.1 vs 4.3) and Prompt Appropriateness 1 (5.6 vs 4.7).

In post-study surveys, participants rated the somatic-aware version higher in perceived comfort, emotional pacing, and overall empathy. (see Figure 5) Many described the interaction as “more in tune” or “better at listening” than the baseline facilitator. Interestingly, even those who preferred more directive guidance acknowledged the value of a system that responded to moments of hesitation with space rather than interruption.

These findings support our design hypothesis: somatic markers can serve as a meaningful cue in human–AI interaction. Rather than viewing silence as breakdown, the system learns to recognize it as embodied resistance, reflection, or emotional weight. Through this, the wearable offers a gentler storytelling companion, one that listens not only to words, but to the body behind them.

Portable Somatic Wearable presents an interactive system that uses gaze-based somatic markers to shape real-time, AI-facilitated storytelling. Within a compact, portable pod, participants engage in intimate narrative sessions where micro-expressions—such as hesitation and gaze shifts—dynamically modulate the pacing and tone of the AI's responses. The system's portability allows deployment across diverse cultural and geographic contexts, supporting the collection of embodied interaction data at a global scale. By foregrounding somatic interaction as a meaningful input in narrative experiences, this work contributes to interaction design by demonstrating how bodily signals can act as co-authors in emotionally adaptive, participatory storytelling systems.