STAND - Stress and Anxiety Detection and Level Estimation

STAND (STress and ANxiety Detection) is a multimodal digital phenotyping project aimed at developing AI systems capable of detecting and quantifying stress and anxiety in real time. The project integrates smartphone-based video, game-based behavioral signals, ecological momentary assessments (EMA), and wearable biosensing to build rich, individualized profiles of stress and affective dynamics.

The STAND mobile app engages participants in a set of cognitively demanding and stress-inducing games while simultaneously capturing frontal-face video and behavioral performance metrics (e.g., reaction time, accuracy, hesitation patterns). These recordings serve as training data for computer-vision and multimodal learning models that estimate moment-level stress and anxiety levels.

To expand the ecological validity and sensing depth, the newest phase of STAND also incorporates wearable devices during gameplay to collect heart rate, HRV, EDA, and movement signals. This enables the creation of multimodal datasets that fuse physiological, behavioral, and visual markers of stress—supporting the development of robust, real-world affective-detection models.

Current development focuses on enhancing the app’s gameplay experience, refining data capture workflows, and expanding engagement features, including:

• Automatic upload of all captured video, sensor data, and gameplay logs to secured AWS storage
• Gamification elements such as daily rewards, streak counters, and a weekly leaderboard to encourage sustained participation
• Integration of a second stress-inducing game to broaden the types of cognitive and emotional challenges presented
• Synchronization of wearable biosignals to enrich multimodal model training for stress and anxiety detection

The STAND platform is actively evolving toward a scalable multimodal dataset and serves as the foundation for my upcoming work on personalized affective modeling during my UCSF postdoctoral fellowship.

You can find the code for this app in this public repository on Github.