論文 - 南澤　孝太

FEELTECH Wear : Enhancing Mixed Reality Experience with Wrist to Finger Haptic Attribution

Umehara R., Taguchi H., Horie A., Kamiyama Y., Sakamoto S., Ishikawa H., Minamizawa K.

Proceedings - SIGGRAPH 2024 Emerging Technologies 2024年07月

　概要を見る

FEEL TECH Wear is a system that facilitates haptic interactions while keeping most of the palm free, by presenting directional force through rotational skin-stretch distribution feedback to the wrist and providing texture sensation through vibration feedback to the fingertips. With advancements in hand tracking and passthrough technologies, hand interactions in Mixed Reality (MR) environments have become more accessible, necessitating palm-free haptic feedback methods that do not hinder interactions with real objects or impair vision-based hand tracking. The hardware of FEEL TECH Wear primarily consists of two components: a hand-mounted device for each hand and a control unit located at the back of the head. The hand-mounted device is equipped with four channels of rotational skin-stretch tactors at the wrist and vibration tactors at the thumb and index finger. Using FEEL TECH Wear, three applications have been realized: haptic feedback for virtual objects, haptic augmentation for real objects, and haptic guidance towards objects.

• Access to Document (DOI)

HaptoRoom: Using Vibrotactile Floor Interfaces To Enable Reconfigurable Haptic Interaction Onto Any Furniture Surfaces

Tsujita K., Yoshida T., Kobayashi K., Horie A., Hanamitsu N., Minamizawa K.

Proceedings - SIGGRAPH 2024 Emerging Technologies 2024年07月

　概要を見る

Vibrotactile interaction is an essential source of HCI and should be on a large scale, such as room scale, whereas in the past many devices have been small. However, there is a practical difficulties for implementing vibrotactile actuators and sensors to every furnitures in the room. To address this problem, our approach, HaptoRoom, archives reconfigurable ubiquitous haptic interaction to existing furniture without additional equipment by installing actuators only in the floor. A user can design detailed spatial distribution of vibrotactile pattern, leveraging with multi-point individual control of actuators array inside floor interfaces.

• Access to Document (DOI)

The Malleable-Self Experience: Transforming Body Image by Integrating Visual and Whole-Body Haptic Stimuli

Person T., Hanamitsu N., Hynds D., Wakisaka S., Isobe K., Mochizuki L., Mizuguchi T., Minamizawa K.

Proceedings - SIGGRAPH 2024 Emerging Technologies 2024年07月

　概要を見る

The Malleable-Self Experience comprises the integration of the visual element of virtual reality (VR) with the whole-body haptic sensations of the Synesthesia X1 haptic chair. The goal is to induce a provocative experience that expands one's understanding of the self by creating a malleable perception of the body image. We explore the effects of visual and whole-body haptic integration on augmenting body image during dynamic transformations of visual representations of the body in VR. We design the plausibility of these perceptual augmentations using a specific sequence of multisensory events: (1) establishing body ownership of a virtual body anchored in the same self-located space as the participant, (2) separating the virtual body to hover above the participant's physical body, enhanced by accompanying haptic stimuli to increase proprioceptive uncertainty, and (3) transforming the virtual body with integrated visuo-haptic stimuli to sustain perceptual congruency.

• Access to Document (DOI)

Cymatics Cup: Shape-Changing Drinks by Leveraging Cymatics

Chen W., Yang Y., Liu K.H., Pai Y.S., Yamaoka J., Minamizawa K.

Conference on Human Factors in Computing Systems - Proceedings 2024年05月

　概要を見る

To enhance the dining experience, prior studies in Human-Computer Interaction (HCI) and gastrophysics have demonstrated that modifying the static shape of solid foods can amplify taste perception. However, the exploration of dynamic shape-changing mechanisms in liquid foods remains largely untapped. In the present study, we employ cymatics, a scientific discipline focused on utilizing sound frequencies to generate patterns in liquids and particles-to augment the drinking experience. Utilizing speakers, we dynamically reshaped liquids exhibiting five distinct taste profiles and evaluated resultant changes in taste perception and drinking experience. Our research objectives extend beyond merely augmenting taste from visual to tactile sensations; we also prioritize the experiential aspects of drinking. Through a series of experiments and workshops, we revealed a significant impact on taste perception and overall drinking experience when mediated by cymatics efects. Building upon these findings, we designed and developed tableware to integrate cymatics principles into gastronomic experiences.

• Access to Document (DOI)

Maintaining Continuing Bonds in Bereavement: A Participatory Design Process of Be.side

Kim J., Uriu D., Barbareschi G., Kamiyama Y., Minamizawa K.

Conference on Human Factors in Computing Systems - Proceedings 2024年05月

　概要を見る

During the grieving process, physical objects often serve as catalysts for remembering and honouring the relationship with departed loved ones. Leveraging a participatory design approach, we created Be.side, a fully customisable multi-modal artefact that incorporates scent, sound, and heartbeat stimulation and acts as a touch-point between the deceased and the bereaved. We conducted a four-week study with three participants to understand how the artefact, continuously attuned to each participant, helped to continue bonds with the deceased. Our results show that Be.side's bespoke elements helped participants to evoke memories of the deceased. Participants created personalised rituals for remembrance. They sustained bonds by not only interacting with Be.side but also participating in the research. Finally, highlighting that remembrance can both provide comfort and deepen sadness, we discuss future design considerations.

• Access to Document (DOI)

People with Disabilities Redefining Identity through Robotic and Virtual Avatars: A Case Study in Avatar Robot Cafe

Hatada Y., Barbareschi G., Takeuchi K., Kato H., Yoshifuji K., Minamizawa K., Narumi T.

Conference on Human Factors in Computing Systems - Proceedings 2024年05月

　概要を見る

Robotic avatars and telepresence technology enable people with disabilities to engage in physical work. Despite the recent popularity of the metaverse, few studies have explored the use of virtual avatars and environments by people with disabilities. In this study, seven disabled participants working in a cafe where remote customer service is provided via robotic avatars, were engaged in the development and use of personalized virtual avatars displayed on a large screen in-situ in combination with existing physical robots, creating a hybrid cyber-physical space. We conducted longitudinal semi-structured interviews to investigate the psychological changes experienced by the participants. The results revealed that mass-produced robotic avatars allowed participants to not disclose their disability if they did not want to, but also backgrounded their identities; by contrast, customized virtual avatars shaped without physical constraints, highlighted their personalities. The combined use of robotic and virtual avatars complemented each other and can support pilots in redefining their identity.

• Access to Document (DOI)

SealMates: Supporting Communication in Video Conferencing using a Collective Behavior-Driven Avatar

Armstrong M., Yang C.L., Skiers K., Lim M., Gunasekaran T.S., Wang Z., Narumi T., Minamizawa K., Pai Y.S.

Proceedings of the ACM on Human-Computer Interaction （Proceedings of the ACM on Human-Computer Interaction）  8 （ CSCW1 ） 1 - 23 2024年04月

　概要を見る

The limited nonverbal cues and spatially distributed nature of remote communication make it challenging for unacquainted members to be expressive during social interactions over video conferencing. Though it enables seeing others' facial expressions, the visual feedback can instead lead to unexpected self-focus, resulting in users missing cues for others to engage in the conversation equally. To support expressive communication and equal participation among unacquainted counterparts, we propose SealMates, a behavior-driven avatar in which the avatar infers the engagement level of the group based on collective gaze and speech patterns and then moves across interlocutors' windows in the video conferencing. By conducting a controlled experiment with 15 groups of triads, we found the avatar's movement encouraged people to experience more self-disclosure and made them perceive everyone was equally engaged in the conversation than when there was no behavior-driven avatar. We discuss how a behavior-driven avatar influences distributed members' perceptions and the implications of avatar-mediated communication for future platforms.

• Access to Document (DOI)

Innermost Echoes: Integrating Real-Time Physiology into Live Music Performances

Hynds D., Chernyshov G., Zheng D., Uyama A., Li J., Matsumoto K., Pogorzhelskiy M., Kunze K., Ward J.A., Minamizawa K.

ACM International Conference Proceeding Series 2024年02月

　概要を見る

In this paper, we propose a method for utilizing musical artifacts and physiological data as a means for creating a new form of live music experience that is rooted in the physiology of the performers and audience members. By utilizing physiological data (namely Electrodermal Activity (EDA) and Heart Rate Variability (HRV)) and applying this data to musical artifacts including a robotic koto (a traditional 13-string Japanese instrument fitted with solenoids and linear actuators), a Eurorack synthesizer, and Max/MSP software, we aim to develop a new form of semi-improvisational and significantly indeterminate performance practice. It has since evolved into a multi-modal methodology which honors improvisational performance practices and utilizes physiological data which offers both performers and audiences an ever-changing and intimate experience. In our first exploratory phase, we focused on the development of a means for controlling a bespoke robotic koto in conjunction with a Eurorack synthesizer system and Max/MSP software for controlling the incoming data. We integrated a reliance on physiological data to infuse a more directly human elements into this artifact system. This allows a significant portion of the decision-making to be directly controlled by the incoming physiological data in real-time, thereby affording a sense of performativity within this non-living system. Our aim is to continue the development of this method to strike a novel balance between intentionality and impromptu performative results.

• Access to Document (DOI)

Impact of Vibrotactile Triggers on Mental Well-Being through ASMR Experience in VR

D Peng, T Person, X Shen, YS Pai, G Barbareschi, S Li, K Minamizawa

arXiv preprint arXiv:2404.12567 2024年

Modulation of Fronto-striatal Functional Connectivity and Social Interaction Through Conveying Digital Somatic Information

T Matsuhashi, M Ito, T Kuhara, K Hosokawa, K Inukai, K Minamizawa, ...

2024年

SealMates: Improving Communication in Video Conferencing using a Collective Behavior-Driven Avatar

M Armstrong, CL Yang, K Skiers, M Lim, TS Gunasekaran, Z Wang, ...

Proceedings of the ACM on Human-Computer Interaction 8 (CSCW1), 1-23 2024年

FEEL TECHWear : Enhancing Mixed Reality Experience with Wrist to Finger Haptic

Umehara, R; Taguchi, H; Horie, A; Kamiyama, Y; Sakamoto, S; Ishikawa, H; Minamizawa, K

PROCEEDINGS OF THE SIGGRAPH 2024 EMERGING TECHNOLOGIES 2024年

• Access to Document (DOI)

DexteriSync: A Hand Thermal I/O Exoskeleton for Morphing Finger Dexterity Experience

X Shen, Y Kamiyama, K Minamizawa, J Nishida

Proceedings of the 37th Annual ACM Symposium on User Interface Software and … 2024年

Design and Validation of Pseudo-Force Haptic Device for Actual Walking

Maeda T., Yoshimura T., Yamamoto J., Sakai H., Minamizawa K.

IEEE Haptics Symposium, HAPTICS    104 - 110 2024年

ISSN  23247347

　概要を見る

Pedestrian navigation systems on smartphones have become integral to modern society. However, a crucial problem in pedestrian navigation is that visual and/or auditory cues can distract situational awareness while walking, which may lead to serious consequences. To address this problem, various haptic navigation devices have been developed. Among these, a pseudo-force sensation elicited by asymmetric vibrations on the skin is considered a promising method owing to its intuitive directional cues. Nevertheless, pseudo-force sensation for pedestrian navigation has yet to be well validated in actual walking situations. Here, we propose an untethered pseudo-force haptic device for pedestrian navigation. Our device controls the direction of pseudo-force sensation by delivering asymmetric vibrations to the fingertips using a pair of vibrotactile actuators. Perceptual and navigation experiments were conducted to examine the effectiveness of the proposed pseudo-force navigation device. Our results suggest that pseudo-force navigation is effective during actual walking.

• Access to Document (DOI)

Brain Body Jockey project: Transcending Bodily Limitations in Live Performance via Human Augmentation

G Barbareschi, S Zhou, A Ryoichi, M Kawaguchi, M Armstrong, M Ogino, ...

The 26th International ACM SIGACCESS Conference on Computers and … 2024年

Augmenting perceived stickiness of physical objects through tactile feedback after finger lift-off

Kurogi T., Inoue Y., Fujiwara T., Minamizawa K.

Frontiers in Robotics and AI 11 2024年

ISSN  2296-9144

　概要を見る

Haptic Augmented Reality (HAR) is a method that actively modulates the perceived haptics of physical objects by presenting additional haptic feedback using a haptic display. However, most of the proposed HAR research focuses on modifying the hardness, softness, roughness, smoothness, friction, and surface shape of physical objects. In this paper, we propose an approach to augment the perceived stickiness of a physical object by presenting additional tactile feedback at a particular time after the finger lifts off from the physical object using a thin and soft tactile display suitable for HAR. To demonstrate this concept, we constructed a thin and soft tactile display using a Dielectric Elastomer Actuator suitable for HAR. We then conducted two experiments to validate the effectiveness of the proposed approach. In Experiment 1, we showed that the developed tactile display can augment the perceived stickiness of physical objects by presenting additional tactile feedback at appropriate times. In Experiment 2, we investigated the stickiness experience obtained by our proposed approach and showed that the realism of the stickiness experience and the harmony between the physical object and the additional tactile feedback are affected by the frequency and presentation timing of the tactile feedback. Our proposed approach is expected to contribute to the development of new applications not only in HAR, but also in Virtual Reality, Mixed Reality, and other domains using haptic displays.

• Access to Document (DOI)

" I Wanted to Create my Ideal Self": Exploring Avatar Perception of LGBTQ+ Users for Therapy in Virtual Reality

A Kundu, G Barbareschi, M Kawaguchi, Y Yano, M Ohashi, K Kitaoka, ...

arXiv preprint arXiv:2409.00383 2024年

テレビ局のメタバース事業における可搬式床型触覚提示システムの活用

辻田喜琉， 山村風子， 香川凌也， 横井勝， 吉田貴寿， 堀江新， 南澤孝太

エンタテインメントコンピューティングシンポジウム 2024 論文集 2024, 286-289 2024年

ロボットアバターを介した身体融合

田中由浩， 南澤孝太， 湯川光

バイオメカニズム学会誌= Journal of the Society of Biomechanisms Japan/バイオメカニ … 2024年

SeaHare: An omidirectional electric wheelchair integrating independent, remote and shared control modalities

G Barbareschi, A Ryoichi, M Kawaguchi, M Takeda, K Minamizawa

The 26th International ACM SIGACCESS Conference on Computers and … 2024年