Papers - Isogawa, Mariko

Listening Human Behavior: 3D Human Pose Estimation With Acoustic Signals

Y Shibata, Y Kawashima, M Isogawa, G Irie, A Kimura, Y Aoki

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  2023

Research paper (international conference proceedings), Accepted

Importance ranking estimation of objects in a city scene

Y Nitta, M Isogawa, R Yonetani, M Sugimoto

IEICE Technical Report; IEICE Tech. Rep. 122 (367), 43-48  2023

Importance Rank-Learning of Objects in Urban Scenes for Assisting Visually Impaired People

Y Nitta, M Isogawa, R Yonetani, M Sugimoto

IEEE Access  2023

Research paper (scientific journal), Accepted

疎な計測情報に基づくシーン状態推定の展望 [塾内限定]

M Isogawa

慶應義塾大学 理工学部  2023

Research paper (other academic)

Video and audio presentation device, video and audio presentation method, and program

M Isogawa, D Mikami, K Takahashi, Y Kusachi

US Patent 11,715,240  2023

Silhouette-Based 3D Human Pose Estimation Using a Single Wrist-Mounted 360° Camera.

Ryosuke Hori, Ryo Hachiuma, Mariko Isogawa, Dan Mikami, Hideo Saito

IEEE Access 10   54957 - 54968 2022

Research paper (scientific journal),  ISSN  2169-3536

• Access to Document (DOI)

Bilateral video magnification filter

S Takeda, K Niwa, M Isogawa, S Shimizu, K Okami, Y Aono

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern …  2022

Learning data generation device, learning data generation method, and program

D Mikami, M Isogawa, H Yabushita, Y Kusachi

US Patent App. 17/620,608  2022

Information completion apparatus, method and program

M Isogawa, D Mikami, K Takahashi, Y Kusachi

US Patent App. 17/613,464  2022

Coordinate system conversion parameter estimating apparatus, method and program

K Takahashi, D Mikami, M Isogawa, Y Kusachi

US Patent App. 17/435,759  2022

Rotation state estimation device, method and program

D Mikami, M Isogawa, K Takahashi, Y Kusachi

US Patent App. 17/438,867  2022

Virtual environment construction apparatus, video presentation apparatus, model learning apparatus, optimal depth decision apparatus, methods for the same, and program

K Takahashi, D Mikami, M Isogawa, A Kojima, H Kimata, A Matsumoto

US Patent App. 17/845,010  2022

Virtual environment construction apparatus, video presentation apparatus, model learning apparatus, optimal depth decision apparatus, methods for the same, and program

K Takahashi, D Mikami, M Isogawa, A Kojima, H Kimata, A Matsumoto

US Patent 11,395,947  2022

Video synchronization device, video synchronization method, and program

SUN Siqi, K Takahashi, D Mikami, M Isogawa, Y Kusachi

US Patent App. 17/442,077  2022

Training data generation apparatus, method and program

M Isogawa, D Mikami, K Takahashi, Y Kusachi

US Patent App. 17/429,547  2022

Motion classification model learning apparatus, motion classification apparatus, motion classification model learning method and program

M Isogawa, D Mikami, K Takahashi, H Kimata, A Matsumoto

US Patent App. 17/059,121  2021

Silhouette-based synthetic data generation for 3D human pose estimation with a single wrist-mounted 360° camera

R Hori, R Hachiuma, H Saito, M Isogawa, D Mikami

2021 IEEE International Conference on Image Processing (ICIP), 1304-1308    1304 - 1308 2021

• Access to Document (DOI)

Multi-view video synchronization using motion rhythms of human joints

S Sun, K Takahashi, D Mikami, M Isogawa, Y Kusachi

ITE Transactions on Media Technology and Applications 8 (2), 100-110  2020

Research paper (scientific journal)

Prodebnet: projector deblurring using a convolutional neural network

Y Kageyama, M Isogawa, D Iwai, K Sato

Optics Express 28 (14), 20391-20403 28 ( 14 ) 20391 - 20403 2020

Research paper (scientific journal)

　View Summary

Projection blur can occur in practical use cases that have non-planar and/or multi-projection display surfaces with various scattering characteristics because the surface often causes defocus and subsurface scattering. To address this issue, we propose ProDebNet, an end-to-end real-time projection deblurring network that synthesizes a projection image to minimize projection blur. The proposed method generates a projection image without explicitly estimating any geometry or scattering characteristics of the projection screen, which makes real-time processing possible. In addition, ProDebNet does not require real captured images for training data; we design a "pseudo-projected" synthetic dataset that is well-generalized to real-world blur data. Experimental results demonstrate that the proposed ProDebNet compensates for two dominant types of projection blur, i.e., defocus blur and subsurface blur, significantly faster than the baseline method, even in a real-projection scene.

• Access to Document (DOI)

Optical Non-Line-of-Sight Physics-Based 3D Human Pose Estimation.

Mariko Isogawa, Ye Yuan 0007, Matthew O'Toole, Kris M. Kitani

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) (Computer Vision Foundation / IEEE)     7011 - 7020 2020

• Access to Document (DOI)

Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation.

Mariko Isogawa, Ye Yuan 0007, Matthew O'Toole, Kris Kitani

CoRR abs/2003.14414 2020

Optical non-line-of-sight physics-based 3d human pose estimation

M Isogawa, Y Yuan, M O'Toole, KM Kitani

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern …  2020

Efficient Non-Line-of-Sight Imaging from Transient Sinograms.

Mariko Isogawa, Dorian Chan, Ye Yuan 0007, Kris Kitani, Matthew O'Toole

Computer Vision - ECCV 2020 - 16th European Conference (Springer)     193 - 208 2020

• Access to Document (DOI)

Efficient Non-Line-of-Sight Imaging from Transient Sinograms.

Mariko Isogawa, Dorian Chan, Ye Yuan 0007, Kris Kitani, Matthew O'Toole

CoRR abs/2008.02787 2020

Efficient non-line-of-sight imaging from transient sinograms

M Isogawa, D Chan, Y Yuan, K Kitani, M O’Toole

European Conference on Computer Vision, 193-208  2020

Virtual environment construction apparatus, method, and computer readable medium

K Takahashi, D Mikami, M Isogawa, A Kojima, H Kimata, A Matsumoto

US Patent 10,850,177  2020

Image quality assessment for inpainted images via learning to rank

M Isogawa, D Mikami, K Takahashi, H Kimata

Multimedia Tools and Applications 78 (2), 1399-1418 78 ( 2 ) 1399 - 1418 2019

Research paper (scientific journal)

• Access to Document (DOI)

Which is the better inpainted image? Training data generation without any manual operations

M Isogawa, D Mikami, K Takahashi, D Iwai, K Sato, H Kimata

International Journal of Computer Vision 127 (11), 1751-1766 127 ( 11-12 ) 1751 - 1766 2019

Research paper (scientific journal)

• Access to Document (DOI)

Image-based center of mass estimation of the human body via 3D shape and kinematic structure

T Kaichi, S Mori, H Saito, K Takahashi, D Mikami, M Isogawa, Y Kusachi

Sports Engineering 22 (3), 1-9 22 ( 3-4 )  2019

Research paper (scientific journal),  ISSN  1369-7072

• Access to Document (DOI)

Generative adversarial network based image blur compensation for projection-based mixed reality

Y Kageyama, M Isogawa, D Iwai, K Sato

2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), 227-228 (IEEE)     227 - 228 2019

• Access to Document (DOI)

Easy extrinsic calibration of vr system and multi-camera based marker-less motion capture system

K Takahashi, D Mikami, M Isogawa, S Sun, Y Kusachi

2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct … (IEEE)     83 - 88 2019

• Access to Document (DOI)

BODY SHAPE AND CENTER OF MASS ESTIMATION USING MULTI-VIEW IMAGES

K Ino, K Takahashi, M Isogawa, Y Kusachi, D Mikami, Y Sugiura, H Saito

ISBS Proceedings Archive 37 (1), 161  2019

新しいトレーニング法を探る~ VR トレーニングシステムとスポーツ脳科学~

三上弾， 高橋康輔， 五十川麻理子， 西條直樹

研究報告コンピュータビジョンとイメージメディア (CVIM) 2019 (13), 1-1 2019 ( CVIM-216 )  2019

VR イメージトレーニングシステ

D Mikami, K Takahashi, H Yabushita, M Isogawa, Y Kusachi, N Saijo

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and …  2019

Vr-based batter training system with motion sensing and performance visualization

K Takahashi, D Mikami, M Isogawa, Y Kusachi, N Saijo

2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 1353-1354 (IEEE)     1353 - 1354 2019

• Access to Document (DOI)

VR 技術を活用したスポーツトレーニングの試み

三上弾， 西條直樹， 高橋康輔， 五十川麻理子， 薮下浩子， 柏野牧夫， ...

日本画像学会誌 58 (3), 316-323 58 ( 3 )  2019

ISSN  1344-4425

Extrinsic camera calibration of display-camera system with cornea reflections

K Takahashi, D Mikami, M Isogawa, A Kojima, H Kimata

IEICE TRANSACTIONS on Information and Systems 101 (12), 3199-3208 101-D ( 12 ) 3199 - 3208 2018

Research paper (scientific journal),  ISSN  0913-5685

• Access to Document (DOI)

Mask optimization for image inpainting

M Isogawa, D Mikami, D Iwai, H Kimata, K Sato

IEEE Access 6, 69728-69741 6   69728 - 69741 2018

Research paper (scientific journal)

• Access to Document (DOI)

MIRU2017 若手プログラム実施報告

浦西友樹， 五十川麻理子， 井下智加， 牛久祥孝， 大倉史生， 川西康友， ...

研究報告コンピュータビジョンとイメージメディア (CVIM) 2018 (40), 1-7  2018

Human pose as calibration pattern; 3D human pose estimation with multiple unsynchronized and uncalibrated cameras

K Takahashi, D Mikami, M Isogawa, H Kimata

Proceedings of the IEEE Conference on Computer Vision and Pattern … (Computer Vision Foundation / IEEE Computer Society)     1775 - 1782 2018

• Access to Document (DOI)

Estimation of center of mass for sports scene using weighted visual hull

T Kaichi, S Mori, H Saito, K Takahashi, D Mikami, M Isogawa, H Kimata

Proceedings of the IEEE Conference on Computer Vision and Pattern … (Computer Vision Foundation / IEEE Computer Society)     1809 - 1815 2018

• Access to Document (DOI)

Virtual reality-based sports training system and its application to baseball

D Mikami, K Takahashi, N Saijo, M Isogawa, T Kimura, H Kimata

NTT Technical Review 16 (3) 16 ( 3 ) 1 - 6 2018

ISSN  1348-3447

　View Summary

This article introduces a system we are developing that provides a first-person-view experience to users. It is aimed at preparing athletes to engage in sports competitions by enabling them to experience game situations from their own point of view before the game. The use of video has rapidly become popular in the sports field in various ways such as for scouting the opposing team before the game. Such videos are usually captured from a position different from that of players while they are playing the game. Thus, they cannot simulate the game very well. Our system provides a virtual reality-based firstperson-view experience by using information captured or measured from locations that will not disturb the game. We describe in this article how our system worked in a trial with a professional baseball team.

What can VR systems tell sports players? Reaction-based analysis of baseball batters in virtual and real worlds

M Isogawa, D Mikami, T Fukuda, N Saijo, K Takahashi, H Kimata, ...

2018 IEEE conference on virtual reality and 3D user interfaces (VR), 587-588 (IEEE Computer Society)     587 - 588 2018

• Access to Document (DOI)

Image and video completion via feature reduction and compensation

M Isogawa, D Mikami, K Takahashi, A Kojima

Multimedia Tools and Applications 76 (7), 9443-9462 76 ( 7 ) 9443 - 9462 2017

Research paper (scientific journal)

• Access to Document (DOI)

Extrinsic camera calibration without visible corresponding points using omnidirectional cameras

S Miyata, H Saito, K Takahashi, D Mikami, M Isogawa, A Kojima

IEEE Transactions on Circuits and Systems for Video Technology 28 (9), 2210-2219 28 ( 9 ) 2210 - 2219 2017

Research paper (scientific journal),  ISSN  1051-8215

• Access to Document (DOI)

野球のボール追跡による多視点カメラの時間的および幾何学的キャリブレーションとボールの 3 次元軌跡復元

宮田省吾， 斎藤英雄， 高橋康輔， 三上弾， 五十川麻理子， 木全英明

研究報告コンピュータビジョンとイメージメディア (CVIM) 2017 (40), 1-5 116 ( 411(PRMU2016 127-151) )  2017

ISSN  0913-5685

Event timing detection method using quasi-high frame generation in low dimensional space

A Matsumoto, D Mikami, K Takahashi, M Isogawa, H Kimata

IEICE Technical Report; IEICE Tech. Rep. 116 (496), 1-5 116 ( 494(IMQ2016 20-55) )  2017

ISSN  0913-5685

Ball 3D trajectory reconstruction without preliminary temporal and geometrical camera calibration

S Miyata, H Saito, K Takahashi, D Mikami, M Isogawa, H Kimata

Proceedings of the IEEE Conference on Computer Vision and Pattern … (IEEE Computer Society)     164 - 169 2017

• Access to Document (DOI)

Which is the better inpainted image? learning without subjective annotation

M Isogawa, D Mikami, K Takahashi, H Kimata

28th British Machine Vision Conference, BMVC 2017 (BMVA Press)   2017

人体関節を用いた多視点カメラの外部キャリブレーション法に関する検討

高橋康輔， 三上弾， 五十川麻理子， 木全英明

研究報告コンピュータグラフィックスとビジュアル情報学 (CG) 2017 (30), 1-8 2017 ( DCC-17 )  2017

Extrinsic camera calibration with minimal configuration using cornea model and equidistance constraint

K Takahashi, D Mikami, M Isogawa, A Kojima

IPSJ Transactions on Computer Vision and Applications 8, 20-28 8   20 - 28 2016

Research paper (scientific journal)

• Access to Document (DOI)

Cornea-reflection-based Extrinsic Camera Calibration without a Direct View.

K Takahashi, D Mikami, M Isogawa, A Kojima

VISIGRAPP (3: VISAPP), 17-26 (SciTePress)     17 - 26 2016

• Access to Document (DOI)

Eye gaze analysis and learning-to-rank to obtain the most preferred result in image inpainting

M Isogawa, D Mikami, K Takahashi, A Kojima

2016 IEEE International Conference on Image Processing (ICIP), 3538-3542 (IEEE)     3538 - 3542 2016

• Access to Document (DOI)

複数カメラ映像を用いた仮想全天球映像合成に関する検討

高橋康輔， 三上弾， 五十川麻理子， 小島明

研究報告エンタテインメントコンピューティング (EC) 2015 (9), 1-6  2015

Ranking model for image inpainting

M Isogawa, D Mikami, K Takahashi, A Kojima

IEICE Technical Report; IEICE Tech. Rep. 115 (245), 49-54 115 ( 245(MVE2015 21-36) )  2015

ISSN  0913-5685

Immersive Previous Experience in VR for Sports Performance Enhancement

D Mikami, M Isogawa, K Takahashi, H Takada, A Kojima

Proc. of 3rd International Congress on Sport Sciences Research and …  2015

Extrinsic Camera Calibration of Display-Camera System using Cornea Reflection

K Takahashi, D Mikami, M Isogawa, A Kojima

IEICE Technical Report; IEICE Tech. Rep. 115 (245), 61-66  2015

Content Completion in Lower Dimensional Feature Space through Feature Reduction and Compensation.

Mariko Isogawa, Dan Mikami, Kosuke Takahashi, Akira Kojima

2015 IEEE International Symposium on Mixed and Augmented Reality(ISMAR) (IEEE Computer Society)     156 - 159 2015

• Access to Document (DOI)

Content completion by restoring in narrow feature space and compensating lost features

M Isogawa, D Mikami, K Takahashi, A Kojima

IEICE Technical Report; IEICE Tech. Rep. 115 (76), 37-42  2015

Automatic Visual Feedback from Multiple Views for Motor Learning.

Dan Mikami, Mariko Isogawa, Kosuke Takahashi, Akira Kojima

2015 IEEE International Symposium on Mixed and Augmented Reality(ISMAR) (IEEE Computer Society)     212 - 213 2015

• Access to Document (DOI)

A Study on Virtual Omnidirectional Video Synthesis from Multiple Cameras

K Takahashi, D Mikami, M Isogawa, A Kojima

IEICE Technical Report; IEICE Tech. Rep. 115 (76), 43-48  2015

[POSTER] Toward enhancing robustness of DR system: Ranking model for background inpainting

M Isogawa, D Mikami, K Takahashi, A Kojima

2015 IEEE International Symposium on Mixed and Augmented Reality, 178-179  2015

[POSTER] Content Completion in Lower Dimensional Feature Space through Feature Reduction and Compensation

M Isogawa, D Mikami, K Takahashi, A Kojima

2015 IEEE International Symposium on Mixed and Augmented Reality, 156-159  2015

[POSTER] Automatic visual feedback from multiple views for motor learning

D Mikami, M Isogawa, K Takahashi, A Kojima

2015 IEEE International Symposium on Mixed and Augmented Reality, 212-213  2015

特徴量の削減と合成に基づく低次元特徴量空間を用いたコンテンツコンプリーション

五十川麻理子， 三上弾， 高橋康輔， 小島明

研究報告エンタテインメントコンピューティング (EC) 2015 (8), 1-6 115 ( 76(MVE2015 1-7) )  2015

ISSN  0913-5685

Toward Enhancing Robustness of DSystem: Ranking Model for Background Inpainting.

Mariko Isogawa, Dan Mikami, Kosuke Takahashi, Akira Kojima

2015 IEEE International Symposium on Mixed and Augmented Reality(ISMAR) (IEEE Computer Society)     178 - 179 2015

• Access to Document (DOI)

Virtual Omnidirectional Video Synthesis with Multiple Cameras for Sports Training.

M Isogawa, D Mikami, K Takahashi, A Kojima

icSPORTS, 271-275 (SciTePress)     271 - 275 2015

• Access to Document (DOI)

Making graphical information visible in real shadows on interactive tabletops

M Isogawa, D Iwai, K Sato

IEEE Transactions on Visualization and Computer Graphics 20 (9), 1293-1302 20 ( 9 ) 1293 - 1302 2014

Research paper (scientific journal), Lead author

　View Summary

We introduce a shadow-based interface for interactive tabletops. The proposed interface allows a user to browse graphical information by casting the shadow of his/her body, such as a hand, on a tabletop surface. Central to our technique is a new optical design that utilizes polarization in addition to the additive nature of light so that the desired graphical information is displayed only in a shadow area on a tabletop surface. In other words, our technique conceals the graphical information on surfaces other than the shadow area, such as the surface of the occluder and non-shadow areas on the tabletop surface. We combine the proposed shadow-based interface with a multi-touch detection technique to realize a novel interaction technique for interactive tabletops. We implemented a prototype system and conducted proof-of-concept experiments along with a quantitative evaluation to assess the feasibility of the proposed optical design. Finally, we showed implemented application systems of the proposed shadow-based interface.

• Access to Document (DOI)

影のメタファを利用したポインティングインタフェースにおける影の制約緩和による操作性の向上 (< 特集> 複合現実感 6)

武内真梨奈， 五十川麻理子， 岩井大輔， 佐藤宏介

日本バーチャルリアリティ学会論文誌 19 (2), 207-214  2014

Research paper (scientific journal)

特徴量の削減と合成に基づく体感品質に優れた画像修復

五十川麻理子， 三上弾， 小島明

電子情報通信学会技術研究報告; 信学技報 114 (239), 37-42 114 ( 239(MVE2014 32-48) )  2014

ISSN  0913-5685

Improvement of Operability by Loosing Spatial Relationship with Real Shadow on the Pointing Interface which Uses a Shadow Metaphor

M TAKEUCHI, M ISOGAWA, D IWAI, K SATO

日本バーチャルリアリティ学会論文誌 19 (2), 207-214 19 ( 2 )  2014

ISSN  1344-011X

Weak perspective shadow interface for seated user's pointing on large wall display

M Takeuchi, M Isogawa, D Iwai, K Sato

2014 IEEE/SICE International Symposium on System Integration, 316-321 (IEEE)     316 - 321 2014

Accepted

　View Summary

Shadow is widely researched as a user interface for manipulating graphical information on large displays. The shadow of one's fingertip is used as a pointing cursor whose position and size (i.e., gain and reaching area) can be intuitively adjusted, because one thoroughly comprehend the shadow's perspective projection behavior. The goal of this research is to allow a seated user, who cannot sufficiently move his/her fingertip, to manipulate the shadow cursor. We achieve this by relaxing the perspective constraints, i.e., applying weak perspective projection model. Through an experiment, we confirmed that the pointing performance of seated users is improved in the proposed method compared with the conventional method. Specifically, the users completed pointing tasks 25.1 % faster with 54.2 % fewer errors when the target size was small, 34.5 % faster with 64.2 % fewer errors when the target size was medium, and with 58.3 % fewer errors when the target size was large.

体感品質の良い多視点映像提示手法

三上弾

電子情報通信学会技術研究報告= IEICE technical report: 信学技報 114 (114), 7-12 114 ( 114(MVE2014 13-31) )  2014

ISSN  0913-5685

Optically hiding of tabletop information with polarized complementary image projection - Your shadow reveals it!

Mariko Miki, Daisuke Iwai, Kosuke Sato

Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS'11    260 - 261 2011

Accepted

　View Summary

We propose the concept and implementation of a graphical information hiding technique for interactive tabletops where users can view the information by simply casting real shadows. We placed three projectors (one in the rear and two in the front) in such a way that the rear one projects graphical information onto a tabletop surface, and the front ones project a complementary image, so that the combined image displayed on the surface becomes uniformly gray, thus hiding the information from the viewer. Users can view the hidden information by blocking the light from the front projector, revealing the complementary image that is being projected onto the occluder. We use the other front projector and polarization filters to make the complementary image projected onto the occluder also uniformly gray. Because the technique completely relies on optical phenomena, users can interact with the system without suffering from any false recognitions or delays. © 2011 ACM.

• Access to Document (DOI)