研究者詳細 - 斎藤　英雄

論文 - 斎藤　英雄

分割表示 356 件中 41 - 60 件目／全件表示 >>

3D PixMix: Image Inpainting in 3D Environments

Mori S., Herling J., Broll W., Kawai N., Saito H., Schmalstieg D., Kalkofen D.

Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 （Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018） 1 - 2 2018年07月

ISSN 9781538675922

© 2018 IEEE. State of the art methods for diminished reality (DR) propagate pixel information from keyframes to later frames to achieve real-time image inpainting in 3D spaces. However, this approach assumes a planar scene and produces artifacts, if the scene geometry is not sufficiently planar. In this paper, we present 3D PixMix, a new real-time inpainting method that addresses non-planar scenes by considering both color and depth information in the inpainting process. We define cost functions for both the color and the geometric appearance in the inpainting scheme and use an RGB-D sensor for depth fusion using a SLAM. Comparison results against the conventional PixMix show that 3D PixMix obtains the equivalent or even better quality in 3D scenes with additional depth information.

Access to Document (DOI)

Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry

Yamaguchi M., Truong T.P., Mori S., Nozick V., Saito H., Yachida S., Sato H.

IEICE Transactions on Information and Systems （IEICE Transactions on Information and Systems） E101D （ 5 ） 1296 - 1307 2018年05月

ISSN 09168532

© 2018 The Institute of Electronics, Information and Communication Engineers. In this paper, we propose a method to generate a threedimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.

Access to Document (DOI)

Line-based SLAM using non-overlapping cameras in an urban environment

Kawasaki A., Hara K., Saito H.

IEICE Transactions on Information and Systems （IEICE Transactions on Information and Systems） E101D （ 5 ） 1232 - 1242 2018年05月

ISSN 09168532

© 2018 The Institute of Electronics, Information and Communication Engineers. We propose a method of line-based Simultaneous Localization and Mapping (SLAM) using non-overlapping multiple cameras for vehicles running in an urban environment. It uses corresponding line segments between images taken by different frames and different cameras. The contribution is a novel line segment matching algorithm by warping processing based on urban structures. This idea significantly improves the accuracy of line segment matching when viewing direction are very different, so that a number of correspondences between front-view and rear-view cameras can be found and the accuracy of SLAM can be improved. Additionally, to enhance the accuracy of SLAM we apply a geometrical constraint of urban area for initial estimation of 3D mapping of line segments and optimization by bundle adjustment. We can further improve the accuracy of SLAM by combining points and lines. The position error is stable within 1.5m for the entire image dataset evaluated in this paper. The estimation accuracy of our method is as high as that of ground truth captured by RTK-GPS. Our high accuracy SLAM algorithm can be apply for generating a road map represented by line segments. According to an evaluation of our generating map, true positive rate around the vehicle exceeding 70% is achieved.

Access to Document (DOI)

Simultaneous object segmentation and recognition by merging CNN outputs from uniformly distributed multiple viewpoints

Nakajima Y., Saito H.

IEICE Transactions on Information and Systems （IEICE Transactions on Information and Systems） E101D （ 5 ） 1308 - 1316 2018年05月

ISSN 09168532

© 2018 The Institute of Electronics, Information and Communication Engineers. We propose a novel object recognition system that is able to (i) work in real-time while reconstructing segmented 3D maps and simultaneously recognize objects in a scene, (ii) manage various kinds of objects, including those with smooth surfaces and those with a large number of categories, utilizing a CNN for feature extraction, and (iii) maintain high accuracy no matter how the camera moves by distributing the viewpoints for each object uniformly and aggregating recognition results from each distributed viewpoint as the same weight. Through experiments, the advantages of our system with respect to current state-of-the-art object recognition approaches are demonstrated on the UW RGB-D Dataset and Scenes and on our own scenes prepared to verify the effectiveness of the Viewpoint-Class-based approach.

Access to Document (DOI)

Synchronizing 3D point cloud from 3D scene flow estimation with 3D Lidar and RGB camera

Usami H., Saito H., Kawai J., Itani N.

IS and T International Symposium on Electronic Imaging Science and Technology （IS and T International Symposium on Electronic Imaging Science and Technology） 1 - 6 2018年01月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

© 2018 Society for Imaging Science and Technology. We present a method for synchronizing three-dimensional (3D) point cloud from 3D scene with estimation using a 3D Lidar and an RGB camera. These 3D points sensed by the 3D Lidar are not captured at the same time, which makes it difficult to measure the correct shape of the object in a dynamic scene. In our method, we generate synchronized 3D points at arbitrary times using linear interpolation in four-dimensional space, time and space. For interpolating the 3D point, we obtain corresponding 3D point matching with the pixel value captured by the RGB camera in a continuous frame. The experimental results demonstrate the effectiveness of the presented method by depicting a synchronized 3D point cloud that is correctly shaped.

Access to Document (DOI)

Vehicle pose estimation from drive recorder images by monocular SLAM and matching with rendered 3D point cloud of surrounding environment

Kurobe A., Kinoshita H., Saito H.

IS and T International Symposium on Electronic Imaging Science and Technology （IS and T International Symposium on Electronic Imaging Science and Technology） 2371 - 2376 2018年

© 2018, Society for Imaging Science and Technology. Vehicle pose estimation is a vital technology for reconstructing the circumstaces of traffic accidents. We propose a novel method for reconstructing the trajectory of vehicles from drive recorder images and a point cloud around the road. First, we apply ORB-SLAM to image sequence of the drive recorder for obtaining the vehicle pose trajectory; however this is based on relative coordinates and a relative scale. For estimating the absolute coordinates and scale of the trajectory, which cannot be obtained from a monocular SLAM like ORB-SLAM, we match the feature points detected in the image sequence with the three-dimensional (3D) point cloud of surrounding environment. For finding 3D points matching the feature points, we generate candidate images by the rendering 3D point cloud of the surrounding environment using the position initially estimated by the Global Positioning System (GPS). Next, we match to obtain the 3D two-dimensional (2D) generated images and drive recorder image to get 3D-2D point correspondences between the 3D point cloud and the drive recorder images; thus, we can convert the relative estimation of the camera pose by ORB-SLAM to the coordinates of the 3D point cloud of the surrounding environment. In the evaluation experiments, we confirmed the effectiveness of our method by comparing the vehicle poses estimated by our method, with those of RTKGPS, which exhibits high measurement precision.

Access to Document (DOI)

Synthesis of wide-FOV RGB-D images by registration and upsampling of 3D Lidar with omnidirectional RGB camera

Usami H., Miyata S., Saito H.

23rd International Display Workshops in conjunction with Asia Display, IDW/AD 2016 （23rd International Display Workshops in conjunction with Asia Display, IDW/AD 2016） 4 2340 - 2342 2018年

ISSN 9781510845510

Multi-view surface inspection using a rotating table

Kaichi T., Mori S., Saito H., Sugano J., Adachi H.

IS and T International Symposium on Electronic Imaging Science and Technology （IS and T International Symposium on Electronic Imaging Science and Technology） 2021 - 2026 2018年

© 2018, Society for Imaging Science and Technology. In this paper, we introduce a method to visually inspect in- dustrial parts by using multi-view images of an industrial part on a rotating table. During the visual inspection of the parts' sur- faces, the relationship between the camera pose, the light vectors, and the normal vectors of a part's surface is the key factor in de- tecting abnormalities. We can change the relationship between the three factors by rotating the part; then, the abnormalities are visible in certain positional relationships. We, therefore, track the points on a part's surfaces in an image sequence and discrimi- nate abnormal points (such as points on scratches or dents) from normal points based on pixel value transitions while rotating the part. The experimental results, which were based on real data of industrial parts, showed that the proposed method could detect the abnormalities on the surfaces.

Access to Document (DOI)

Constant velocity 3d convolution

Sekikawa Y., Ishikawa K., Saito H.

IEEE Access （IEEE Access） 6 76490 - 76501 2018年

© 2018 IEEE. We propose a novel 3-D convolution method, cv3dconv, for extracting spatiotemporal features from videos. It reduces the number of sum-of-products operations in 3-D convolution by thousands of times by assuming the constant moving velocity of the features. We observed that a specific class of video sequences, such as video captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2-D features in a temporal dimension. Our principal finding is that a 3-D kernel, represented by constant velocity, can be decomposed into a convolution of a 2-D-shaped kernel and a 3-D-velocity kernel, which is parameterized using only two parameters. We derived an efficient recursive algorithm for this class of 3-D convolution, which is exceptionally suited for sparse spatiotemporal data, and this parameterized decomposed representation imposes a structured regularization along a temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of the cv3dconv by adopting it for deep neural networks (DNNs) in visual odometry estimation task using publicly available event-based camera dataset captured in urban road scene. Our DNN architecture improves the estimation accuracy for about 30% compared with the existing states-of-the-arts architecture designed for event data.

Access to Document (DOI)

An overview of augmented visualization: Observing the real world as desired

Mori S., Saito H.

APSIPA Transactions on Signal and Information Processing （APSIPA Transactions on Signal and Information Processing） 7 2018年

Copyright © The Authors, 2018 This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.. Over 20 years have passed since a free-viewpoint video technology has been proposed with which a user's viewpoint can be freely set up in a reconstructed three-dimensional space of a target scene photographed by multi-view cameras. This technology allows us to capture and reproduce the real world as recorded. Once we capture the world in a digital form, we can modify it as augmented reality (i.e., placing virtual objects in the digitized real world). Unlike this concept, the augmented world allows us to see through real objects by synthesizing the backgrounds that cannot be observed in our raw perspective directly. The key idea is to generate the background image using multi-view cameras, observing the backgrounds at different positions and seamlessly overlaying the recovered image in our digitized perspective. In this paper, we review such desired view-generation techniques from the perspective of free-view point image generation and discuss challenges and open problems through a case study of our implementations.

Access to Document (DOI)

A simple 3D scanning system of human foot using a smartphone with a depth camera

Kobayashi T., Ienaga N., Sugiura Y., Saito H., Miyata N., Tada M.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering （Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering） 84 （ 12 ） 996 - 1002 2018年

ISSN 09120289

© 2018 Japan Society for Precision Engineering. All rights reserved. This paper presents a method to measure foot shape from a 3D point cloud as input captured from multiple directions using a smartphone depth camera. Such a 3D point cloud could potentially include noise or omit parts of the foot due to occlusion. To deal with this occlusion problem, we propose to use a dataset of 3D foot shapes collected by a precise 3D shape scanner of foot shapes. According to the dataset of 3D foot shapes, we can generate a deformable model by performing a principal component analysis (PCA) on the dataset. Then we minimize the error of the shape represented by the deformable model and the 3D point cloud acquired by the smartphone camera, to recover a complete 3D shape of the entire foot with high accuracy. We test this method by comparing the 3D shape produced by our proposed method to the 3D shape precisely measured by the 3D scanner. Our proposed method can scan the foot shape with an error of about 1,13mm.

Access to Document (DOI)

A refocus-interface for diminished reality work area visualization

Maezawa M., Mori S., Saito H.

IS and T International Symposium on Electronic Imaging Science and Technology （IS and T International Symposium on Electronic Imaging Science and Technology） 1121 - 1124 2018年

© 2018, Society for Imaging Science and Technology. In this paper, we present a refocus interface to set the parameters used for diminished reality (DR)-based work area visualization and a multiview camera-based rendering scheme. The refocus interface allows the user to determine two planes - one for setting a virtual window, through which the user can observe the background occluded by an object, and the other for a background plane, which is used for the subsequent background rendering. The background is rendered considering the geometric and appearance relationships of the multiview cameras observing the scene. Our preliminary results demonstrate that our DR system can visualize the hidden background.

Access to Document (DOI)

Synthesis of top view image and the detection of obstacles using multiple cameras for monitoring rea around a truck

Uehara K., Saito H., Yamamoto K., Sato H.

23rd International Display Workshops in conjunction with Asia Display, IDW/AD 2016 （23rd International Display Workshops in conjunction with Asia Display, IDW/AD 2016） 4 2304 - 2307 2018年

ISSN 9781510845510

Augmented visualization: Observing as desired

Mori S., Saito H.

Proceedings - 9th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017 （Proceedings - 9th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017） 2018-February 351 - 356 2017年12月

研究論文（国際会議プロシーディングス）, 査読有り, ISSN 9781538615423

© 2017 IEEE. Over 20 years have passed since a free-viewpoint video technology has been proposed in which a user's viewpoint can be freely set up reconstructed 3D space of a target scene photographed by multi-view cameras. This technology allows us to capture and reproduce the real world as it recorded. Once we capture the world in a digital form, we can modify the world as augmented reality does (i.e., placing virtual objects in the digitized real world). As oppose to this concept, the augmented world also allows us to see through real objects by synthesizing the backgrounds that cannot be observed in our raw perspective directly. The key idea is to generate the background image using multi-view cameras observing the backgrounds at different positions, and seamlessly overlaying the recovered image in our digitized perspective. In this article, our studies regarding such desired view generation techniques are reviewed, and its future directions and open problems are discussed.

Access to Document (DOI)

回転台を用いて撮像した多視点画像列による工業部品の外観検査

鶏内朋也, 森尚平, 斎藤英雄, 菅野純一, 足立秀之

精密工学会誌 83 （ 12 ） 1184 - 1191 2017年12月

研究論文（学術雑誌）, 査読有り

Visual inspection is a vital step for maintaining the quality of industrial parts. In visual inspection, the relation between the three factors, camera pose, lights direction and normal vector of a part is very important to detect abnormalities on industrial part. We propose a visual inspection method for industrial parts using an image sequence captured while rotating an industrial part using a rotating table. By rotating a part, we can easily change the relation between three factors. We track the points on part's surfaces in an image sequence. We discriminate abnormal surface points such as scratches from the normal ones based on pixel value transitions in an image sequence. For accurately tracking the points on the part surface, we propose a novel camera calibration method of the rotating table and the camera with a telecentric lens and a pose estimation method for the rotating part. We use angular invariant feature vectors for improving robustness against types and shapes of industrial parts to be inspected. We presented experimental results using real data of industrial parts and verified that the proposed method could detect scratches on surfaces.

Access to Document (DOI)

都市環境における線分特徴の方向性分布を利用したSLAM

上原啓, 斎藤英雄, 原　孝介

画像電子学会誌 46 （ 4 ） 522 - 532 2017年12月

研究論文（学術雑誌）, 共著, 査読有り

本論文では，線分特徴の事前分布を考慮した自動運転車のためのSLAM (Simultaneous Localization and Mapping) を提案する．提案手法は，道路標示や建物など，線分を検出しやすい物体が存在する都市環境を走行する車両に搭載されたカメラにより撮影される動画像を入力とすることを前提とし，この動画像から検出される線分特徴の方向に関する制約を加えることによりSLAM の精度を向上させるものである．線分特徴の方向に関する制約を加えるため，本手法では道路標示中の線分が車両進行方向に対して平行，もしくは垂直であるものが多いことに着目し，それら2 つの線分間の角度分布が混合ガウス分布に則ると仮定し，バンドル調整による最適化に導入する．これにより，カメラの位置姿勢と線分特徴の位置推定の精度を高めることが期待できる．本手法の有効性を確かめるため，カメラを1 つ用いたシングルカメラシステムと，2 つ用いたマルチカメラシステムの両方に提案手法を適用し精度検証を行った．その結果マルチカメラによる従来手法の精度と，シングルカメラによる本手法の精度が同等となった．また提案手法により生成した路面地図は，線分の方向に関する制約を導入したことにより補正され，精度向上を確認した．

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

Meerits S., Nozick V., Saito H.

Journal of Real-Time Image Processing （Journal of Real-Time Image Processing） 16 （ 6 ） 1 - 13 2017年11月

研究論文（学術雑誌）, ISSN 1861-8200

© 2017, The Author(s). We present a novel 3D reconstruction system that can generate a stable triangle mesh using data from multiple RGB-D sensors in real time for dynamic scenes. The first part of the system uses moving least squares (MLS) point set surfaces to smooth and filter point clouds acquired from RGB-D sensors. The second part of the system generates triangle meshes from point clouds. The whole pipeline is executed on the GPU and is tailored to scale linearly with the size of the input data. Our contributions include changes to the MLS method for improving meshing, a fast triangle mesh generation method and GPU implementations of all parts of the pipeline.

Access to Document (DOI)

Semantic Object Selection and Detection for Diminished Reality Based on SLAM with Viewpoint Class

Nakajima Yoshikatsu, Mori Shohei, Saito Hideo

2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) 338 - 343 2017年10月

研究論文（国際会議プロシーディングス）, 査読有り

We propose a novel diminished reality method which is able to (i) automatically recognize the region to be diminished, (ii) work with a single RGB-D sensor, and (iii) work without pre-processing to generate a 3D model of the target scene by utilizing SLAM, segmentation, and recognition framework. Especially, regarding the recognition of the area to be diminished, our method is able to maintain high accuracy no matter how the camera moves by distributing the viewpoints for each object uniformly and aggregating recognition results from each distributed viewpoint as the same weight. These advantages are demonstrated on the UW RGB-D Dataset and Scenes.

Access to Document (DOI)

An Instant See-Through Vision System Using a Wide Field-of-View Camera and a 3D-Lidar

Oishi Kei, Mori Shohei, Saito Hideo

2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) 344 - 347 2017年10月

研究論文（国際会議プロシーディングス）

Diminished reality (DR) enables us to see through real objects occluding some areas in our field of view. This interactive display has various applications, such as see-through vision to visualize invisible areas, work area visualization in surgery and landscape simulation. In this paper, we propose two underlying problems in see-through vision, in which hidden areas are observed in real time. First, see-through vision methods require a common area to calibrate every camera in the environment. However, the field of view is limited and many approaches rely on a time-consuming calibration, sensors, or fiducial markers. Second, see-through vision applications assume that the background is planar to ease image alignment. We therefore present a place-and-play see-through vision system using a wide field-of-view RGB-D camera. We validated the accuracy and the robustness of our system and showed results in various environments to show the applicability.

Access to Document (DOI)

Diminished Reality for Privacy Protection by Hiding Pedestrians in Motion Image Sequences Using Structure from Motion

Yagi Kentaro, Hasegawa Kunihiro, Saito Hideo

2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) 334 - 337 2017年10月

研究論文（国際会議プロシーディングス）, 査読有り

We present a method for generating images in which people are hidden from image sequences taken with a hand-held camera. Our method is basically used for privacy protection of people whose images are unintentionally captured in image sequences. We hide people from images by reconstructing a 3D model of background and projecting it to 2D images. By detecting the area in which people are present beforehand, we can reconstruct a 3D model of the background without people. In the experiment, We compare our method with some conventional approaches for diminished reality.

Access to Document (DOI)

　前のページ - 次のページ　

斎藤 英雄

論文 - 斎藤 英雄

斎藤　英雄

論文 - 斎藤　英雄