研究者詳細 - 斎藤　英雄

論文 - 斎藤　英雄

分割表示 356 件中 21 - 40 件目／全件表示 >>

Eventnet: Asynchronous recursive event processing

Sekikawa Y., Hara K., Saito H.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition （Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition） 2019-June 3882 - 3891 2019年06月

ISSN 9781728132938

© 2019 IEEE. Event cameras are bio-inspired vision sensors that mimic retinas to asynchronously report per-pixel intensity changes rather than outputting an actual intensity image at regular intervals. This new paradigm of image sensor offers significant potential advantages; namely, sparse and non-redundant data representation. Unfortunately, however, most of the existing artificial neural network architectures, such as a CNN, require dense synchronous input data, and therefore, cannot make use of the sparseness of the data. We propose EventNet, a neural network designed for real-time processing of asynchronous event streams in a recursive and event-wise manner. EventNet models dependence of the output on tens of thousands of causal events recursively using a novel temporal coding scheme. As a result, at inference time, our network operates in an event-wise manner that is realized with very few sum-of-the-product operations--look-up table and temporal feature aggregation--which enables processing of 1 mega or more events per second on standard CPU. In experiments using real data, we demonstrated the real-time performance and robustness of our framework.

Access to Document (DOI)

Simultaneous 3D Tracking and Reconstruction of Multiple Moving Rigid Objects

Ozawa T., Nakajima Y., Saito H.

Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 （Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019） 2019年05月

ISSN 9781728115719

© 2019 IEEE. Most SLAM works based on the assumption of static scene, so localization of the camera and mapping of the scene can fail and lose accuracy of the scene, including moving objects. This paper presents a method for simultaneous mapping of moving objects in the target scene and localization of the moving camera, based on geometrical segmentation of each temporal frame. Taking advantage of segmentation of the target scene, using only the geometric structure of the scene, our method can estimate relative pose for camera and every geometrically segmented areas even without recognizing each object. For confirming the effectiveness of the proposed method, we experimentally show that our method can estimate relative poses for all segmented areas in the scene, so that we can achieve SLAM for the scene, including multiple moving objects.

Access to Document (DOI)

Semantic Segmentation of 3D Point Cloud to Virtually Manipulate Real Living Space

Ishikawa Y., Hachiuma R., Ienaga N., Kuno W., Sugiura Y., Saito H.

ISSN 9781728115719

© 2019 IEEE. This paper presents a method for the virtual manipulation of real living space using semantic segmentation of a 3D point cloud captured in the real world. We applied PointNet to segment each piece of furniture from the point cloud of a real indoor environment captured by moving a RGB-D camera. For semantic segmentation, we focused on local geometric information not used in PointNet, and we proposed a method to refine the class probability of labels attached to each point in PointNet's output. The effectiveness of our method was experimentally confirmed. We then created 3D models of real-world furniture using a point cloud with corrected labels, and we virtually manipulated real living space using Dollhouse VR, a layout system.

Access to Document (DOI)

Automatic eyeglasses replacement for a 3d virtual try-on system

Kobayashi T., Sugiura Y., Saito H., Uema Y.

ACM International Conference Proceeding Series （ACM International Conference Proceeding Series） 2019年03月

ISSN 9781450365475

©2019 Association for Computing Machinery. This paper presents a 3D virtual eyeglasses try-on system for practical use. For fitting eyeglasses in a shop, consumers wish to look at themselves in a mirror while trying on various eyeglass styles. However, for people who need to wear eyeglasses for correcting problems with eyesight, it is impossible for them to clearly observe their face in the mirror without wearing eyeglasses. This makes fitting them for new eyeglasses difficult. This research proposes a virtual try-on system that can be used while wearing eyeglasses.We replace the user's eyeglasses in the input video with new eyeglasses virtually. Moreover, a fast and accurate face tracking tool enables our system to automatically display 3D virtual glasses following a user's head motion. Experimental results demonstrate that the proposed method can render virtual glasses naturally while the user is wearing real eyeglasses.

Access to Document (DOI)

Removing fences from sweep motion videos using global 3D reconstruction and fence-aware light field rendering

Lueangwattana C., Mori S., Saito H.

Computational Visual Media （Computational Visual Media） 5 （ 1 ） 21 - 32 2019年03月

ISSN 20960433

© 2018, The Author(s). Diminishing the appearance of a fence in an image is a challenging research area due to the characteristics of fences (thinness, lack of texture, etc.) and the need for occluded background restoration. In this paper, we describe a fence removal method for an image sequence captured by a user making a sweep motion, in which occluded background is potentially observed. To make use of geometric and appearance information such as consecutive images, we use two well-known approaches: structure from motion and light field rendering. Results using real image sequences show that our method can stably segment fences and preserve background details for various fence and background combinations. A new video without the fence, with frame coherence, can be successfully provided.

Access to Document (DOI)

Self-assessment application of flexion and extension

Ienaga N., Sugiura Y., Saito H., Fujita K.

2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019 （2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019） 150 - 152 2019年03月

ISSN 9781728105437

© 2019 IEEE. The measurement of range of motion during medical examination is a basic procedure in diagnosis and treatment evaluation. However, accurate measurement requires experience, and it is difficult for patients to measure range of motion by themselves. If patients can easily and accurately measure the range by their own, it will encourage patients to continue rehabilitation and will be useful for helping patients to determine when they should see a doctor. We developed a wrist flexion and extension measurement system using images captured by a smart phone so that patients can measure range of motion without expert knowledge or experience. First, the system calculates a middle line for the forearm and an outside line for the dorsal hand based on user inputs. Afterward, the angle formed by these lines is calculated. The experimental results show that the average absolute errors are 4.82° for flexion and 8.74° for extension.

Access to Document (DOI)

Volumetric Representation of Semantically Segmented Human Body Parts Using Superquadrics

Hachiuma R., Saito H.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) （Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)） 11883 LNCS 52 - 61 2019年

ISSN 9783030319076

© Springer Nature Switzerland AG 2019. Superquadrics are one of the ideal shape representations for adapting various kinds of primitive shapes with a single equation. This paper revisits the task of representing a 3D human body with multiple superquadrics. As a single superquadric surface can only represent symmetric primitive shapes, we present a method that segments the human body into body parts to estimate their superquadric parameters. Moreover, we propose a novel initial parameter estimation method by using 3D skeleton joints. The results show that superquadric parameters are estimated, which represent human body parts volumetrically.

Access to Document (DOI)

Vehicle trajectory estimation method by drive recorder images and point cloud of surrounding environment

Kurobe A., Kinoshita H., Saito H.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering （Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering） 85 （ 3 ） 274 - 281 2019年

ISSN 09120289

© 2019 Japan Society for Precision Engineering. All rights reserved. The on-the-spot inspections after accident are based on information obtained from the brake marks left on the road and the damage situation of the surroundings. However, this method is affected by the situation before the accident and cannot estimate quantitative guessing. Moreover, technologies such as automatic braking system and lane keeping system, have been developed actively. In developing these technologies, vehicle trajectory is one of the most important data. In this paper, we propose a novel method of estimating vehicle trajectory by using drive recorder images and point cloud which is measured by Mobile Mapping System and LiDAR. In the evaluation experiments, we confirmed the effectiveness of our method by comparing the vehicle poses estimated with RTK-GPS and showing the average errors are within one meter.

Access to Document (DOI)

A Thumb Tip Wearable Device Consisting of Multiple Cameras to Measure Thumb Posture

Ienaga N., Kawai W., Fujita K., Miyata N., Sugiura Y., Saito H.

ISSN 9783030210731

© 2019, Springer Nature Switzerland AG. Today, cameras have become smaller and cheaper and can be utilized in various scenes. We took advantage of that to develop a thumb tip wearable device to estimate joint angles of a thumb as measuring human finger postures is important in terms of human-computer interface and to analyze human behavior. The device we developed consists of three small cameras attached at different angles so the cameras can capture the four fingers. We assumed that the appearance of the four fingers would change depending on the joint angles of the thumb. We made a convolutional neural network learn a regression relationship between the joint angles of the thumb and the images taken by the cameras. In this paper, we captured the keypoint positions of the thumb with a USB sensor device and calculated the joint angles to construct a dataset. The root mean squared error of the test data was 6.23° and 4.75° .

Access to Document (DOI)

Semi-independent stereo visual odometry for different field of view cameras

Truong T.P., Nozick V., Saito H.

ISSN 9783030110086

© Springer Nature Switzerland AG 2019. This paper presents a pipeline for stereo visual odometry using cameras with different fields of view. It gives a proof of concept about how a constraint on the respective field of view of each camera can lead to both an accurate 3D reconstruction and a robust pose estimation. Indeed, when considering a fixed resolution, a narrow field of view has a higher angular resolution and can preserve image texture details. On the other hand, a wide field of view allows to track features over longer periods since the overlap between two successive frames is more substantial. We propose a semi-independent stereo system where each camera performs individually temporal multi-view optimization but their initial parameters are still jointly optimized in an iterative framework. Furthermore, the concept of lead and follow camera is introduced to adaptively propagate information between the cameras. We evaluate the method qualitatively on two indoor datasets, and quantitatively on a synthetic dataset to allow the comparison across different fields of view.

Access to Document (DOI)

Line-based SLAM considering prior distribution of distance and angle of line features in an urban environment

Uehara K., Saito H., Hara K.

Communications in Computer and Information Science （Communications in Computer and Information Science） 983 105 - 127 2019年

ISSN 9783030122089

© Springer Nature Switzerland AG 2019. In this paper, we propose a line-based SLAM from an image sequence captured by a camera mounted on a vehicle in consideration with the prior distribution of line features that detected in an urban environments. Since such scenes captured by the vehicle in urban envirounments can be expected to include a lot of line segments detected from road markings and buildings, we employ line segments as features for our SLAM. We use additional prior regarding the line segments so that we can improve the accuracy of the SLAM. We assume that the angle of the vector of the line segments to the vehicle’s direction of travel conform to four-component Gaussian mixture distribution. We define a new cost function considering the prior distribution and optimize the relative camera pose, position, and the 3D line segments by bundle adjustment. The prior distribution is also extended into 2D, the distance and angle of the line segments. In addition, we make digital maps from the detected line segments. Our method increases the accuracy of localization and corrects tilted lines in the digital maps. We apply our method to both the single-camera system and the multi-camera system for demonstrate the accuracy improvement by the prior distribution of distance and angle of line features.

Access to Document (DOI)

Learning an optimisable semantic segmentation map with image conditioned variational autoencoder

Zhuang P., Sekikawa Y., Hara K., Saito H.

ISSN 9783030306441

© Springer Nature Switzerland AG 2019. Recent semantic segmentation systems have achieved significant improvement by performing pixel-wise training with hierarchical features using deep convolutional neural network models. While the learning process usually requires pixel-level annotated images, it is difficult to get desirable amounts of fine-labeled data and thus the training set size is more likely to be limited, often in thousands. This means that top methods for a dataset can be fine-tuned for a specific situation, making the generalization ability unclear. In real-world applications like self-driving systems, ambiguous region or lack of context information can cause errors in the predicted results. Resolving such ambiguities is crucial for subsequent operations to be performed safely. We are inspired by work from CodeSLAM where optimizable pixel-wise depth representation is learned. We modify the regression method to work on the pixel-wise classification problem. By training a variational auto-encoder network conditioned with a color image, the computed latent space works as a low-dimensional representation of semantic segmentation, which can be efficiently optimized. As a consequence, our model can correct the error or ambiguity of the prediction during the inference phase given useful scene information. We show how this approach works by giving partial scene truth and perform optimization on the latent variable.

Access to Document (DOI)

Image-based center of mass estimation of the human body via 3D shape and kinematic structure

Kaichi T., Mori S., Saito H., Takahashi K., Mikami D., Isogawa M., Kusachi Y.

Sports Engineering （Sports Engineering） 22 （ 3-4 ） 2019年

ISSN 13697072

© 2019, International Sports Engineering Association. This paper presents a method to estimate a time-sequential trajectory of the center of mass (CoM) of an athlete from a multi-view set of cameras. Collecting the CoM typically requires large-scale measuring systems or attaching sensors to the athletes. To mitigate such hardware limitations, the present study takes a multi-view video-based approach. The proposed method reconstructs subjects’ voxels from a set of multi-view frames and weights each voxel with body part-dependent weights to calculate a CoM. Our results, using real data measured in a studio, showed that the proposed method can estimate CoM within 20 mm concerning center of pressure measures.

Access to Document (DOI)

Efficient object-oriented semantic mapping with object detector

Nakajima Y., Saito H.

IEEE Access （IEEE Access） 7 3206 - 3213 2019年

© 2018 IEEE. Incrementally, building a 3D map in which object instances are semantically annotated has a wide range of applications, including scene understanding, human-robot interactions, and simultaneous localization and mapping extensions. Although researchers are developing efficient and accurate systems, these methods still face a critical issue: Real-time processing, because the task requires a series of heavy processing components, e.g., camera pose estimation, 3D map reconstruction, and especially recognition. In this paper, we propose a novel object-oriented semantic mapping approach aiming at overcoming such issues by introducing highly accurate object-oriented semantic scene reconstruction in real time. For high efficiency, the proposed method employs a fast and scalable object detection algorithm for exploiting semantic information from the incoming frames. These outputs are integrated into geometric regions of the 3D map, which are carried by the geometric-based incremental segmentation method. The strategy of assigning class probabilities to each segmented region, not each element (e.g., surfels and voxels), notably reduces the computational cost, as well as the memory footprint. In addition to efficiency, by geometrically segmenting the 3D map first, clear boundaries between objects appear. We complementarily improve the geometric-based segmentation results beyond the geometric only to the semantic-aware representation. We validate the proposed method's accuracy and computational efficiency through experiments in a common office scene.

Access to Document (DOI)

Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation

Nakajima Y., Tateno K., Tombari F., Saito H.

IEEE International Conference on Intelligent Robots and Systems （IEEE International Conference on Intelligent Robots and Systems） 385 - 392 2018年12月

ISSN 9781538680940

© 2018 IEEE. We propose an efficient and scalable method for incrementally building a dense, semantically annotated 3D map in real-time. The proposed method assigns class probabilities to each region, not each element (e.g., surfel and voxel), of the 3D map which is built up through a robust SLAM framework and incrementally segmented with a geometric-based segmentation method. Differently from all other approaches, our method has a capability of running at over 30Hz while performing all processing components, including SLAM, segmentation, 2D recognition, and updating class probabilities of each segmentation label at every incoming frame, thanks to the high efficiency that characterizes the computationally intensive stages of our framework. By utilizing a specifically designed CNN to improve the frame-wise segmentation result, we can also achieve high accuracy. We validate our method on the NYUv2 dataset by comparing with the state of the art in terms of accuracy and computational efficiency, and by means of an analysis in terms of time and space complexity.

Access to Document (DOI)

Estimation of center of mass for sports scene using weighted visual hull

Kaichi T., Mori S., Saito H., Takahashi K., Mikami D., Isogawa M., Kimata H.

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops （IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops） 2018-June 1890 - 1896 2018年12月

ISSN 9781538661000

© 2018 IEEE. This paper presents a method to estimate the 3D position of a center of mass (CoM) of a human body from a set of multi-view images. As a well-known fact, in sports, collections of CoM are important for analyzing the athletes' performance. Most conventional studies in CoM estimation require installing a measuring system (e.g., a force plate or optical motion capture system) or attaching sensors to the athlete. While such systems reliably estimate CoM, casual settings are preferable for simplifying preparations. To address this issue, the proposed method takes a vision-based approach that does not require specialized hardware and wearable devices. Our method calculates subject's CoM using voxels with body parts dependent weighting. This individual voxel reconstruction and voxel-wise weighting reflects the differences in each body shape, and are expected to contribute to higher performance in analysis. The results using real data demonstrated the performance of the proposed method were compared to force plate data, and provided a 3D CoM visualization in a dynamic scene.

Access to Document (DOI)

FusionMLS: Highly dynamic 3D reconstruction with consumer-grade RGB-D cameras

Meerits S., Thomas D., Nozick V., Saito H.

Computational Visual Media （Computational Visual Media） 4 （ 4 ） 287 - 303 2018年12月

ISSN 20960433

© 2018, The Author(s). Multi-view dynamic three-dimensional reconstruction has typically required the use of custom shutter-synchronized camera rigs in order to capture scenes containing rapid movements or complex topology changes. In this paper, we demonstrate that multiple unsynchronized low-cost RGB-D cameras can be used for the same purpose. To alleviate issues caused by unsynchronized shutters, we propose a novel depth frame interpolation technique that allows synchronized data capture from highly dynamic 3D scenes. To manage the resulting huge number of input depth images, we also introduce an efficient moving least squares-based volumetric reconstruction method that generates triangle meshes of the scene. Our approach does not store the reconstruction volume in memory, making it memory-efficient and scalable to large scenes. Our implementation is completely GPU based and works in real time. The results shown herein, obtained with real data, demonstrate the effectiveness of our proposed method and its advantages compared to state-of-the-art approaches.

Access to Document (DOI)

Estimation of runners’ number of steps, stride length and speed transition from video of a 100-meter race

Yagi K., Hasegawa K., Sugiura Y., Saito H.

MMSports 2018 - Proceedings of the 1st International Workshop on Multimedia Content Analysis in Sports, Co-located with MM 2018 （MMSports 2018 - Proceedings of the 1st International Workshop on Multimedia Content Analysis in Sports, Co-located with MM 2018） 87 - 95 2018年10月

ISSN 9781450359818

© 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. The purpose of this study is sensing movements of 100-m runners from video that is publicly available, for example, Internet broadcasts. Normally, information that can be obtained from a video is limited to the number of steps and average stride length. However, our proposed method makes it possible to measure not only this information, but also time-scale information like every stride length and speed transition from the same input. Our proposed method can be divided into three steps. First, we generate a panoramic image of the 100-m track. By doing this, we can estimate where the runners are running in a frame at the 100-meter scale. Second, we detect whether the runner steps in the frame. For this process, we utilize the detected track lines and leg joint positions of runners. Finally, we project every steps to the overview image of the 100-m track to estimate the stride length at the 100-m scale. In the experiment part, we apply our method to various race videos. We evaluate the accuracy of our method via comparison with the data measured using typical methods. In addition, we evaluate the accuracy of estimation of the number of steps and show visualized runners’ steps and speed transitions.

Access to Document (DOI)

1 <sup>st</sup> ACM international workshop on multimedia content analysis in sports

Lienhart R., Moeslund T., Saito H.

MM 2018 - Proceedings of the 2018 ACM Multimedia Conference （MM 2018 - Proceedings of the 2018 ACM Multimedia Conference） 2125 - 2126 2018年10月

ISSN 9781450356657

© 2018 Association for Computing Machinery. The first ACM International Workshop on Multimedia Content Analysis in Sports (ACM MMSports'18) is held in Seoul, South Korea on October 26th, 2018 and is co-located with the ACM International Conference on Multimedia 2018 (ACM Multimedia 2018). The goal of this workshop is to bring together researchers and practitioners from academia and industry to address challenges and report progress in mining and content analysis of multimedia/multimodal data in sports. The combination of sports and modern technology offers a novel and intriguing field of research with promising approaches for visual broadcast augmentation, understanding, statistical analysis and evaluation, and sensor fusion. There is a lack of research communities focusing on the fusion of multiple modalities. We are helping to close this research gap with this first workshop of a serious workshops on multimedia content analysis in sports.

Access to Document (DOI)

Constant velocity 3D convolution

Sekikawa Y., Ishikawa K., Hara K., Yoshida Y., Suzuki K., Sato I., Saito H.

Proceedings - 2018 International Conference on 3D Vision, 3DV 2018 （Proceedings - 2018 International Conference on 3D Vision, 3DV 2018） 343 - 351 2018年10月

ISSN 9781538684252

© 2018 IEEE. We propose a novel three-dimensional (3D)-convolution method, cv3dconv, for detecting spatiotemporal features from videos. It reduces the number of sum-of-products of 3D convolution by thousands of times by assuming the constant moving velocity of the camera. We observed that a specific class of video sequences, such as those captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2D features in the temporal dimension. Our principal finding is that the 3D kernel, represented by the constant-velocity, can be decomposed into a convolution of a 2D kernel representing the shapes and a 3D kernel representing the velocity. We derived the efficient recursive algorithm for this class of 3D convolution which is exceptionally suited for sparse data, and this parameterized decomposed representation imposes a structured regularization along the temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of cv3dconv for the visual odometry estimation task using real event camera data captured in urban road scene.

Access to Document (DOI)

　前のページ - 次のページ　

斎藤 英雄

論文 - 斎藤 英雄

斎藤　英雄

論文 - 斎藤　英雄