論文 - 斎藤　英雄

An omnidirectional vision system for bus safety surveillance

Hung Dao Huu, Saito Hideo, Yamamoto Keiichi, Sato Hiromitsu

Proceedings of 3rd Workshop on Activity Monitoring by Multiple Distributed Sensing (AMMDS), 12th edition of the IEEE AVSS conference 2015年08月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

　概要を見る

<p>Sudden brake seems to be unavoidable in driving practice to prevent from accidence. The sudden brake may often cause falling risks to on-board passengers in public transport and even threaten the lives of the senior. Thus, this paper presents an omnidirectional vision system to assist bus drivers in ensuring the safety for passengers. To this end, we configure an auxiliary camera in the boarding gate to recognize the age of passengers and an omnidirectional camera on the ceiling of the bus to detect and track the passengers. Their trajectories and positions are mapped into a bus layout, together with their age information transferred from the auxiliary camera. This bus layout is shown on a small display in front of bus drivers. Instead of looking to the mirror to guess and gather information of passengers, the driver can collect such necessary information for safe driving. We set up experiments on a real bus and demonstrate the system by preliminary results.</p>

• Access to Document (DOI)

Mutliple Object Tracking for End-to-End Transmission System of Free-Viewpoint Video

Shogo Miyata, Hideo Saito

Proceedings of International Display Manufacturing Conference (IDMC) & 3D Systems and Applications (3DSA) 2015    3DSA S5-5 2015年08月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

　概要を見る

In this paper, we propose a framework for end-to-end transmission system of free-viewpoint video. On soccer game taken multiple cameras, we adapt efficient multiple object tracking method to our system and generate free-viewpoint video represented billboard.

Polymorphic cataloguing and viewing system for using digital archives

Ishikawa Hiroyo, Saito Hideo, Miyashita Yamato, Kaneko Kunitake

Proceedings of 2014 International Conference on Virtual Systems & Multimedia (VSMM) （IEEE）     176 - 179 2015年06月

研究論文（国際会議プロシーディングス）, 共著

　概要を見る

<p>We propose a digital content viewing system which visualizes the polymorphic relationships among digital content interactively and uses it as a visual interface. The relationships are expressed by associating of two content and grouping and shown by directed graphs. On the graphs, nodes mean the content files or the groups, and edges mean correspondences or group members. The graphs are archived as relation catalogues separately from content files. We visualize the relations of content on a directed graph. To visualize the graphs, we use a method of locating nodes by horizontal directions and height. Nodes linked with a target node will be located on a horizontal circle and their height is decided by their relation catalogue. After deciding the node location, nodes and links are drawn using 3D graphics and animation. The visualization result called 'polymorphic view' is used as a visual interface for selecting content. Selecting content on the visualized graph will expand new links and new content nodes additionally. We can show another content by tracing graph links. It becomes easy to understand the relations between digital content, because we can change the direction and size of the directed graph. We developed a content viewer system (MoSaIC-II) using the polymorphic view as a visual interface and we can see the usability of the polymorphic view.</p>

• Access to Document (DOI)

3D Line Segment Based Model Generation by RGB-D Camera for Camera Pose Estimation

Yusuke Nakayama, Hideo Saito, Masayoshi Shimizu, Nobuyasu Yamaguchi

Computer Vision - ACCV 2014 Workshops Lecture Notes in Computer Science 9010   459 - 472 2015年04月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

　概要を見る

In this paper, we propose a novel method for generating 3D line segment based model from an image sequence taken with a RGB-D camera. Constructing 3D geometrical representation by 3D model is essential for model based camera pose estimation that can be performed by corresponding 2D features in images with 3D features of the captured scene. While point features are mostly used as such features for conventional camera pose estimation, we aim to use line segment features for improving the performance of the camera pose estimation. In this method, using RGB images and depth images of two continuous frames, 2D line segments from the current frame and 3D line segments from the previous frame are corresponded. The 2D-3D line segment correspondences provide camera pose of the current frame. All of 2D line segments are finally back-projected to the world coordinate based on the estimated camera pose for generating 3D line segment based model of the target scene. In experiments, we confirmed that the proposed method can successfully generate line segment based models, while 3D models based on the point features often fail to successfully represent the target scene.

• Access to Document (DOI)

ボールマークと軌跡に基づく卓球ボールの3次元回転計測

玉城　将，斎藤 英雄，吉田　和人

電子情報通信学会論文誌 D J98-D （ 4 ） 630 - 650 2015年04月

研究論文（学術雑誌）, 共著, 査読有り,  ISSN  1881-0225

　概要を見る

卓球においてボールの回転は戦術的に重要な要素と考えられている．しかし，従来の回転計測法は，表面全体にマークを追加したボールを高速度カメラを用いて撮影するものであり，撮影されるデータ量が膨大になる，あるいは試合と同じ条件下の回転による攻防を分析できない等の問題がある．ボールの運動モデルに基づき軌跡から回転を推定する方法も提案されているが，正確な回転計測は難しい．そこで本研究では，高速度カメラを用いることを前提とせず，且つ試合で用いられる卓球ボールを対象とできる回転計測法を提案する．提案手法ではまずボールのマークを検出し，検出されたマーク領域の重心から回転軸を算出する．高速度カメラを用いないことから，回転軸周りの回転方向及び隣接フレーム間の回転角速度は一意には求まらないが，回転角速度の全ての候補値に対応する軌跡を運動モデルに基づきシミュレートし，実測された軌跡に最も近い軌跡を探索することで最も真値に近い回転角速度を選択する．回転する卓球ボールを対象とした実画像実験により，本手法が実用的な精度で回転軸及び回転角速度を計測できることを示す．

• Access to Document (DOI)

カメラ位置姿勢推定のためのRGB-Dカメラによる3次元線分モデル生成法

中山祐介, 斎藤英雄, 清水雅芳, 山口伸康

映像情報メディア学会誌 69 （ 4 ） J148 - J159 2015年03月

研究論文（学術雑誌）, 査読有り,  ISSN  1342-6907

　概要を見る

本論文では，カメラ位置姿勢推定のための3次元線分モデルを，対象シーンをRGB-Dカメラで撮影するだけで簡易に生成する手法を提案する．従来の特徴点マッチングによるモデル生成は，特徴点が検出されない場合は失敗するという問題点があった．そのような場面でも線分は安定して検出されるため，3次元線分モデルの生成法が求められている．本提案手法では線分特徴記述子を用いた線分マッチングにより，簡易に，そして3次元情報が必要となるその場で3次元線分モデルを生成する．まず，入力されたRGB画像と距離画像から前フレームの3次元線分と現フレームの2次元線分の対応付けを行う．この2次元3次元線分対応から現フレームのカメラ位置姿勢を推定し，フレーム中の2次元線分を3次元空間上へ逆投影して3次元線分モデルとする．実験により，提案手法が特徴点マッチングではモデルが生成できないシーンでも正確なモデルを生成できることを確認した．

• Access to Document (DOI)

環境３Ｄモデルによるカメラトラッキングを用いた温度分布のAR表示システム

中川航, 松本一紀, ドゥソルビエフランソワ, 杉本麻樹, 斎藤英雄, 仙田修司, 柴田剛志, 池谷彰彦

映像情報メディア学会誌 69 （ 4 ） J160 - J168 2015年03月

研究論文（学術雑誌）, 共著,  ISSN  1342-6907

　概要を見る

近年、温度カメラは、機械設備・建築物の検査や，人間や動物の体温の検査等，多くの目的に利用されるようになってきた．本論文では，RGB-Dカメラで取得した3次元環境モデルを利用して、このモデル上に温度分布をレンダリング表示することにより，対象環境の温度分布を自由な視点から観察可能なシステムを提案する．本システムでは，３Ｄ環境モデルを利用したカメラ位置姿勢のトラッキング手法に基づき，事前に取得しておいた温度分布に対して，現在の温度変化を表示することも可能である．提案システムの有効性を検証するために，幾つかのシーンに対して本システムを利用して表示した温度分布や温度変化分布を示す．

• Access to Document (DOI)

Visualization of Temperature Change Using RGB-D Camera and Thermal Camera

Wataru Nakagawa, Kazuki Matsumoto, Francois de Sorbier, Maki Sugimoto, Hideo Saito, Shuji Senda, Takashi Shibata, Akihiko Iketani

Computer Vision - ECCV 2014 Workshops Lecture Notes in Computer Science Volume （Springer International Publishing）  8925   386 - 400 2015年03月

研究論文（国際会議プロシーディングス）, 共著, 査読有り,  ISSN  0302-9743

　概要を見る

In this paper, we present a system for visualizing temperature changes in a scene using an RGB-D camera coupled with a thermal camera. This system has applications in the context of maintenance of power equipments. We propose a two-stage approach made of with an offline and an online phases. During the first stage, after the calibration, we generate a 3D reconstruction of the scene with the color and the thermal data. We then apply the Viewpoint Generative Learning (VGL) method on the colored 3D model for creating a database of descriptors obtained from features robust to strong viewpoint changes. During the second online phase we compare the descriptors extracted from the current view against the ones in the database for estimating the pose of the camera. In this situation, we can display the current thermal data and compare it with the data saved during the offline phase.

• Access to Document (DOI)

Marker-less AR system based on line segment feature, The Engineering Reality of Virtual Reality 2015

Yusuke Nakayama, Hideo Saito, Masayoshi Shimizu, and Nobuyasu Yamaguchi

Proc. SPIE 9392, The Engineering Reality of Virtual Reality 2015    93920I 2015年03月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

　概要を見る

We propose a method of marker-less AR which uses line segment feature. Estimating camera poses is important
part of AR system. In most of conventional marker-less AR system, feature point matching between a model and
its image is required for the camera pose estimation. However, a sufficient number of correspondence points is not
always detected to estimate accurate camera poses. To solve this problem, we propose the use of line segments
feature that can possibly be detected even when only a few feature points can be detected. In this paper, we
propose a marker-less AR system that uses line segment features for camera pose estimation. In this system,
we propose a novel descriptor of the line segment feature for achieving fast computation of the camera pose
estimation. We also construct a database which contains a k-d tree of line feature and 3D line segment position
for nding 2D-3D line segment correspondences from input image and the database, so that we can estimate the
camera pose and perform AR. We demonstrated that our proposed method can estimate the camera pose and
provide robust marker-less AR in the situation where point matching method fails.

• Access to Document (DOI)

AR visualization of thermal 3D model by hand-held cameras

Kazuki Matsumoto, Wataru Nakagawa, Hideo Saito, Maki Sugimoto, Takashi Shibata, and Shoji Yachida

Proceedings of the Int. Conf. on Computer Vision Theory and Applications (VISAPP2015)    480 - 487 2015年03月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

Illumination estimation and relighting using an RGB-D camera

Yohei Ogura, Takuya Ikeda, Francois de Sorbier, and Hideo Saito

Proceedings of the Int. Conf. on Computer Vision Theory and Applications (VISAPP2015)    305 - 312 2015年03月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

　概要を見る

n this paper, we propose a relighting system combined with a
n illumination estimation method using RGB-
D camera. Relighting techniques can achieve the photometri
c registration of composite images. They often
need illumination environments of the scene which include a
target object and the background scene. Some
relighting methods obtain the illumination environments b
eforehand. In this case, they cannot be used under
the unknown dynamic illumination environment. Some on-lin
e illumination estimation methods need light
probes which can be invade the scene geometry. In our method,
the illumination environment is estimated
from pixel intensity, normal map and surface reflectance bas
ed on inverse rendering in on-line processing.
The normal map of the arbitrary object which is used in the ill
umination estimation part and the relighting part
is calculated from the denoised depth image on each frame. Re
lighting is achieved by calculating the ratio for
the estimated Illumination environment of the each scene. T
hus our implementation can be used for dynamic
illumination or a dynamic object.

Extraction of blood vessels in retinal images using resampling high-order background estimation

Paripurana Sukritta, Chiracharit Werapon, Chamnongthai Kosin, Saito Hideo

IEICE Transactions on Information and Systems E98D （ 3 ） 692 - 703 2015年03月

研究論文（学術雑誌）, 共著,  ISSN  0916-8532

　概要を見る

<p>In retinal blood vessel extraction through background removal, the vessels in a fundus image which appear in a higher illumination variance area are often missing after the background is removed. This is because the intensity values of the vessel and the background are nearly the same. Thus, the estimated background should be robust to changes of the illumination intensity. This paper proposes retinal blood vessel extraction using background estimation. The estimated background is calculated by using a weight surface fitting method with a high degree polynomial. Bright pixels are defined as unwanted data and are set as zero in a weight matrix. To fit a retinal surface with a higher degree polynomial, fundus images are reduced in size by different scaling parameters in order to reduce the processing time and complexity in calculation. The estimated background is then removed from the original image. The candidate vessel pixels are extracted from the image by using the local threshold values. To identify the true vessel region, the candidate vessel pixels are dilated from the candidate. After that, the active contour without edge method is applied. The experimental results show that the efficiency of the proposed method is higher than the conventional low-pass filter and the conventional surface fitting method. Moreover, rescaling an image down using the scaling parameter at 0.25 before background estimation provides as good a result as a non-rescaled image does. The correlation value between the non-rescaled image and the rescaled image is 0.99. The results of the proposed method in the sensitivity, the specificity, the accuracy, the area under the receiver operating characteristic (ROC) curve (AUC) and the processing time per image are 0.7994, 0.9717, 0.9543, 0.9676 and 1.8320 seconds for the DRIVE database respectively.</p>

• Access to Document (DOI)

RGB-Dカメラから取得した3次元点群の平面フィッティングを用いた実時間ノイズ低減と補間

松本一紀，フランソワ ドゥ ソルビエ，斎藤英雄

映像情報メディア学会誌 69 （ 3 ） J113 - J120 2015年02月

研究論文（学術雑誌）, 査読有り,  ISSN  1342-6907

　概要を見る

近年登場した距離センサーを搭載したデバイスの中でもその価格と汎用性ゆえに最も多く利用されているのがMicrosoftのKinectに代表されるRGB-Dカメラである．しかし，RGB-Dカメラから得られる３次元点群は多くのノイズを含んでおり，精密な距離情報の取得が求められるアプリケーションでの利用は依然として困難である．そこで本論文ではRGB-Dカメラから得られた３次元点群の観測ノイズの低減と補間を実時間で行う手法を提案する．本手法はRGB-Dカメラから得られた３次元点群からSuperpixel Segmentaiotnとラベリングを併用して平面で構成される小領域群を検出し，領域毎に平面方程式を推定し３次元点群をフィッティングさせることにより，もとの距離画像に含まれるノイズの低減と補間を行うものである．本手法では検出された領域内の点群の法線分布を算出し，その領域が平面の場合平面フィッティングを適用し曲面の場合Joint Bilateral Filterを適用することで，曲面分布した点群を保持しながら平面分布した点群のみを平面フィッティングすることができる．また，領域分割と平面フィッティングはすべて局所領域ごとに並列化できる手法を用いているため，本手法の実装はすべてGPUで行い実時間での処理が可能である．従来の手法と比べて，本手法はRGB-Dカメラから取得した点群に対して，曲面領域を保持したまま平面領域を平滑化することでノイズ低減と補間を行うことができる．

• Access to Document (DOI)

手持ちカメラを用いた走者ストロボ画像の自動生成

長谷川邦洋，斎藤 英雄

精密工学会誌 81 （ 2 ） 156 - 163 2015年02月

研究論文（学術雑誌）, 共著, 査読有り

　概要を見る

This paper discusses the method for the automatically generating stroboscopic image from a video image using a handheld camera comprehensively. The stroboscopic image can be used in order to measure stride length and speed of the runner based on the authors' previously proposed method, in which it is shown that measuring stride length and speed of the runner from the stroboscopic image is theoretically possible. The proposed automatic stroboscopic image generating method has two steps. The first step is generating a stitched background image from input video image. In the second step, runner images extracted from arbitrary frames are overlapped onto the stitched background image. Thus, this method can generate a stroboscopic image including the runners in arbitrary frames without overwriting them in some frames. These processes can be executed automatically, and these are noteworthy features. In the experiments under various environment to show the effectiveness of this method, it was confirmed that could generate a stroboscopic images well in respectively environment. It is also demonstrated that stride length and speed of the runner captured into the generated stroboscopic image can be measured with satisfactory accuracy by manually selected landing points in the stroboscopic image.

• Access to Document (DOI)

Plane fitting and depth variance based upsampling for noisy depth map from 3D-ToF cameras in real-time

Matsumoto Kazuki, De Sorbier Francois, Saito Hideo

Proceedings of the International Conference on Pattern Recognition Applications and Methods (ICPRAM-2015) （SCITEPRESS (Science and Technology Publications, Lda.)）     150 - 157 2015年01月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

　概要を見る

<p>Recent advances of ToF depth sensor devices enables us to easily retrieve scene depth data with high frame rates. However, the resolution of the depth map captured from these devices is much lower than that of color images and the depth data suffers from the optical noise effects. In this paper, we propose an efficient algorithm that upsamples depth map captured by ToF depth cameras and reduces noise. The upsampling is carried out by applying plane based interpolation to the groups of points similar to planar structures and depth variance based joint bilateral upsampling to curved or bumpy surface points. For dividing the depth map into piecewise planar areas, we apply superpixel segmentation and graph component labeling. In order to distinguish planar areas and curved areas, we evaluate the reliability of detected plane structures. Compared with other state-of-the-art algorithms, our method is observed to produce an upsampled depth map that is smoothed and closer to the ground truth depth map both visually and numerically. Since the algorithm is parallelizable, it can work in real-time by utilizing highly parallel processing capabilities of modern commodity GPUs.</p>

• Access to Document (DOI)

Real Time Detection and Tracking of Printed Maps Based on Road Structure

斎藤 英雄

ITE Transactions on Media Technology and Applications 3 （ 1 ） 85 - 94 2015年01月

研究論文（学術雑誌）, 共著, 査読有り

　概要を見る

This paper presents a method to detect and track a paper map and overlay virtual data over it. First, we take a picture of the map and identify the road colors using our road area extraction method. Using the color information we can then extract in real time the roads from a video and use the position of the intersections to identify the map from a database without any further user input. The main issue is to get reliable intersection positions in real time even though the map can contain a lot of overlapping data. Finally, using the location of the intersections we can track and augment the map with virtual information. Our method enables the real time matching, 3D tracking and augmentation of a paper map based on a database that can contain several thousands of intersections coordinates.

• Access to Document (DOI)

REAL-TIME ENHANCEMENT OF RGB-D POINT CLOUDS USING PIECEWISE PLANE FITTING

Kazuki Matsumoto, Francois de Sorbier and Hideo Saito

Proceedings of the 2014 5th European Workshop on Visual Information Processing (EUVIP2014) （IEEE）  2014年12月

研究論文（国際会議プロシーディングス）, 共著, 査読有り

• Access to Document (DOI)

非同期カメラを用いたボールの3次元軌跡復元法

玉城将，斎藤 英雄

精密工学会誌 （精密工学会）  80 （ 12 ） 1157 - 1165 2014年12月

研究論文（学術雑誌）, 共著, 査読有り,  ISSN  0912-0289

　概要を見る

Ball trajectory is an important performance indicator in ball game. However, Measuring 3D trajectories is often unusable because synchronized cameras and control points inside the playing field are generally required. A method that reconstructs 3D trajectory of ball with unsynchronized cameras is proposed in this paper. Proposed method consists of ball detection, camera calibration, and trajectory reconstruction. At first, ball candidates are detected based on the appearance. For the camera calibration, highly probable balls are extracted from the candidates based on the motion. Imaged ball trajectories are then reconstructed. Next, fundamental matrix is computed from the corresponding points, which are estimated with the ball trajectories, and roughly estimated temporal offset, which is estimated manually. The estimated fundamental matrix is assumed to be inaccurate due to the inaccuracy of the temporal offset. The key feature of this method is to accurately estimate temporal offset and fundamental matrix by optimizing them iteratively based on the error of fundamental matrix. After the calibration, balls are extracted robustly with geometrical and temporal relationship between cameras. A ball trajectory is finally reconstructed as connected trajectories which separated at collisions. It is experimentally demonstrated that the proposed method can calibrate cameras precisely and reconstruct accurate ball trajectories.

• Access to Document (DOI)

RGB-Dカメラを用いた実時間リライティング映像生成システムの開発

池田拓也, 小倉洋平, ドゥ ソルビエ フランソワ, 斎藤 英雄

映像情報メディア学会誌 （映像情報メディア学会）  68 （ 12 ） J558 - J568 2014年11月

研究論文（学術雑誌）, 査読有り,  ISSN  1342-6907

　概要を見る

リライティングとは，実画像から，それを撮影した照明環境を別の正面環境で撮影したかのような画像を合成する処理のことであり，その実現には，対象物体の3次元形状（法線分布），反射特性，光源環境が必要である．本論文では，RGB-Dカメラを用いて実時間リライティングを実現可能なシステムを提案する．本システムでは，正確な物体領域抽出のためにノイズを含む距離画像の修正を行う．領域抽出を行った後，対象物体の法線分布を推定する．光源環境は，観測された鏡面球から取得し，任意の環境に対してリライティングを行う．また，実時間処理を実現するために，GPUを用いた並列処理を行う．我々の実装では，約10fpsの速度でリライティングが可能である．提案システムの有効性を確認するために行った実験では，対象物体に対して事前に撮影した動的光源環境へリライティングを行い，提案システムの性能を評価した．

• Access to Document (DOI)

Auto-Generation of Runner's Stroboscopic Image and Measuring Landing Points Using a Handheld Camera

K. Hasegawa and H. Saito

Proc. 2014 Irish Machine Vision and Image Processing (IMVIP2014)    169 - 174 2014年08月

研究論文（国際会議プロシーディングス）, 共著, 査読有り