Humanoid robot's visual imitation of 3-D motion of a human subject using neural-network-based inverse kinematics

Chih Lyang Hwang, Bo Lin Chen, Huei Ting Syu, Chao Kuei Wang, Mansour Karkoub

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

A sequence of 3-D motion images of a human facing an imitator [i.e., a humanoid robot (HR)] is captured using a stereo-vision system installed on the HR. After acquiring enough motion sequences via image ratio and background registration, the estimation of seven feature points (i.e., the head, two arm tips, two leg tips, and two elbows) of the human are obtained. Using five feature points (i.e., the head, two arm tips, and two leg tips), the extraction of key posture frames from the recorded videos is achieved. To ensure a stable imitation motion of the HR, the human motions of the lower body (LB) and the upper body (UB) are imitated separately. All 11 stable motions for the LB are classified by the motion direction, the motion state, and the support phase, whereas only the two arm tips are employed to imitate the motions of the UB. Due to the disadvantages of traditional inverse kinematics (IK), neural-network-based IK is developed to reduce the computation time and improve the accuracy of the time response of the system. Finally, the combined motions of the LB and the UB with suitable interpolation and time intervals are applied to imitate the 3-D motion of a human. The corresponding experimental results confirm the efficacy of the proposed technique.

Original languageEnglish
Article number6913510
Pages (from-to)685-696
Number of pages12
JournalIEEE Systems Journal
Volume10
Issue number2
DOIs
Publication statusPublished - 1 Jun 2016

Fingerprint

Inverse kinematics
Robots
Neural networks
Stereo vision
Interpolation

Keywords

  • Humanoid robot (HR)
  • Motion detection
  • Multilayer neural network (MLNN) modeling
  • Posture estimation
  • Stereo vision for 3-D localization
  • Visual imitation

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Cite this

Humanoid robot's visual imitation of 3-D motion of a human subject using neural-network-based inverse kinematics. / Hwang, Chih Lyang; Chen, Bo Lin; Syu, Huei Ting; Wang, Chao Kuei; Karkoub, Mansour.

In: IEEE Systems Journal, Vol. 10, No. 2, 6913510, 01.06.2016, p. 685-696.

Research output: Contribution to journalArticle

Hwang, Chih Lyang ; Chen, Bo Lin ; Syu, Huei Ting ; Wang, Chao Kuei ; Karkoub, Mansour. / Humanoid robot's visual imitation of 3-D motion of a human subject using neural-network-based inverse kinematics. In: IEEE Systems Journal. 2016 ; Vol. 10, No. 2. pp. 685-696.
@article{ccc23cbecad043948b6b0369708f3b32,
title = "Humanoid robot's visual imitation of 3-D motion of a human subject using neural-network-based inverse kinematics",
abstract = "A sequence of 3-D motion images of a human facing an imitator [i.e., a humanoid robot (HR)] is captured using a stereo-vision system installed on the HR. After acquiring enough motion sequences via image ratio and background registration, the estimation of seven feature points (i.e., the head, two arm tips, two leg tips, and two elbows) of the human are obtained. Using five feature points (i.e., the head, two arm tips, and two leg tips), the extraction of key posture frames from the recorded videos is achieved. To ensure a stable imitation motion of the HR, the human motions of the lower body (LB) and the upper body (UB) are imitated separately. All 11 stable motions for the LB are classified by the motion direction, the motion state, and the support phase, whereas only the two arm tips are employed to imitate the motions of the UB. Due to the disadvantages of traditional inverse kinematics (IK), neural-network-based IK is developed to reduce the computation time and improve the accuracy of the time response of the system. Finally, the combined motions of the LB and the UB with suitable interpolation and time intervals are applied to imitate the 3-D motion of a human. The corresponding experimental results confirm the efficacy of the proposed technique.",
keywords = "Humanoid robot (HR), Motion detection, Multilayer neural network (MLNN) modeling, Posture estimation, Stereo vision for 3-D localization, Visual imitation",
author = "Hwang, {Chih Lyang} and Chen, {Bo Lin} and Syu, {Huei Ting} and Wang, {Chao Kuei} and Mansour Karkoub",
year = "2016",
month = "6",
day = "1",
doi = "10.1109/JSYST.2014.2343236",
language = "English",
volume = "10",
pages = "685--696",
journal = "IEEE Systems Journal",
issn = "1932-8184",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

TY - JOUR

T1 - Humanoid robot's visual imitation of 3-D motion of a human subject using neural-network-based inverse kinematics

AU - Hwang, Chih Lyang

AU - Chen, Bo Lin

AU - Syu, Huei Ting

AU - Wang, Chao Kuei

AU - Karkoub, Mansour

PY - 2016/6/1

Y1 - 2016/6/1

N2 - A sequence of 3-D motion images of a human facing an imitator [i.e., a humanoid robot (HR)] is captured using a stereo-vision system installed on the HR. After acquiring enough motion sequences via image ratio and background registration, the estimation of seven feature points (i.e., the head, two arm tips, two leg tips, and two elbows) of the human are obtained. Using five feature points (i.e., the head, two arm tips, and two leg tips), the extraction of key posture frames from the recorded videos is achieved. To ensure a stable imitation motion of the HR, the human motions of the lower body (LB) and the upper body (UB) are imitated separately. All 11 stable motions for the LB are classified by the motion direction, the motion state, and the support phase, whereas only the two arm tips are employed to imitate the motions of the UB. Due to the disadvantages of traditional inverse kinematics (IK), neural-network-based IK is developed to reduce the computation time and improve the accuracy of the time response of the system. Finally, the combined motions of the LB and the UB with suitable interpolation and time intervals are applied to imitate the 3-D motion of a human. The corresponding experimental results confirm the efficacy of the proposed technique.

AB - A sequence of 3-D motion images of a human facing an imitator [i.e., a humanoid robot (HR)] is captured using a stereo-vision system installed on the HR. After acquiring enough motion sequences via image ratio and background registration, the estimation of seven feature points (i.e., the head, two arm tips, two leg tips, and two elbows) of the human are obtained. Using five feature points (i.e., the head, two arm tips, and two leg tips), the extraction of key posture frames from the recorded videos is achieved. To ensure a stable imitation motion of the HR, the human motions of the lower body (LB) and the upper body (UB) are imitated separately. All 11 stable motions for the LB are classified by the motion direction, the motion state, and the support phase, whereas only the two arm tips are employed to imitate the motions of the UB. Due to the disadvantages of traditional inverse kinematics (IK), neural-network-based IK is developed to reduce the computation time and improve the accuracy of the time response of the system. Finally, the combined motions of the LB and the UB with suitable interpolation and time intervals are applied to imitate the 3-D motion of a human. The corresponding experimental results confirm the efficacy of the proposed technique.

KW - Humanoid robot (HR)

KW - Motion detection

KW - Multilayer neural network (MLNN) modeling

KW - Posture estimation

KW - Stereo vision for 3-D localization

KW - Visual imitation

UR - http://www.scopus.com/inward/record.url?scp=84907855380&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84907855380&partnerID=8YFLogxK

U2 - 10.1109/JSYST.2014.2343236

DO - 10.1109/JSYST.2014.2343236

M3 - Article

VL - 10

SP - 685

EP - 696

JO - IEEE Systems Journal

JF - IEEE Systems Journal

SN - 1932-8184

IS - 2

M1 - 6913510

ER -