摘要

We propose the deep hierarchical network (DHN) for the quantitative analysis of facial palsy. Facial palsy, also known as Bell's palsy, is the most common type of facial nerve palsy that results in the loss of muscle control in the affected facial regions. Typical symptoms include facial deformity and facial expression dysfunction. To the best of our best knowledge, all approaches for the automatic detection of facial palsy consider hand-crafted features. This paper reports the first deep-learning-based approach developed for the real-time quantitative analysis of facial palsy. The proposed DHN consists of three component networks: The first detects the subject's face, the second detects the facial landmarks and line segments on the detected face, and the third detects the local palsy regions. The first component network is built on the YOLO2 detector. The second component network is developed on a fused network architecture that incorporates a line segment learning network for locating the facial landmarks and line segments. The third component network is developed on an object detection network with the line-segment-embedded input that combines the landmarked region and the line segments detected by the second component network. The novelties of this research include: 1) the modification of a state-of-the-art edge detector for extracting the facial line segments; 2) the embedding of the line segment learning for the detection of facial landmarks and local palsy regions; 3) the quantitative description of the facial palsy syndrome intensity; and 4) the release of the first clinically labeled database, the YouTube Facial Palsy (YFP) database. The making of the YFP database solves the issue that previous methods were all evaluated on proprietary databases, making the comparison of different methods extremely difficult. The YFP database includes 32 videos of 21 patients collected from YouTube and labeled by clinic specialists. To enhance the robustness against facial expression variations, we include the CK+ facial expression database in the training. We show that the proposed DHN not only just detects the local palsy regions but also captures the intensity of the facial palsy syndrome over time, enabling the quantitative description of the syndrome. The experiments show that the proposed approach offers an accurate and efficient real-time solution for facial palsy analysis.
原文英語
文章編號8558494
頁(從 - 到)4833-4842
頁數10
期刊IEEE Access
7
DOIs
出版狀態已發佈 - 一月 1 2019

指紋

Network components
Chemical analysis
Detectors
Network architecture
Muscle
Experiments

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

引用此文

Deep hierarchical network with line segment learning for quantitative analysis of facial palsy. / Hsu, Gee Sern Jison; Kang, Jiunn Horng; Huang, Wen Fong.

於: IEEE Access, 卷 7, 8558494, 01.01.2019, p. 4833-4842.

研究成果: 雜誌貢獻文章

@article{575d4279e38c4815ab899fac87480ef7,
title = "Deep hierarchical network with line segment learning for quantitative analysis of facial palsy",
abstract = "We propose the deep hierarchical network (DHN) for the quantitative analysis of facial palsy. Facial palsy, also known as Bell's palsy, is the most common type of facial nerve palsy that results in the loss of muscle control in the affected facial regions. Typical symptoms include facial deformity and facial expression dysfunction. To the best of our best knowledge, all approaches for the automatic detection of facial palsy consider hand-crafted features. This paper reports the first deep-learning-based approach developed for the real-time quantitative analysis of facial palsy. The proposed DHN consists of three component networks: The first detects the subject's face, the second detects the facial landmarks and line segments on the detected face, and the third detects the local palsy regions. The first component network is built on the YOLO2 detector. The second component network is developed on a fused network architecture that incorporates a line segment learning network for locating the facial landmarks and line segments. The third component network is developed on an object detection network with the line-segment-embedded input that combines the landmarked region and the line segments detected by the second component network. The novelties of this research include: 1) the modification of a state-of-the-art edge detector for extracting the facial line segments; 2) the embedding of the line segment learning for the detection of facial landmarks and local palsy regions; 3) the quantitative description of the facial palsy syndrome intensity; and 4) the release of the first clinically labeled database, the YouTube Facial Palsy (YFP) database. The making of the YFP database solves the issue that previous methods were all evaluated on proprietary databases, making the comparison of different methods extremely difficult. The YFP database includes 32 videos of 21 patients collected from YouTube and labeled by clinic specialists. To enhance the robustness against facial expression variations, we include the CK+ facial expression database in the training. We show that the proposed DHN not only just detects the local palsy regions but also captures the intensity of the facial palsy syndrome over time, enabling the quantitative description of the syndrome. The experiments show that the proposed approach offers an accurate and efficient real-time solution for facial palsy analysis.",
keywords = "bell palsy, face alignment, face recognition, Facial palsy, medical image diagnosis",
author = "Hsu, {Gee Sern Jison} and Kang, {Jiunn Horng} and Huang, {Wen Fong}",
year = "2019",
month = "1",
day = "1",
doi = "10.1109/ACCESS.2018.2884969",
language = "English",
volume = "7",
pages = "4833--4842",
journal = "IEEE Access",
issn = "2169-3536",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Deep hierarchical network with line segment learning for quantitative analysis of facial palsy

AU - Hsu, Gee Sern Jison

AU - Kang, Jiunn Horng

AU - Huang, Wen Fong

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We propose the deep hierarchical network (DHN) for the quantitative analysis of facial palsy. Facial palsy, also known as Bell's palsy, is the most common type of facial nerve palsy that results in the loss of muscle control in the affected facial regions. Typical symptoms include facial deformity and facial expression dysfunction. To the best of our best knowledge, all approaches for the automatic detection of facial palsy consider hand-crafted features. This paper reports the first deep-learning-based approach developed for the real-time quantitative analysis of facial palsy. The proposed DHN consists of three component networks: The first detects the subject's face, the second detects the facial landmarks and line segments on the detected face, and the third detects the local palsy regions. The first component network is built on the YOLO2 detector. The second component network is developed on a fused network architecture that incorporates a line segment learning network for locating the facial landmarks and line segments. The third component network is developed on an object detection network with the line-segment-embedded input that combines the landmarked region and the line segments detected by the second component network. The novelties of this research include: 1) the modification of a state-of-the-art edge detector for extracting the facial line segments; 2) the embedding of the line segment learning for the detection of facial landmarks and local palsy regions; 3) the quantitative description of the facial palsy syndrome intensity; and 4) the release of the first clinically labeled database, the YouTube Facial Palsy (YFP) database. The making of the YFP database solves the issue that previous methods were all evaluated on proprietary databases, making the comparison of different methods extremely difficult. The YFP database includes 32 videos of 21 patients collected from YouTube and labeled by clinic specialists. To enhance the robustness against facial expression variations, we include the CK+ facial expression database in the training. We show that the proposed DHN not only just detects the local palsy regions but also captures the intensity of the facial palsy syndrome over time, enabling the quantitative description of the syndrome. The experiments show that the proposed approach offers an accurate and efficient real-time solution for facial palsy analysis.

AB - We propose the deep hierarchical network (DHN) for the quantitative analysis of facial palsy. Facial palsy, also known as Bell's palsy, is the most common type of facial nerve palsy that results in the loss of muscle control in the affected facial regions. Typical symptoms include facial deformity and facial expression dysfunction. To the best of our best knowledge, all approaches for the automatic detection of facial palsy consider hand-crafted features. This paper reports the first deep-learning-based approach developed for the real-time quantitative analysis of facial palsy. The proposed DHN consists of three component networks: The first detects the subject's face, the second detects the facial landmarks and line segments on the detected face, and the third detects the local palsy regions. The first component network is built on the YOLO2 detector. The second component network is developed on a fused network architecture that incorporates a line segment learning network for locating the facial landmarks and line segments. The third component network is developed on an object detection network with the line-segment-embedded input that combines the landmarked region and the line segments detected by the second component network. The novelties of this research include: 1) the modification of a state-of-the-art edge detector for extracting the facial line segments; 2) the embedding of the line segment learning for the detection of facial landmarks and local palsy regions; 3) the quantitative description of the facial palsy syndrome intensity; and 4) the release of the first clinically labeled database, the YouTube Facial Palsy (YFP) database. The making of the YFP database solves the issue that previous methods were all evaluated on proprietary databases, making the comparison of different methods extremely difficult. The YFP database includes 32 videos of 21 patients collected from YouTube and labeled by clinic specialists. To enhance the robustness against facial expression variations, we include the CK+ facial expression database in the training. We show that the proposed DHN not only just detects the local palsy regions but also captures the intensity of the facial palsy syndrome over time, enabling the quantitative description of the syndrome. The experiments show that the proposed approach offers an accurate and efficient real-time solution for facial palsy analysis.

KW - bell palsy

KW - face alignment

KW - face recognition

KW - Facial palsy

KW - medical image diagnosis

UR - http://www.scopus.com/inward/record.url?scp=85058105069&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058105069&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2018.2884969

DO - 10.1109/ACCESS.2018.2884969

M3 - Article

AN - SCOPUS:85058105069

VL - 7

SP - 4833

EP - 4842

JO - IEEE Access

JF - IEEE Access

SN - 2169-3536

M1 - 8558494

ER -