Agglomerative algorithm to discover semantics from unstructured big data

I-Jen Chiang

doi:10.1109/BigData.2015.7363920

Agglomerative algorithm to discover semantics from unstructured big data

I-Jen Chiang

Taipei Medical University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The paper presents a graph model and an agglomerative algorithm for text document clustering. Given a set of documents, the associations among frequently co-occurring terms in any of the documents naturally form a graph, which can be decomposed into connected components at various levels. Each connected component represents a concept in the collection. These concepts can categorize documents into different semantic classes. The experiments on three different data sets from news, Web, and medical literatures have shown our algorithm is significantly better than traditional clustering algorithms, such as k-means, principal direction division partitioning, AutoClass and hierarchical clustering.

Original language	English
Title of host publication	Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1556-1563
Number of pages	8
ISBN (Print)	9781479999255
DOIs	https://doi.org/10.1109/BigData.2015.7363920
Publication status	Published - Dec 22 2015
Event	3rd IEEE International Conference on Big Data, IEEE Big Data 2015 - Santa Clara, United States Duration: Oct 29 2015 → Nov 1 2015

Other

Other	3rd IEEE International Conference on Big Data, IEEE Big Data 2015
Country	United States
City	Santa Clara
Period	10/29/15 → 11/1/15

Fingerprint

Keywords

agglomerative document categorization/clustering
association rules
hierarchical clustering
hypergraph

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications
Information Systems
Software

Cite this

Chiang, I-J. (2015). Agglomerative algorithm to discover semantics from unstructured big data. In Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015 (pp. 1556-1563). [7363920] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2015.7363920

Chiang, I-J 2015, Agglomerative algorithm to discover semantics from unstructured big data. in Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015., 7363920, Institute of Electrical and Electronics Engineers Inc., pp. 1556-1563, 3rd IEEE International Conference on Big Data, IEEE Big Data 2015, Santa Clara, United States, 10/29/15. https://doi.org/10.1109/BigData.2015.7363920

@inproceedings{9e31510de1b04a80879d4938402f23fa,

title = "Agglomerative algorithm to discover semantics from unstructured big data",

abstract = "The paper presents a graph model and an agglomerative algorithm for text document clustering. Given a set of documents, the associations among frequently co-occurring terms in any of the documents naturally form a graph, which can be decomposed into connected components at various levels. Each connected component represents a concept in the collection. These concepts can categorize documents into different semantic classes. The experiments on three different data sets from news, Web, and medical literatures have shown our algorithm is significantly better than traditional clustering algorithms, such as k-means, principal direction division partitioning, AutoClass and hierarchical clustering.",

keywords = "agglomerative document categorization/clustering, association rules, hierarchical clustering, hypergraph",

author = "I-Jen Chiang",

year = "2015",

month = dec

day = "22",

doi = "10.1109/BigData.2015.7363920",

language = "English",

isbn = "9781479999255",

pages = "1556--1563",

booktitle = "Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

note = "3rd IEEE International Conference on Big Data, IEEE Big Data 2015 ; Conference date: 29-10-2015 Through 01-11-2015",

}

TY - GEN

T1 - Agglomerative algorithm to discover semantics from unstructured big data

AU - Chiang, I-Jen

PY - 2015/12/22

Y1 - 2015/12/22

N2 - The paper presents a graph model and an agglomerative algorithm for text document clustering. Given a set of documents, the associations among frequently co-occurring terms in any of the documents naturally form a graph, which can be decomposed into connected components at various levels. Each connected component represents a concept in the collection. These concepts can categorize documents into different semantic classes. The experiments on three different data sets from news, Web, and medical literatures have shown our algorithm is significantly better than traditional clustering algorithms, such as k-means, principal direction division partitioning, AutoClass and hierarchical clustering.

AB - The paper presents a graph model and an agglomerative algorithm for text document clustering. Given a set of documents, the associations among frequently co-occurring terms in any of the documents naturally form a graph, which can be decomposed into connected components at various levels. Each connected component represents a concept in the collection. These concepts can categorize documents into different semantic classes. The experiments on three different data sets from news, Web, and medical literatures have shown our algorithm is significantly better than traditional clustering algorithms, such as k-means, principal direction division partitioning, AutoClass and hierarchical clustering.

KW - agglomerative document categorization/clustering

KW - association rules

KW - hierarchical clustering

KW - hypergraph

UR - http://www.scopus.com/inward/record.url?scp=84963745319&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84963745319&partnerID=8YFLogxK

U2 - 10.1109/BigData.2015.7363920

DO - 10.1109/BigData.2015.7363920

M3 - Conference contribution

AN - SCOPUS:84963745319

SN - 9781479999255

SP - 1556

EP - 1563

BT - Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 3rd IEEE International Conference on Big Data, IEEE Big Data 2015

Y2 - 29 October 2015 through 1 November 2015

ER -

Access to Document

10.1109/BigData.2015.7363920