Research Article
Filtering Events using Clustering in Heterogeneous Security Logs

H. Asif-Iqbal, Nur Izura Udzir, Ramlan Mahmod and Abdul Azim Abd.Ghani

Information Technology Journal, 2011, 10(4), 798-806.

Abstract

Log files are rich sources of information exhibiting the actions performed during the usage of a computer system in our daily work. In this study we concentrate on parsing/isolating logs from different sources and then clustering the logs using data mining tool (Weka) to filter the unwanted entries in the logs which will greatly help in correlating the events from different logs. Unfortunately parsing heterogeneous logs to extract the attribute values becomes tedious, since every type of log is stored in a proprietary format. We propose a framework that has the ability to parse and isolate a variety of logs, followed by clustering the logs to identify and remove unneeded entries. Experiments involving a range of logs, reveals the fact that clustering has the capacity to group log entries with a higher degree of accuracy, thereby assisting to identify correctly the entries to be removed.

ASCI-ID: 28-1239

Cited References Fulltext

Cited By

Architectures and Protocols for Secure Information Technology Infrastructures

Advances in Information Security, Privacy, and Ethics, 2014, (), 184. DOI: 10.4018/978-1-4666-4514-1.ch007

2017 13th IEEE Conference on Automation Science and Engineering (CASE)

2017 13th IEEE Conference on Automation Science and Engineering (CASE), 2017, (), 1136. DOI: 10.1109/COASE.2017.8256257

An unsupervised heterogeneous log-based framework for anomaly detection

TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES, 2016, 24(), 1117. DOI: 10.3906/elk-1302-19

An Integrated Method for Anomaly Detection From Massive System Logs

IEEE Access, 2018, 6(), 30602. DOI: 10.1109/ACCESS.2018.2843336

2020 16th IEEE International Colloquium on Signal Processing & Its Applications (CSPA)

2020 16th IEEE International Colloquium on Signal Processing & Its Applications (CSPA), 2020, (), 18. DOI: 10.1109/CSPA48992.2020.9068720

2021 IEEE 11th IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE)

2021 IEEE 11th IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), 2021, (), 82. DOI: 10.1109/ISCAIE51753.2021.9431794

A refined filter for UHAD to improve anomaly detection

Security and Communication Networks, 2016, 9(14), 2434. DOI: 10.1002/sec.1514

A Survey on Forensics and Compliance Auditing for Critical Infrastructure Protection

IEEE Access, 2024, 12(), 2409. DOI: 10.1109/ACCESS.2023.3348552

2022 IEEE 10th Conference on Systems, Process & Control (ICSPC)

2022 IEEE 10th Conference on Systems, Process & Control (ICSPC), 2022, (), 176. DOI: 10.1109/ICSPC55597.2022.10001797

Combining K-Means and XGBoost Models for Anomaly Detection Using Log Datasets

Electronics, 2020, 9(7), 1164. DOI: 10.3390/electronics9071164

Utilizing Renewable Energy, Technology, and Education for Industry 5.0

Advances in Chemical and Materials Engineering, 2024, (), 292. DOI: 10.4018/979-8-3693-2814-9.ch013

Proceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis

Proceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, 2023, (), 581. DOI: 10.1145/3624062.3624128

RETRACTED: An enhanced network intrusion detection system for malicious crawler detection and security event correlations in ubiquitous banking infrastructure

International Journal of Pervasive Computing and Communications, 2022, 18(1), 59. DOI: 10.1108/IJPCC-04-2021-0102