NUST Institutions Library Catalogue

Malware analysis is a critical component of modern cybersecurity, enabling the identification,
understanding, and mitigation of malicious software. This dissertation investigates
the evolution of machine learning–based approaches for large-scale, behavior-driven
malware classification, with the objective of improving detection accuracy, scalability,
and robustness across diverse computing platforms.
Initially, this research presents a comprehensive review of existing machine learning–
based malware detection and classification techniques, highlighting their strengths
and limitations. Static analysis–based approaches, which extract features directly from
executable files without program execution, have gained widespread adoption due to
their low computational cost and ease of deployment. However, these methods are highly
susceptible to obfuscation, packing, and evasion techniques. Dynamic analysis–based approaches,
which examine malware behavior during execution in controlled environments,
have been increasingly explored. Although dynamic analysis offers improved resilience
against static evasion techniques, many existing approaches rely on a limited set of
behavioral features, such as API call sequences. This narrow feature representation
limits their ability to capture complex and evolving malware behaviors, thereby reducing
classification performance and generalization in real-world, large-scale environments.
Subsequently, this dissertation proposes a novel malware classification framework
based on a stack ensemble machine learning model that leverages large-scale IoT malware
behavioral data. The proposed framework integrates multi-dimensional dynamic
features, including memory dump characteristics, file system activities, network interactions,
process behaviors, command executions, URL communications, and memory access patterns. A key contribution of this work is its scalability and computational
efficiency, which make it well-suited for deployment in resource-constrained IoT environments
while achieving high classification accuracy.