I2S Masters/ Doctoral Theses

Person re-identification (Re-ID) and multi-object tracking in unconstrained surveillance environments pose significant challenges within the field of computer vision. These complexities stem mainly from occlusion, variability in appearance, and identity switching across various camera views. This research outlines a comprehensive and innovative agenda aimed at tackling these issues, employing a series of increasingly advanced deep learning architectures, culminating in a groundbreaking occlusion-aware Vision Transformer framework.

At the heart of this work is the introduction of Deep SORT with Multiple Inputs (Deep SORT-MI), a cutting-edge real-time Re-ID system featuring a dual-metric association strategy. This strategy adeptly combines Mahalanobis distance for motion-based tracking with cosine similarity for appearance-based re-identification. As a result, this method significantly decreases identity switching compared to the baseline SORT algorithm on the MOT-16 benchmark, thereby establishing a robust foundation for metric learning in subsequent research.

Expanding on this foundation, a novel pose-estimation framework integrates 2D skeletal keypoint features extracted via OpenPose directly into the association pipeline. By capturing the spatial relationships among body joints along with appearance features, this system enhances robustness against posture variations and partial occlusion. Consequently, it achieves substantial reductions in false positives and identity switches compared to earlier methods, showcasing its practical viability.

Furthermore, a Diverse Detector Integration (DDI) study meticulously assessed the influence of detector choices—including YOLO v4, Faster R-CNN, MobileNet SSD v2, and Deep SORT—on the efficacy of metric learning-based tracking. The results reveal that YOLO v4 consistently delivers exceptional tracking accuracy on both the MOT-16 and MOT-17 datasets, establishing its superiority in this competitive landscape.

In conclusion, this body of research notably advances occlusion-aware person Re-ID by illustrating a clear progression from metric learning to pose-guided feature extraction and ultimately to transformer-based global attention modeling. The findings underscore that lightweight, meticulously parameterized Vision Transformers can achieve impressive generalization for occlusion detection, even under constrained data scenarios. This opens up exciting prospects for integrated detection, localization, and re-identification in real-world surveillance systems, promising to enhance their effectiveness and reliability.

Runtime estimation plays a structural role in reservation-based scheduling for High Performance Computing (HPC) systems, where predicted walltimes directly influence reservation timing, backfilling feasibility, and overall queue dynamics. This raises a fundamental question of whether improved runtime prediction accuracy necessarily translates into improved scheduling performance. In this work, we conduct an empirical study of runtime estimation under EASY Backfilling using an application-driven workload consisting of MRI-based brain segmentation jobs. Despite identical configurations and uniform metadata, runtimes exhibit substantial variability driven by intrinsic input structure. To capture this variability, we develop a feature-driven machine learning (ML) framework that extracts region-wise features from MRI volumes to predict job runtimes without relying on historical execution traces or scheduling metadata. We integrate these ML-derived predictions into an EASY Backfilling scheduler implemented in the Batsim simulation framework. Our results show that regression accuracy alone does not determine scheduling performance. Instead, scheduling performance depends strongly on estimation bias and its effect on reservation timing and runtime exceedances. In particular, mild multiplicative calibration of ML-based runtime estimates stabilizes scheduler behavior and yields consistently competitive performance across workload and system configurations. Comparable performance can also be observed with certain levels of uniform overestimation; however, calibrated ML predictions provide a systematic mechanism to control estimation bias without relying on arbitrary static inflation. In contrast, underestimation consistently leads to severe performance degradation and cascading job terminations. These findings highlight runtime estimation as a structural control input in backfilling-based HPC scheduling and demonstrate the importance of evaluating prediction models jointly with scheduling dynamics rather than through regression metrics alone.

Modern intelligent systems increasingly rely on continuous sensor data streams for perception, decision-making, and control, making sensors a critical yet underexplored attack surface. While prior research has demonstrated the feasibility of sensor-based attacks, recent advances in mobile operating systems and machine learning-based defenses have significantly reduced their practicality, rendering them more detectable, resource-intensive, and constrained by evolving permission and context-aware security models.

This dissertation revisits sensor exploitation under these modern constraints and develops a unified, cross-layer perspective that improves both practicality and stealth of sensor-enabled attacks. We identify three fundamental challenges: (i) the difficulty of reliably manipulating physical sensor signals in noisy, real-world environments; (ii) the effectiveness of context-aware defenses in detecting anomalous sensor behavior on mobile devices, and (iii) the lack of lightweight coordination for practical sensor-based side- and covert-channels.

To address the first challenge, we propose a physical-domain attack framework that integrates signal modeling, simulation-guided attack synthesis, and real-time adaptive targeting, enabling robust adversarial perturbations with high attack success rates even under environmental uncertainty. As a case study, we demonstrate an infrared laser-based adversarial example attack against face recognition systems, which achieves consistently high success rates across diverse conditions with practical execution overhead.

To improve attack stealth against context-aware defenses, we introduce an auto-contextualization mechanism that synchronizes malicious sensor actuation with legitimate application activity. By aligning injected signals with both statistical patterns and semantic context of benign behavior, the approach renders attacks indistinguishable from normal system operations and benign sensor usage. We validate this design using three Android logic bombs, showing that auto-contextualized triggers can evade both rule-based and learning-based detection mechanisms.

Finally, we extend sensor exploitation beyond the traditional attack-channel plane by introducing a lightweight control-plane protocol embedded within sensor data streams. This protocol encodes control signals directly into sensor observations and leverages simple signal-processing primitives to coordinate multi-stage attacks without relying on privileged APls or explicit inter-process communication. The resulting design enables low-overhead, stealthy coordination of cross-device side- and covert-channels.

Together, these contributions establish a new paradigm for sensor exploitation that spans physical, contextual, and control-plane dimensions. By bridging these layers, this dissertation demonstrates that sensor-based attacks remain not only feasible but also practical and stealthy in modern computer systems.

Metagenomic sequencing has become a central paradigm for studying complex microbial communities and their interactions with the host, with emerging applications in clinical prediction and disease modeling. In this work, we first investigate two representative application scenarios: predicting immune checkpoint inhibitor response in non-small cell lung cancer using gut microbial signatures, and characterizing host–microbiome interactions in neonatal systems. The proposed reference-free neural network captures both compositional and functional signals without reliance on reference genomes, while the neonatal study demonstrates how environmental and genetic factors reshape microbial communities and how probiotic intervention can mitigate pathogen-induced immune activation.

These studies highlight both the promise and the inherent difficulty of metagenomic analysis: transforming raw sequencing data into clinically actionable insights remains an algorithmically fragmented and computationally intensive process. This challenge arises from two key limitations: the lack of a unified algorithmic foundation for sequence alignment and the absence of systematic approaches for selecting and organizing analytical tools. Motivated by these challenges, we present a unified computational framework for metagenomic analysis that integrates complementary algorithmic and systems-level solutions.

First, to resolve fragmentation at the alignment level, we develop the Versatile Alignment Toolkit (VAT), a unified algorithmic system for biological sequence alignment across diverse applications. VAT introduces an asymmetric multi-view k-mer indexing scheme that integrates multiple seeding strategies within a single architecture and enables dynamic seed-length adjustment via longest common prefix (LCP)–based inference without re-indexing. A flexible seed-chaining mechanism further supports diverse alignment scenarios, including collinear, rearranged, and split alignments. Combined with a hardware-efficient in-register bitonic sorting algorithm and dynamic index-loading strategy, VAT achieves high efficiency and broad applicability across read mapping, homology search, and whole-genome alignment. Second, to address the challenge of tool selection and pipeline construction, we develop SNAIL, a natural language processing system for automated recognition of bioinformatics tools from large-scale and rapidly growing scientific literature. By integrating XGBoost and Transformer-based models such as SciBERT, SNAIL enables structured extraction of analytical tools and supports automated, reproducible pipeline construction.

Together, this work establishes a unified framework that is grounded in real-world applications and addresses key bottlenecks in metagenomic analysis, enabling more efficient, scalable, and clinically actionable workflows.

Graph Neural Networks (GNNs) achieve state-of-the-art performance in modeling complex biological and behavioral systems, yet their "black-box" nature limits their utility for scientific discovery and clinical translation. Standard post-hoc explainability methods typically attribute importance to low-level features, such as individual nodes or edges, which often fail to map onto the high-level, domain-specific concepts utilized by experts. To address this gap, this thesis explores diverse methodological strategies for achieving Concept-Level Interpretability in GNNs, demonstrating how deep learning models can be structurally and analytically aligned with expert domain knowledge. This theme is explored through two distinct methodological paradigms applied to critical challenges in neuroscience and clinical psychology. First, we introduce an interpretable-by-design approach for modeling brain structure-function coupling. By employing an ensemble of GNNs conceptually biased via input graph filtering, the model enforces verifiably disentangled node embeddings. This allows for the quantitative testing of specific structural hypotheses, revealing that a minority of strong anatomical connections disproportionately drives functional connectivity predictions. Second, we present a post-hoc conceptual alignment paradigm for quantifying atypical motor signatures in Autism Spectrum Disorder (ASD). Utilizing a Spatio-Temporal Graph Autoencoder (STGCN-AE) trained on normative skeletal data, we establish an unsupervised anomaly detection system. To provide clinical interpretability, the model's reconstruction error is systematically aligned with a library of human-interpretable kinematic features, such as postural sway and limb jerk. Explanatory meta-modeling via XGBoost and SHAP analysis further translates this abstract loss into a multidimensional clinical signature. Together, these applications demonstrate that integrating concept-level interpretability through either architectural design or systematic post-hoc alignment enables GNNs to serve as robust tools for hypothesis testing and clinical assessment.

Electronic Health Records (EHRs) provide rich longitudinal patient information, but their high dimensionality, sparsity, heterogeneity, and temporal complexity make robust representation learning difficult. This dissertation studies how to improve patient and medical concept representation learning in EHRs and consequently enhance healthcare predictive tasks by integrating domain knowledge, knowledge graphs, large language models (LLMs), and contrastive learning. First, it introduces an ontology-aware temporal contrastive framework for survival analysis that learns discriminative patient representations from censored and observed trajectories by modeling temporal distinctiveness in longitudinal EHR data. Second, it proposes a multi-ontology representation learning framework that jointly propagates knowledge within and across diagnosis, medication, and procedure ontologies, enabling richer medical concept embeddings, especially under limited data and for rare conditions. Third, it develops an LLM-enriched, text-attributed medical knowledge graph framework that combines EHR-derived statistical evidence with type-constrained LLM reasoning to infer semantic relations, generate contextual node and edge descriptions, and co-learn concept embeddings through joint language-model and graph-neural-network training. Together, these studies advance a unified view of EHR representation learning in which structured medical knowledge, textual semantics, and temporal patient trajectories are jointly leveraged to build more accurate, interpretable, and robust healthcare prediction models.

Android is the most widely adopted mobile operating system, supporting billions of devices and driven by a robust app ecosystem.  Its permission-based security model aims to enforce the Principle of Least Privilege (PoLP), restricting apps to only the permissions it needs.  However, many apps still request excessive permissions, increasing the risk of data leakage and malicious exploitation. Previous research on overprivileged permission has become ineffective due to outdated methods and increasing technical complexity.  The introduction of runtime permissions and scoped storage has made some of the traditional analysis techniques obsolete.  Additionally, developers often are not transparent in explaining the usage of app permissions on the Play Store, misleading users unknowingly and unwillingly granting unnecessary permissions. This combination of overprivilege and poor transparency poses significant security threats to Android users.  Recently, the rise of local large language models (LLMs) has shown promise in various security fields. The main focus of this study is to analyze whether an app is overpriviledged based on app description provided on the Play Store using Local LLM. Finally, we conduct a manual evaluation to validate the LLM’s findings, comparing its results against human-verified response.

Individuals with Autism Spectrum Disorder (ASD) frequently experience elevated stress and are at higher risk for mood disorders such as anxiety and depression. Sensory over-responsivity, social challenges, and difficulties with emotional recognition and regulation contribute to such heightened stress. This study presents a proof-of-concept system that detects and mitigates stress through interactions with a virtual pet. Designed for young adults with high-functioning autism, and potentially useful for people beyond that group, the system monitors simulated heart rate, skin resistance, body temperature, and environmental sound and light levels. Upon detection of stress or potential triggers, the system alerts the user and offers stress-reduction activities via a virtual pet, including guided deep-breathing exercises and interactive engagement with the virtual companion. Through combining real-time stress detection with interactive interventions on a single platform, the system aims to help autistic individuals recognize and manage stress more effectively.

Facial recognition systems increasingly rely on machine learning services, yet they remain vulnerable to cyber-attacks. While traditional adversarial attacks target input data, an underexplored threat comes from weight manipulation attacks, which directly modify model parameters and can compromise deployed systems in cyber-physical settings. This paper investigates defenses against Weight Surgery, a weight manipulation attack that modifies the final linear layer of neural networks to merge or shatter classes without requiring access to training data. We propose a computationally lightweight defense capable of detecting sample pairs affected by Weight Surgery at low false-positive rates. The defense is designed to operate in realistic deployment scenarios, selecting its sensitivity parameter 𝛾 using only benign samples to meet a target false-positive rate. Evaluation on 1000 independently attacked models demonstrates that our method achieves over 95% recall at a target false-positive rate of 0.001. Performance remains strong even under stricter conditions: at FPR = 0.0001, recall is 92.5%, and at 𝛾=0.98, FPR drops to 0.00001 while maintaining 88.9% recall. These results highlight the robustness and practicality of the defense, offering an effective safeguard for neural networks against model-targeted attacks.

Microelectronics supply chains increasingly rely on globally distributed design, fabrication, integration, and deployment processes, making traditional assumptions of trusted hardware inadequate. Security in this setting can be understood through a zero-trust microelectronics supply-chain model, in which neither manufacturing partners nor procured hardware platforms are assumed trustworthy by default. Two complementary threat scenarios are considered in the proposed research. In the first scenario, custom Integrated Circuits (ICs) fabricated through potentially untrusted foundries are examined, where design-for-security protections intended to prevent piracy, overproduction, and intellectual-property theft can themselves become vulnerable to attacks. In this scenario, hardware Trojan-assisted meta-attacks are used to show that such protections can be systematically identified and subverted by fabrication-stage adversaries. In the second scenario, commercial off-the-shelf ICs are considered from the perspective of end users and procurers, where internal design visibility is unavailable and hardware trustworthiness cannot be directly verified. For this setting, runtime-oriented protection mechanisms are developed to safeguard sensitive computation against malicious hardware behavior and side-channel leakage. Building on these two scenarios, a future research direction is outlined for side-channel-driven vulnerability discovery in off-the-shelf devices, motivated by the need to evaluate and test such platforms prior to deployment when no design information is available. The proposed direction explores gray-box security evaluation using power and electromagnetic side-channel analysis to identify anomalous behaviors and potential vulnerabilities in opaque hardware platforms. Together, these directions establish a foundation for analyzing and mitigating security risks across zero-trust microelectronics supply chains.

Many High Performance Computing (HPC) applications following the Bulk Synchronous Parallel (BSP) model are increasingly deployed in cloud-native, multi-tenant container environments such as Kubernetes. Unlike dedicated HPC clusters, these shared platforms introduce resource virtualization and variability, making BSP applications more susceptible to performance fluctuations.

Workload imbalance across supersteps can trigger the straggler effect, where faster tasks wait at synchronization barriers for slower ones, increasing overall execution time. Existing BSP resource management approaches typically assume static workloads and reuse a single configuration throughout execution. However, real-world workloads vary due to dynamic data and system conditions, making static configurations suboptimal. This limitation underscores the need for adaptive resource management strategies that respond to workload changes while considering reconfiguration costs.

To address these limitations, we evaluate a dynamic, data-driven resource management framework tailored for cloud-native BSP applications. This approach integrates workload profiling, time-series forecasting, and predictive performance modeling to estimate task execution behavior under varying workload and resource conditions. The framework explicitly models the trade-off between performance gains achieved through reconfiguration and the associated checkpointing and migration costs incurred during container reallocation. Multiple reconfiguration strategies are evaluated, spanning simple window-based heuristics, dynamic programming methods, and reinforcement learning approaches. Through extensive experimental evaluation, this framework demonstrates up to 24.5% improvement in total execution time compared to a baseline static configuration. Furthermore, we systematically analyze the performance of each strategy under varying workload characteristics, simulation lengths, and checkpoint penalties, and provide guidance on selecting the most appropriate strategy for a given workload environment.