I2S Masters/ Doctoral Theses

Deep learning leads the performance in many areas of computer vision. However, after a decade of research, it tends to require larger datasets and more complex models, leading to heightened resource consumption across all fronts. Regrettably, meeting these requirements proves challenging in many real-life scenarios. First, both data collection and labeling processes entail substantial labor and time investments. This challenge becomes especially pronounced in domains such as medicine, where identifying rare diseases demands meticulous data curation. Secondly, the large size of state-of-the-art models, such as ViT, Stable Diffusion, and ConvNext, hinders their deployment on resource-constrained platforms like mobile devices. Research indicates pervasive redundancies within current neural network structures, exacerbating the issue. Lastly, even with ample datasets and optimized models, the time required for training and inference remains prohibitive in certain contexts. Consequently, there is a burgeoning interest among researchers in exploring avenues for efficient artificial intelligence.

This study endeavors to delve into various facets of efficiency within computer vision, including data efficiency, model efficiency, as well as training and inference efficiency. The data efficiency is improved from the perspective of increasing information brought by given image inputs and reducing redundancies of RGB image formats. To achieve this, we propose to integrate both spatial and frequency representations to finetune the classifier. Additionally, we propose explicitly increasing the input information density in the frequency domain by deleting unimportant frequency channels. For model efficiency, we scrutinize the redundancies present in widely used vision transformers. Our investigation reveals that trivial attention in their attention modules covers useful non-trivial attention due to its large amount. We propose mitigating the impact of accumulated trivial attention weights. To increase training efficiency, we propose SuperLoRA, a generation of LoRA adapter, to fine-tune pretrained models with few iterations and extremely-low parameters. Finally, a model simplification pipeline is proposed to further reduce inference time on mobile devices. By addressing these challenges, we aim to advance the practicality and performance of computer vision systems in real-world applications.

The last century has seen incredible growth in the field of quantum computing. Quantum computation offers the opportunity to find efficient solutions to certain computational problems which are intractable on classical computers. One class of problems that seems to benefit from quantum computing is the Hidden Subgroup Problem (HSP). The HSP includes, as special cases, the problems of integer factoring, discrete logarithm, shortest vector, and subset sum - making the HSP incredibly important in various fields of research.                               

The presented research examines the HSP for Dihedral groups with order 2^n and proves a quantum polynomial-time reduction to the so-called Codomain Fiber Intersection Problem (CFIP). The usual approach to the HSP relies on harmonic analysis in the domain of the problem and the best-known algorithm using this approach is sub-exponential, but still super-polynomial. The algorithm we will present deviates from the usual approach by focusing on the structure encoded in the codomain and uses this structure to direct a “walk” down the subgroup lattice terminating at the hidden subgroup.                               

Though the algorithm presented here is specifically designed for the DHSP, it has potential applications to many other types of the HSP. It is hypothesized that any group with a sufficiently structured subgroup lattice could benefit from the analysis developed here. As this approach diverges from the standard approach to the HSP it could be a promising step in finding an efficient solution to this problem.

Information theory provides methods for quantifying the information content of observed signals and has found application in the radar sensing space for many years. Here, we examine a type of information derived from Fisher information known as Marginal Fisher Information (MFI) and investigate its use to design pulse-agile waveforms. By maximizing this form of information, the expected error covariance about an estimation parameter space may be minimized. First, a novel method for designing MFI optimal waveforms given an arbitrary waveform model is proposed and analyzed. Next, a transformed domain approach is proposed in which the estimation problem is redefined such that information is maximized about a linear transform of the original estimation parameters. Finally, informationally optimal waveform design is paired with informationally optimal estimation (receive processing) and are combined into a cognitive radar concept. Initial experimental results are shown and a proposal for continued research is presented.

In uniform pulse-Doppler radar, there is a well known trade-off between unambiguous Doppler and unambiguous range. Pulse repetition interval (PRI) staggering, a technique that involves modulating the interpulse times, addresses this trade-space allowing for expansion of the unambiguous Doppler domain with little range swath incursion. Random PRI staggering provides additional diversity, but comes at the cost of increased Doppler sidelobes. Thus, careful PRI sequence design is required to avoid spurious sidelobe peaks that could result in false alarms.

In this thesis, two random PRI stagger models are defined and compared, and sidelobe peak mitigation is discussed. First, the co-array concept (borrowed from the intuitively related field of sparse array design in the spatial domain) is utilized to examine the effect of redundancy on sidelobe peaks for random PRI sequences. Then, a sidelobe peak suppression technique is introduced that involves a gradient-based optimization of the random PRI sequences, producing pseudo-random sequences that are shown to significantly reduce spurious Doppler sidelobes in both simulation and experimentally.

Electrostriction in an optical fiber is introduced by the interaction between the forward propagated optical signal and the acoustic standing waves in the radial direction resonating between the center of the core and the cladding circumference of the fiber. The response of electrostriction is dependent on fiber parameters, especially the mode field radius. A novel technique is demonstrated to characterize fiber properties by means of measuring their electrostriction response under intensity modulation. As the spectral envelope of electrostriction-induced propagation loss is anti-symmetrical, the signal-to-noise ratio can be significantly increased by subtracting the measured spectrum from its complex conjugate. It is shown that if the transversal field distribution of the fiber propagation mode is Gaussian, the envelope of the electrostriction-induced loss spectrum closely follows a Maxwellian distribution whose shape can be specified by a single parameter determined by the mode field radius. 

The Black Literature Network Project is a comprehensive initiative to disseminate literature knowledge to students, academics, and the general public. It encompasses four distinct portals, each featuring content created and curated by scholars in the field. These portals include the Novel Generator Machine, Literary Data Gallery, Multithreaded Literary Briefs, and Remarkable Receptions Podcast Series. My significant contribution to this project was creating a standalone website for the Current Archives and Collections Index that offers an easily searchable index of black-themed collections. Additionally, I was exclusively responsible for the complete development of the novel generator tool. This application provides customized book recommendations based on user preferences. As a part of the History of Black Writing (HBW) Program, I had the opportunity to customize an open-source annotation tool called Hypothesis. This customization allowed for its use on all websites related to the Black Literature Network Project by the end users. The Black Book Interactive Project (BBIP) collaborates with institutions and groups nationwide to promote access to Black-authored texts and digital publishing. Through BBIP, we plan to increase black literature’s visibility in digital humanities research.

Machine learning (ML) has transformed numerous domains, demonstrating exceptional performance in autonomous driving, medical diagnosis, and decision-making tasks. Nevertheless, ensuring the trustworthiness of ML models remains a persistent challenge, particularly with the emergence of new applications. The primary challenges in this context are the selection of an appropriate solution from a multitude of options, mitigating adversarial attacks, and advancing towards a unified solution that can be applied universally.

The thesis comprises three interconnected parts, all contributing to the overarching goal of improving trustworthiness in machine learning. Firstly, it introduces an automated machine learning (AutoML) framework that streamlines the training process, achieving optimum performance, and incorporating existing solutions for handling trustworthiness concerns. Secondly, it focuses on enhancing the robustness of machine learning models, particularly against adversarial attacks. A robust detector named "Argos" is introduced as a defense mechanism, leveraging the concept of two "souls" within adversarial instances to ensure robustness against unknown attacks. It incorporates the visually unchanged content representing the true label and the added invisible perturbation corresponding to the misclassified label. Thirdly, the thesis explores the realm of causal ML, which plays a fundamental role in assisting decision-makers and addressing challenges such as interpretability and fairness in traditional ML. By overcoming the difficulties posed by selective confounding in real-world scenarios, the proposed scheme utilizes dual-treatment samples and two-step procedures with counterfactual predictors to learn causal relationships from observed data. The effectiveness of the proposed scheme is supported by theoretical error bounds and empirical evidence using synthetic and real-world child placement data. By reducing the requirement for observed confounders, the applicability of causal ML is enhanced, contributing to the overall trustworthiness of machine learning systems.

The accelerated melting of ice sheets in the polar regions of the world, specifically in Greenland and Antarctica, due to contemporary climate warming is contributing to global sea level rise. To understand and quantify this phenomenon, airborne radars have been deployed to create echogram images that map snow accumulation patterns in these regions. Using advanced radar systems developed by the Center for Remote Sensing and Integrated Systems (CReSIS), a significant amount (1.5 petabytes) of climate data has been collected. However, the process of extracting ice phenomenology information, such as accumulation rate, from the data is limited. This is because the radar echograms require tracking of the internal layers, a task that is still largely manual and time-consuming. Therefore, there is a need for automated tracking.

Machine learning and deep learning algorithms are well-suited for this problem given their near-human performance on optical images. Moreover, the significant overlap between classical radar signal processing and machine learning techniques suggests that fusion of concepts from both fields can lead to optimized solutions for the problem. However, supervised deep learning algorithms suffer the circular problem of first requiring large amounts of labeled data to train the models which do not exist currently.

In this work, we propose custom algorithms, including supervised, semi-supervised, and self-supervised approaches, to deal with the limited annotated data problem to achieve accurate tracking of radiostratigraphic layers in echograms. Firstly, we propose an iterative multi-class classification algorithm, called “Row Block,” which sequentially tracks internal layers from the top to the bottom of an echogram given the surface location. We aim to use the trained iterative model in an active learning paradigm to progressively increase the labeled dataset. We also investigate various deep learning semantic segmentation algorithms by casting the echogram layer tracking problem as a binary and multiclass classification problem. These require post-processing to create the desired vector-layer annotations, hence, we propose a custom connected-component algorithm as a post-processing routine. Additionally, we propose end-to-end algorithms that avoid the post-processing to directly create annotations as vectors. Furthermore, we propose semi-supervised algorithms using weakly-labeled annotations and unsupervised algorithms that can learn the latent distribution of echogram snow layers while reconstructing echogram images from a sparse embedding representation.

A concurrent objective of this work is to provide the deep learning and science community with a large fully-annotated dataset. To achieve this, we propose synchronizing radar data with outputs from a regional climate model to provide a dataset with overlapping measurements that can enhance the performance of the trained models.

Today’s smartphone platforms have millions of applications, which not only access users’ private data but also information from the connected external services and IoT/CPS devices. Mobile application security involves protecting sensitive information and securing communication between the application and external services or devices. We focus on these two key aspects of mobile application security.

In the first part of this dissertation, we aim to ensure the security of user information collected by mobile apps. Mobile apps seek consent from users to approve various permissions to access sensitive information such as location and personal information. However, users often blindly accept permission requests and apps start to abuse this mechanism. As long as a permission is requested, the state-of-the-art security mechanisms will treat it as legitimate. We ask the question whether the permission requests are valid? We attempt to validate permission requests using statistical analysis on permission sets extracted from groups of functionally similar apps. We detected mobile applications with abusive permission access and measure the risk of information leaks through each mobile application.

Second, we propose to investigate the security of auto companion apps. Auto companion apps are mobile apps designed to remotely connect with cars to provide features such as diagnostics, navigation, entertainment, and safety alerts. However, this can lead to several security threats, for instance, onboard information of vehicles can be tracked or altered through a malicious app. We design a comprehensive security analysis framework on automotive companion apps all stages of communication and collaboration between vehicles and companion apps such as connection establishment, authentication, encryption, information storage, and Vehicle diagnostic and control command access. By conducting static and network traffic analysis of Android OBD apps, we identify a series of vulnerability scenarios. We further evaluate these vulnerabilities with vehicle-based testing and identify potential security threats associated with auto companion apps.

Layered attestation is a process by which one can establish trust in a remote party. It is a special case of attestation in which different layers of the attesting system are handled distinctly. This type of trust is desirable because a vast and growing number of people depend on networked devices to go about their daily lives. Current architectures for remote attestation are lacking in process isolation, which is evidenced by the existence of virtual machine escape exploits. This implies a deficiency of trustworthy ways to determine whether a networked Linux system has been exploited. The seL4 microkernel, uniquely in the world, has machine-checked proofs concerning process confidentiality and integrity. The seL4 microkernel is leveraged here to provide a verified level of software-based process isolation. When complemented with a comprehensive collection of measurements, this architecture can be trusted to report its own corruption. The architecture is described, implemented, and tested against a variety of exploits, which are detected using introspective measurement techniques.