I2S Masters/ Doctoral Theses

Regular inspections are crucial for identifying and assessing damage in concrete dams, including a wide range of damage states. Manual inspections of dams are often constrained by cost, time, safety, and inaccessibility. Automating dam inspections using artificial intelligence has the potential to improve the efficiency and accuracy of data analysis. Computer vision and deep learning models have proven effective in detecting a variety of damage features using images, but their success relies on the availability of high-quality and diverse training data. This is because supervised learning, a common machine-learning approach for classification problems, uses labeled examples, in which each training data point includes features (damage images) and a corresponding label (pixel annotation). Unfortunately, public datasets of annotated images of concrete dam surfaces are scarce and inconsistent in quality, quantity, and representation.

To address this challenge, we present a novel approach that involves synthesizing a realistic environment using a 3D model of a dam. By overlaying this model with synthetically created photorealistic damage textures, we can render images to generate large and realistic datasets with high-fidelity annotations. Our pipeline uses NX and Blender for 3D model generation and assembly, Substance 3D Designer and Substance Automation Toolkit for texture synthesis and automation, and Unreal Engine 5 for creating a realistic environment and rendering images. This generated synthetic data is then used to train deep learning models in the subsequent steps. The proposed approach offers several advantages. First, it allows generation of large quantities of data that are essential for training accurate deep learning models. Second, the texture synthesis ensures generation of high-fidelity ground truths (annotations) that are crucial for making accurate detections. Lastly, the automation capabilities of the software applications used in this process provides flexibility to generate data with varied textures elements, colors, lighting conditions, and image quality overcoming the constraints of time. Thus, the proposed approach can improve the automation of dam inspection by improving the quality and quantity of training data.

Machine Learning (ML) has revolutionized various fields, from disease prediction to credit risk evaluation, by harnessing abundant data scattered across diverse sources. However, transporting data to a trusted server for centralized ML model training is not only costly but also raises privacy concerns, particularly with legislative standards like HIPAA in place. In response to these challenges, Federated Learning (FL) has emerged as a promising solution. FL involves training a collaborative model across a network of clients, each retaining its own private data. By conducting training locally on the participating clients, this approach eliminates the need to transfer entire training datasets while harnessing their computation capabilities. However, FL introduces unique privacy risks, security concerns, and robustness challenges. Firstly, FL is susceptible to malicious actors who may tamper with local data, manipulate the local training process, or intercept the shared model or gradients to implant backdoors that affect the robustness of the joint model. Secondly, due to the statistical and system heterogeneity within FL, substantial differences exist between the distribution of each local dataset and the global distribution, causing clients’ local objectives to deviate greatly from the global optima, resulting in a drift in local updates. Addressing such vulnerabilities and challenges is crucial before deploying FL systems in critical infrastructures.

In this dissertation, we present a multi-pronged approach to address the privacy, security, and robustness challenges in FL. This involves designing innovative privacy protection mechanisms and robust aggregation schemes to counter attacks during the training process. To address the privacy risk due to model or gradient interception, we present the design of a reliable and accountable blockchain-enabled privacy-preserving federated learning (PPFL) framework which leverages homomorphic encryption to protect individual client updates. The blockchain is adopted to support provenance of model updates during training so that malformed or malicious updates can be identified and traced back to the source. 

We studied the challenges in FL due to heterogeneous data distributions and found that existing FL algorithms often suffer from slow and unstable convergence and are vulnerable to poisoning attacks, particularly in extreme non-independent and identically distributed (non-IID) settings. We propose a robust aggregation scheme, named CONTRA, to mitigate data poisoning attacks and ensure an accuracy guarantee even under attack. This defense strategy identifies malicious clients by evaluating the cosine similarity of their gradient contributions and subsequently removes them from FL training. Finally, we introduce FL-GMM, an algorithm designed to tackle data heterogeneity while prioritizing privacy. It iteratively constructs a personalized classifier for each client while aligning local-global feature representations. By aligning local distributions with global semantic information, FL-GMM minimizes the impact of data diversity. Moreover, FL-GMM enhances security by transmitting derived model parameters via secure multiparty computation, thereby avoiding vulnerabilities to reconstruction attacks observed in other approaches. 

As internet services like high-definition videos, cloud computing, and artificial intelligence keep growing, optical networks need to keep up with the demand for more capacity. Optical amplifiers play a crucial role in offsetting fiber loss and enabling long-distance wavelength division multiplexing (WDM) transmission in high-capacity systems. Various methods have been proposed to enhance the capacity and reach of fiber communication systems, including advanced modulation formats, dense wavelength division multiplexing (DWDM) over ultra-wide bands, space-division multiplexing, and high-performance digital signal processing (DSP) technologies. To sustain higher data rates while maximizing the spectral efficiency of multi-level modulated signals, a higher Optical signal-to-noise ratio (OSNR) is necessary. Despite advancements in coherent optical communication systems, the spectral efficiency of multi-level modulated signals is ultimately constrained by fiber nonlinearity.

Raman amplification is an attractive solution for wide-band amplification with low noise figures in multi-band systems. Distributed Raman Amplification (DRA) has been deployed in recent high-capacity transmission experiments to achieve a relatively flat signal power distribution along the optical path and offers the unique advantage of using conventional low-loss silica fibers as the gain medium, effectively transforming passive optical fibers into active or amplifying waveguides. Additionally, DRA provides gain at any wavelength by selecting the appropriate pump wavelength, enabling operation in signal bands outside the Erbium-doped fiber amplifier (EDFA) bands. Forward (FW) Raman pumping in DRA can be adopted to further improve the DRA performance as it is more efficient in OSNR improvement because the optical noise is generated near the beginning of the fiber span and attenuated along the fiber. Dual-order FW pumping helps to reduce the non-linear effect of the optical signal and improves OSNR by more uniformly distributing the Raman gain along the transmission span.

The major concern with Forward Distributed Raman Amplification (FW DRA) is the fluctuation in pump power, known as relative intensity noise (RIN), which transfers from the pump laser to both the intensity and phase of the transmitted optical signal as they propagate in the same direction. Additionally, another concern of FW DRA is the rise in signal optical power near the start of the fiber span, leading to an increase in the Kerr-effect-induced non-linear phase shift of the signal. These factors, including RIN transfer-induced noise and non-linear noise, contribute to the degradation of the system performance in FW DRA systems at the receiver. As the performance of DRA with backward pumping is well understood with a relatively low impact of RIN transfer, our study is focused on the FW pumping scheme.

Our research is intended to provide a comprehensive analysis of the system performance impact of dual-order FW Raman pumping, including signal intensity and phase noise induced by the RINs of both the 1st and the 2nd order pump lasers, as well as the impacts of linear and nonlinear noise. The efficiencies of pump RIN to signal intensity and phase noise transfer are theoretically analyzed and experimentally verified by applying a shallow intensity modulation to the pump laser to mimic the RIN. The results indicate that the efficiency of the 2nd order pump RIN to signal phase noise transfer can be more than 2 orders of magnitude higher than that from the 1st order pump. Then the performance of the dual-order FW Raman configurations is compared with that of single-order Raman pumping to understand the trade-offs of system parameters. The nonlinear interference (NLI) noise is analyzed to study the overall OSNR improvement when employing a 2nd order Raman pump.

Finally, a DWDM system with 16-QAM modulation is used as an example to investigate the benefit of DRA with dual order Raman pumping and with different pump RIN levels. We also consider a DRA system using a 1st order incoherent pump together with a 2nd order coherent pump. Although dual-order FW pumping corresponds to a slight increase of linear amplified spontaneous emission (ASE) compared to using only a 1st order pump, its major advantage comes from the reduction of nonlinear interference noise in a DWDM system. Because the RIN of the 2nd order pump has much higher impact than that of the 1st order pump, there should be more stringent requirement on the RIN of the 2nd order pump laser when dual order FW pumping scheme is used for DRA for efficient fiber-optic communication. Also, the result of system performance analysis reveals that higher baud rate systems, like those operating at 100Gbaud, are less affected by pump laser RIN due to the low-pass characteristics of the transfer of pump RIN to signal phase noise.

Emerging applications in next-generation wireless networks are driving the need for innovative communication and computation systems. Notable examples include augmented and virtual reality (AR/VR), autonomous vehicles, and mobile edge computing, all of which demand significant computational and communication resources at the network edge. These demands place a strain on edge devices, which are often resource-constrained. In order to incorporate available communication and computation resources, while enhancing user experience, this PhD research is dedicated to developing joint communication and computation solutions for next generation wireless applications that could potentially operate in high frequencies such as millimeter wave (mmWave) bands.

In the first thrust of this study, we examine the problem of energy-constrained computation offloading to edge servers in a multi-user multi-channel wireless network. To develop a decentralized offloading policy for each user, we model the problem as a partially observable Markov decision problem (POMDP). Leveraging bandit learning methods, we introduce a decentralized task offloading solution, where edge users offload their computation tasks to a nearby edge server using a selected communication channel. The proposed framework aims to meet user's requirements, such as task completion deadline and computation throughput (i.e., the rate at which computational results are produced).

The second thrust of the study emphasizes user-driven requirements for these resource-intensive applications, specifically the Quality of Experience (QoE) in 2D and 3D video streaming. Given the unique characteristics of mmWave networks, we develop a beam alignment and buffer predictive multi-user scheduling algorithm for 2D video streaming applications. This scheduling algorithm balances the trade-off between beam alignment overhead and playback buffer levels for optimal resource allocation across users. Next, we extend our investigation and develop a joint rate adaptation and computation distribution algorithm for 3D video streaming in mmWave-based VR systems. Our proposed framework balances the trade-off between communication and computation resource allocation to enhance the users’ QoE. Our numerical results using real-world mmWave traces and 3D video dataset, show promising improvements in terms of video quality, rebuffering time, and quality variation perceived by users.

Smart grids face challenges in maintaining the balance between generation and consumption at the residential and grid scales with the integration of renewable energy resources. Decentralized, dynamic, and distributed control algorithms are necessary for smart grids to function effectively. The inherent variability and uncertainty of renewables, especially wind and solar energy, complicate the deployment of distributed control algorithms in smart grids. In addition, smart grid systems must handle real-time data collected from interconnected devices and sensors while maintaining reliable and secure communication regardless of network failures. To address these challenges, our research models the integration of renewable energy resources into the smart grid and evaluates how predictive analytics can improve distributed control and energy management, while recognizing the limitations of communication channels and networks.

In the first thrust of this research, we develop a model of a smart grid with renewable energy integration and evaluate how forecasting affects distributed control and energy management. In particular, we investigate how contextual weather information and renewable energy time-series forecasting affect smart grid energy management. In addition to modeling the smart grid system and integrating renewable energy resources, we further explore the use of deep learning methods, such as the Long Short-Term Memory (LSTM) and Transformer models, for time-series forecasting. Time-series forecasting techniques are applied within Reinforcement Learning (RL) frameworks to enhance decision-making processes.

In the second thrust, we note that data collection and sharing across the smart grids require considering the impact of network and communication channel limitations in our forecasting models. As renewable energy sources and advanced sensors are integrated into smart grids, communication channels on wireless networks are overflowed with data, requiring a shift from transmitting raw data to processing only useful information to maximize efficiency and reliability. To this end, we develop a task-oriented communication model that integrates data compression and the effects of data packet queuing with considering limitation of communication channels, within a remote time-series forecasting framework. Furthermore, we jointly integrate data compression technique with age of information metric to enhance both relevance and timeliness of data used in time-series forecasting.

DRAM constitutes over 50% of server cost and 75% of the embodied carbon footprint of a server. To mitigate DRAM cost, far memory architectures have emerged. They can be separated into two broad categories: software-defined far memory (SFM) and disaggregated far memory (DFM). In this work, we compare the cost of SFM and DFM in terms of their required capital investment, operational expense, and carbon footprint. We show that, for applications whose data sets are compressible and have predictable memory access patterns, it takes several years for a DFM to break even with an equivalent capacity SFM in terms of cost and sustainability. We then introduce XFM, a near-memory accelerated SFM architecture, which exploits the coldness of data during SFM-initiated swap ins and outs. XFM leverages refresh cycles to seamlessly switch the access control of DRAM between the CPU and near-memory accelerator. XFM parallelizes near-memory accelerator accesses with row refreshes and removes the memory interference caused by SFM swap ins and outs.

We modify an open source far memory implementation to implement a full-stack, user-level XFM. Our experimental results use a combination of an FPGA implementation, simulation, and analytical modeling to show that XFM eliminates memory bandwidth utilization when performing compression and decompression operations with SFMs of capacities up to 1TB. The memory and cache utilization reductions translate to 5∼27% improvement in the combined performance of co-running applications.

A coalition of companies collaborates collectively and shares information to improve their operations together. Distributed trust and transparency cannot be obtained with centralized file-sharing platforms. File sharing may be done transparently and securely with blockchain technology. This project suggests an inter-organizational secure file-sharing system based on blockchain technology. The group can use it to securely share files in a distributed manner. The creation of smart contracts and the configuration of blockchain networks are carried out by Hyperledger Fabric, an enterprise blockchain platform. Distributed file storage is accomplished through the usage of the Inter Planetary File System (IPFS).

The workflow for file-sharing and identity management procedures is provided in the paper. Using blockchain technology, the recommended approach enables a group of businesses to share files with availability, integrity, and confidentiality. The suggested method uses blockchain to enable safe file exchange amongst a group of enterprises. It offers shared file availability, confidentiality, and integrity. It guarantees complete file encryption. The blockchain provides tamper-resistant storage for the shared file's content ID. On the distributed storage and blockchain ledger, respectively, the encrypted file and file metadata are stored.

This PhD research addresses two challenges in future wireless systems: hybrid precoder design for sub-Terahertz (sub-THz) massive multiple-input multiple-output (MIMO) communications and private federated learning (FL) over wireless channels. The first part of the research introduces a novel hybrid precoding framework that combines true-time delay (TTD) and phase shifters (PS) precoders to counteract the beam squint effect - a significant challenge in sub-THz massive MIMO systems that leads to considerable loss in array gain.

Our research presents a novel joint optimization framework for the TTD and PS precoder design, incorporating realistic time delay constraints for each TTD device. We first derive a lower bound on the achievable rate of the system and show that, in the asymptotic regime, the optimal analog precoder that fully compensates for the beam squint is equivalent to the one that maximizes this lower bound. Unlike previous methods, our framework does not rely on the unbounded time delay assumption and optimizes the TTD and PS values jointly to cope with the practical limitations. Furthermore, we determine the minimum number of TTD devices needed to reach a target array gain using our proposed approach.

Simulations validate that the proposed approach demonstrates performance enhancement, ensures array gain, and achieves computational efficiency. In the second part, the research devises a differentially private FL algorithm that employs time-varying noise perturbation and optimizes transmit power to counteract privacy risks, particularly those stemming from engineering-inversion attacks. This method harnesses inherent wireless channel noise to strike a balance between privacy protection and learning utility. By strategically designing noise perturbation and power control, our approach not only safeguards user privacy but also upholds the quality of the learned FL model. Additionally, the number of FL iterations is optimized by minimizing the upper bound on the learning error. We conduct simulations to showcase the effectiveness of our approach in terms of DP guarantee and learning utility.

This project offers a hands-on investigation of object identification utilizing the YOLO method, Python, and OpenCV. It begins by explaining the YOLO architecture, focusing on the single-stage detection process for bounding box prediction and class probability calculation. The setup phase includes library installation and model configuration, resulting in a smooth implementation procedure. Using OpenCV, the project includes preparatory processes required for object detection in images. The YOLO model is seamlessly integrated into the OpenCV framework, enabling object detection. Post-processing techniques, such as non-maximum suppression, are used to modify detection results and improve accuracy. Visualizations, such as bounding boxes and labels, are used to help interpret the discovered items. The project finishes by investigating potential expansions and optimizations, such as custom dataset training and deployment on edge devices, opening up new paths for further investigation and development. This project provides developers with the tools and knowledge they need to build effective object detection systems for a wide range of applications, from surveillance and security to autonomous vehicles and augmented reality.

This project explores the adaptation of the U-Net convolutional neural network, renowned for its medical image segmentation prowess, to the analysis of Printed Circuit Boards (PCBs). By utilizing the Fine-Printed Circuit Board Image Collection (FPIC) dataset, we address key challenges in PCB inspection, such as the precise segmentation of complex components, handling class imbalances, and capturing minute details.

The U-Net model has been finely tuned with an encoding-decoding architecture, enhanced by convolutional layers, batch normalization, and dropout techniques to extract and reconstruct high-quality features from PCB images effectively. The Dice coefficient, used as the loss function, significantly improves boundary accuracy, and manages class diversity. Throughout extensive training and validation phases, the model has demonstrated superior performance metrics compared to traditional methods, making substantial advancements in automated PCB inspection.

During the rigorous training and validation stages, the U-Net model demonstrated excellent performance metrics, eclipsing traditional inspection methods. For capacitors, the model achieved a training accuracy of 95.03% and a validation accuracy of 95.92%. For resistors, training using transfer learning techniques resulted in even more remarkable performance, with training accuracy reaching 98% and validation accuracy hitting 98.23%. These metrics highlight the model's robustness and accuracy, marking a significant advancement in automated PCB inspection and suggesting the model's potential for wider industrial applications in multiclass component segmentation within complex PCB.