Publications

Preprints

1. Sensor Scheduling in Intrusion Detection Games with Uncertain Payoffs Jayanth Bhargav, Mahsa Ghasemi, Shreyas Sundaram Pre-print (under review) [Abs] [PDF]

   We study the problem of sensor scheduling for an intrusion detection task. We model this as a 2-player zero-sum game over a graph, where the defender (Player 1) seeks to identify the optimal sensor scheduling strategy to minimize the probability of missed detection at minimal cost, while the intruder (Player 2) aims to identify the optimal path selection strategy to maximize missed detection probability at minimal cost. The defender’s strategy space grows exponentially with the number of sensors, making direct computation of the Nash Equilibrium (NE) strategies computationally expensive. To tackle this, we propose a distributed variant of the Weighted Majority algorithm that exploits the structure of the game's payoff matrix, enabling efficient computation of the NE strategies with provable convergence guarantees. Next, we consider a more challenging scenario where the defender lacks knowledge of the true sensor models and, consequently, the game's payoff matrix. For this setting, we develop online learning algorithms that leverage bandit feedback from sensors to estimate the NE strategies. By building on existing results from perturbation theory and online learning in matrix games, we derive high-probability order optimal regret bounds for our algorithms. Finally, through simulations, we demonstrate the empirical performance of our proposed algorithms in both known and unknown payoff scenarios.

1. Online Laplacian-Based Representation Learning in Reinforcement Learning Maheed H. Ahmed, Jayanth Bhargav, Mahsa Ghasemi Pre-print (under review) [Abs] [PDF]

   Representation learning plays a crucial role in reinforcement learning, especially in complex environments with high-dimensional and unstructured states. Effective representations can enhance the efficiency of learning algorithms by improving sample efficiency and generalization across tasks. This paper considers the Laplacian-based framework for representation learning, where the eigenvectors of the Laplacian matrix of the underlying transition graph are leveraged to encode meaningful features from raw sensory observations of the states. Despite the promising algorithmic advances in this framework, it remains an open question whether the Laplacian-based representations can be learned online and with theoretical guarantees along with policy learning. To answer this question, we study online Laplacian-based representation learning, where the graph-based representation is updated simultaneously while the policy is updated by the reinforcement learning algorithm. We design an online optimization formulation by introducing the Asymmetric Graph Drawing Objective (AGDO) and provide a theoretical analysis of the convergence of running online projected gradient descent on AGDO under mild assumptions. Specifically, we show that if the policy learning algorithm induces a bounded drift on the policy, running online projected gradient descent on AGDO exhibits ergodic convergence. Our extensive simulation studies empirically validate the guarantees of convergence to the true Laplacian representation. Furthermore, we provide insights into the compatibility of different reinforcement learning algorithms with online representation learning.

1. Robust Information Selection for Hypothesis Testing with Misclassification Penalties Jayanth Bhargav, Shreyas Sundaram, Mahsa Ghasemi Pre-print (under review) [Abs] [PDF]

   We study the problem of robust information selection for a Bayesian hypothesis testing / classification task, where the goal is to identify the true state of the world from a finite set of hypotheses based on observations from the selected information sources. We introduce a novel misclassification penalty framework, which enables non-uniform treatment of different misclassification events. Extending the classical subset selection framework, we study the problem of selecting a subset of sources that minimize the maximum penalty of misclassification under a limited budget, despite deletions or failures of a subset of the selected sources. We characterize the curvature properties of the objective function and propose an efficient greedy algorithm with performance guarantees. Next, we highlight certain limitations of optimizing for the maximum penalty metric and propose a submodular surrogate metric to guide the selection of the information set. We propose a greedy algorithm with near-optimality guarantees for optimizing the surrogate metric. Finally, we empirically demonstrate the performance of our proposed algorithms in several instances of the information set selection problem.

1. Optimal Sensor and Actuator Selection for Factored Markov Decision Processes: Complexity, Approximability and Algorithms Jayanth Bhargav, Mahsa Ghasemi, Shreyas Sundaram Pre-print (under review) [Abs] [PDF]

   Factored Markov Decision Processes (fMDPs) are a class of Markov Decision Processes (MDPs) in which the states (and actions) can be factored into a set of state (and action) variables. The state space, action space and reward function of a fMDP can be encoded compactly using a factored representation. In this paper, we consider the setting where we have a set of potential sensors to select for the fMDP (at design-time), where each sensor measures a certain state variable and has a selection cost. We formulate the problem of selecting an optimal set of sensors for fMDPs (subject to certain budget constraints) to maximize the expected infinite-horizon discounted return provided by the optimal control policy. We show the fundamental result that it is NP-hard to approximate this optimization problem to within any non-trivial factor. We then study the dual problem of budgeted actuator selection (at design-time) to maximize the expected return under the optimal policy. Again, we show that it is NP-hard to approximate this optimization problem to within any non-trivial factor. Furthermore, with explicit examples, we show the failure of greedy algorithms for both the sensor and actuator selection problems and provide insights into the factors that cause these problems to be challenging. Despite the inapproximability results, through extensive simulations, we show that the greedy algorithm may provide near-optimal performance for actuator and sensor selection in many real-world and randomly generated fMDP instances.

Conference/Workshop Articles

1. AdverSAR - An Attentive Reasoning Framework for Multi-Agent Cooperative Navigation under Adversarial Communication Jayanth Bhargav, Himanshu Sharma, Arnab Bhattacharya, Ted Fujimoto, Aowabin Rahman, Sayak Mukherjee, Thiagarajan Ramachandran, Samrat Chatterjee 2024 Midwest Robotics Workshop (MWRW), Chicago IL. [Abs] [PDF]

   Learning communication is a critical part of Multi-Agent Reinforcement Learning (MARL) algorithms in order to effectively perform multi-agent cooperative tasks. Many algorithms presented in the literature have focused on learning effective communication. However, as the tasks get harder and the number of agents increase, learnin g which communicated information is useful for each agent's decision-making is still a challenging task. Moreover, when deploying trained agents in real-world tasks such as autonomous driving, communication based policies are prone to severe issues due to presence of noise or potential attacks on the communication channels. In this paper, we propose a novel attention-based MARL framework which integrates an auto-encoder network to combat adversarial corruptions in multi-agent message exchange and learns efficient and interpretable relations between different entities. With this framework, agents learn which of the communicated information is useful and when to ignore potentially corrupted messages, and learn policies resilient to adversaries in the communication network. We empirically demonstrate the strength of our model in cooperative multi-agent navigation tasks of varying levels of difficulty, where agents learn policies resilient to corrupted inter-agent communication. We demonstrate how our model learns to integrate informative messages and ignore potentially corrupted ones by evaluating the quality of communicated information, resulting in improved performance in decision-making compared to baselines.

1. Submodular Information Selection for Hypothesis Testing with Misclassification Penalties Jayanth Bhargav, Mahsa Ghasemi, Shreyas Sundaram 6th Annual Learning for Decision and Control Conference (L4DC 2024), Oxford UK. [Abs] [PDF]

   We consider the problem of selecting an optimal subset of information sources for a hypothesis testing/classification task where the goal is to identify the true state of the world from a finite set of hypotheses, based on finite observation samples from the sources. In order to characterize the learning performance, we propose a misclassification penalty framework, which enables non-uniform treatment of different misclassification errors. In a centralized Bayesian learning setting, we study two variants of the subset selection problem: (i) selecting a minimum cost information set to ensure that the maximum penalty of misclassifying the true hypothesis remains bounded and (ii) selecting an optimal information set under a limited budget to minimize the maximum penalty of misclassifying the true hypothesis. Under mild assumptions, we prove that the objective (or constraints) of these combinatorial optimization problems are weak (or approximate) submodular, and establish high-probability performance guarantees for greedy algorithms. Further, we propose an alternate metric for information set selection which is based on the total penalty of misclassification. We prove that this metric is submodular and establish near-optimal guarantees for the greedy algorithms for both the information set selection problems. Finally, we present numerical simulations to validate our theoretical results over several randomly generated instances.

1. On the Complexity and Approximability of Optimal Sensor Selection for Mixed-Observable Markov Decision Processes Jayanth Bhargav, Mahsa Ghasemi, Shreyas Sundaram 2023 American Control Conference (ACC 2023), San Diego CA.
   ACC Student Travel Grant Award [Abs] [PDF]

   Mixed-Observable Markov Decision Processes (MOMDPs) are used to model systems where the state space can be decomposed as a product space of a set of state variables, and the controlling agent is able to measure only a subset of those state variables. In this paper, we consider the setting where we have a set of potential sensors to select for the MOMDP, where each sensor measures a certain state variable and has a selection cost. We formulate the problem of selecting an optimal set of sensors for MOMDPs (subject to certain budget constraints) to maximize the expected infinite-horizon reward of the agent and show that this sensor placement problem is NP-Hard, even when one has access to an oracle that can compute the optimal policy for any given instance. We then study a greedy algorithm for approximate optimization and show that there exist instances of the MOMDP sensor selection problem where the greedy algorithm can perform arbitrarily poorly. Finally, we provide some empirical results of greedy sensor selection over randomly generated MOMDP instances and show that, in practice, the greedy algorithm provides near-optimal solutions for many cases, despite the fact that one cannot provide general theoretical guarantees for its performance. In total, our work establishes fundamental complexity results for the problem of optimal sensor selection (at design-time) for MOMDPs.

1. Deriving Spatial Policies for Overtaking Maneuvers with Autonomous Vehicles Jayanth Bhargav, Johannes Betz, Hongrui Zheng, Rahul Mangharam 2022 IEEE 14th International Conference on COMmunication Systems & NETworkS (COMSNETS), Bengaluru KA.
   ACM Travel Grant Award [Abs] [PDF]

   Planning an accurate and safe trajectory is a crucial element in autonomous driving. To execute complex driving maneuvers like overtaking, motion planning requires an enhanced decision-making algorithm that decides the when, where and how of the overtaking maneuver. This paper proposes an algorithm that increases the likelihood of a safe overtaking maneuver by learning spatial information. Here, spatial information refers to the track portion/curve and the position of the ego vehicle with reference to that. The technique is applied to an autonomous racing setup where vehicles have to detect and operate at the limits of dynamic handling. To learn the spatial information, offline experiments of a 2-player race are conducted to generate probability distributions of overtaking maneuvers conditioned on speed and relative-position of the ego vehicle with respect to the opponent. Furthermore, a Switched Model Predictive Contouring Controller (SMPCC) is proposed for incorporating the policy learning algorithm into the path planning and control setup. Extensive simulations show that the proposed algorithm is able to achieve an increased number of overtakes at different track portions on known and unknown race tracks.

1. Track based Offline Policy Learning for Overtaking Maneuvers with Autonomous Racecars Jayanth Bhargav, Johannes Betz, Hongrui Zheng, Rahul Mangharam 2021 IEEE International Conference on Robotics and Automation (ICRA 2021) - 1st Workshop on Opportunities and Challenges in Autonomous Racing (virtual) [Abs] [PDF]

   The rising popularity of driver-less cars has led to the research and development in the field of autonomous racing, and overtaking in autonomous racing is a challenging task. Vehicles have to detect and operate at the limits of dynamic handling and decisions in the car have to be made at high speeds and high acceleration. One of the most crucial parts in autonomous racing is path planning and decision making for an overtaking maneuver with a dynamic opponent vehicle. In this paper we present the evaluation of a track based offline policy learning approach for autonomous racing. We define specific track portions and conduct offline experiments to evaluate the probability of an overtaking maneuver based on speed and position of the ego vehicle. Based on these experiments we can define overtaking probability distributions for each of the track portions. Further, we propose a switching MPCC controller setup for incorporating the learnt policies to achieve a higher rate of overtaking maneuvers. By exhaustive simulations, we show that our proposed algorithm is able to increase the number of overtakes at different track portions.

1. IoT Based Wireless Sensor Network (WSN) for Condition Monitoring of Low Power Rooftop PV Panels Sushmita Sarkar, K Uma Rao, Jayanth Bhargav, Shrikesh Sheshaprasad, Anirudh Sharma CA 2019 IEEE 4th International Conference on Condition Assessment Techniques in Electrical Systems (CATCON) [Abs] [PDF]

   Rooftop PV systems are now becoming ubiquitous with the reduction in costs of PV panels and related technologies. Developing countries like India have undertaken Green Energy Transition to ensure sustainability and energy self-sufficiency. The increase in consumer participation in view of the nation's goal necessitates technology advancements in condition monitoring of PV systems. As of today, state of art and robust monitoring systems are available and deployed for large solar farms and have proved to increase the operational efficiency of those systems, whereas there are no standard monitoring systems for small-scale installations. This paper presents a survey on Operation and Maintenance (O&M) of PV systems and an IoT based Wireless Sensor Network for PV panel monitoring.

1. Signature Analysis of Electrical Faults in Standalone PV Systems with Storage Sushmita Sarkar, K Uma Rao, Jayanth Bhargav, Shrikesh Sheshaprasad, Anirudh Sharma CA 2019 IEEE 3rd International Conference on Recent Developments in Control, Automation & Power Engineering (RDCAPE) [Abs] [PDF]

   Stand-alone PV systems are increasingly becoming popular as a viable alternative to utility power supply, especially in remote corners, where accessibility is a problem to deploy conventional distribution systems. It is also becoming a choice of conscious consumers who wish to contribute to environment protection through use of green energy. In this context, it is vital to have a fault diagnosis system in place to identify various faults in the PV system. This paper presents a signature analysis of various electrical faults in a stand-alone PV system, with battery storage. The focus of the work is to minimize the sensors to be deployed for detection and classification of the faults.

1. Parametrics for the choice of Embedded Systems and Control Algorithms for Application Specific Robot Designs Jayanth Bhargav, Shourya TR, Arti Anantharaman, C Chinmay, K Uma Rao, Girish Rao Salanke 2018 IEEE 2nd International Conference on Inventive Systems and Control (ICISC) [Abs] [PDF]

   Micro controller contains one or more processor cores along with peripheral devices and memory. Micro-controllers find extensive use in embedded system applications such as automobile engine control systems, bio-medical devices, electronic appliances, etc. They are specific to a particular task, are small and compact in size, and consume less power; all of these features have resulted in the ubiquitous deployment of micro-controllers in various devices. However, it must be noted that a microcontroller is a Control Circuit, and not a Driving Circuit. This statement is supported by the fact that the output current of a microcontroller is too small to drive a motor connected to it directly; in fact, this is an unpropitious experiment and can damage the microcontroller. The microcontroller controls the working of actuators connected to it by following the logic coded onto it, and in this sense, forms the very brain of the embedded system. But in order to drive the motor, we need a type of switch (MOSFET, Relay, etc.) which can take the small output current of the microcontroller and control the power source to the motor. Essentially, a microcontroller accepts real-time inputs and generates an output control signal as per the rules and predefined logical deductions, in the prescribed algorithm. This paper discusses the criterion one must consider for the choice of an embedded system from the plethora of systems available in the market and the choice of control technique for different robotic applications.

Journal Articles

1. A model for effect of partial shading on PV panels with experimental validation Sushmita Sarkar, K. Uma Rao, Prema V., M.S. Bhaskar, Sanjeevikumar Padmanaban,Jayanth Bhargav, Energy Reports, 12 (2024) : 3460-3469 [Abs] [PDF]

   This paper aims to develop and validate an empirical model to quantify the impact of partial shading on photovoltaic (PV) panel performance. Partial shading, a significant challenge in solar power generation, can drastically reduce energy output, yet predicting its effects remains difficult using conventional models. This study introduces a methodology that models partial shading as an equivalent reduction in solar insolation across the entire panel. The model was derived from empirical data gathered under various shading scenarios, and curve-fitting techniques were employed to establish an empirical relationship between shading patterns and power loss. The proposed model was experimentally validated on a rooftop PV system in Bengaluru, India, and demonstrated precise predictions of power loss under different partial shading conditions. The framework presented can be adapted to accommodate location-specific variables, providing a versatile tool for evaluating and optimizing PV system performance in partially shaded urban environments.

1. Novel Modular LS-SVM Based Regression Model for Prediction of Solar Power Sushmita Sarkar, K Uma Rao, Prema V., Jayanth Bhargav, Shourya T.R. International Journal of Engineering Sciences and Management [Abs] [PDF]

   PV power penetration is increasing rapidly. Hence, its impact on the grid operation becomes a major concern for the utilities. Predictive models of renewable energy are an essential requirement to analyze this impact. Design of an accurate, effective and powerful model for forecasting is a highly challenging task in weather dependent renewable energy sources. This paper proposes a novel modular Least Square SVM (LS-SVM) based Regression model for solar-power forecast. The solar power is calculated using predicted value of solar irradiance. This method highly reduces the computation complexity and time. The proposed model uses a Time-Series decoupling strategy where instead oftreating the historical data as a single time series, the data is split into multiple time series. Each time series carries the historical data of solar irradiance at a particular time of the day during the period under consideration. The model has shown promising results with a MAPE of 3.5% for a day ahead prediction

Book Chapters

1. User Interactive GUI for Integrated Design of PV Systems Sushmita Sarkar, K Uma Rao, Prema V, Anirudh Sharma C A, Jayanth Bhargav, Shrikesh Sheshaprasad Intelligent Renewable Energy Systems, Wiley Online Library
   [Abs] [PDF]

   All over the world, penetration of solar PV in electrical grids has seen a steep increase in the last decade. This includes both low power solar roof top installations and large-scale solar farms. Consciousness about climate change and encouraging government policies especially in developing countries have made investment in solar PV farms an attractive option for private investors. Further large amount of infertile land in tropical countries has also boosted installation of PV farms in such spaces. Two important decisions for any PV system are the space required for panel installation and the cost incurred. The cost depends on the type of installation. Popular models are standalone PV micro grid, large farms and that are grid-tied and those that are not tied to the grid. Based on the model, the required power electronics has to be designed. Currently, there are no open-source design tool to help the consumer/investor to design the specifications and ratings of different components. In this paper, a unique and a novel GUI is presented to design the entire solar PV systems. Further break-up of cost components, standards and configuration for inverters are also presented in the GUI.