publications

2024

1. 

   Microgrid Control Under Uncertainty

   Avishai Halev, Yongshuai Liu, and Xin Liu

   2024

   Abs PDF

   Microgrids – decentralized electrical grids that can function both in conjunc- tion with wide area macrogrids and without – are a powerful tool to address energy resiliency and climate change mitigation. Microgrid control, how- ever, remains a challenge; their bespoke nature and the existence of multiple sources of uncertainty lead to a control problem that traditional grid mod- eling and control techniques are ill-suited to handle. We build a microgrid interface that allows off-policy reinforcement learning to achieve performance better than model predictive control and rule based control benchmarks un- der battery model uncertainty and comparable performance with benchmarks under no uncertainty. We utilize domain randomization and random network distillation exploration strategies to improve reinforcement learning perfor- mance. We show that reinforcement learning achieves comparable or better performance on seven of ten analyzed microgrid scenarios of different archi- tectures in forecasts of varying lengths. Our model code is available at https: //github.com/ahalev/Microgrid-Control-Under-Uncertainty and our mi- crogrid simulator is available at https://github.com/ahalev/python-microgrid.

2023

1. 

   Infection prediction in swine populations with machine learning

   Avishai Halev, Beatriz Martínez-López, Maria Clavijo, and 6 more authors

   Scientific Reports, Oct 2023

   Abs PDF

   The pork industry is an essential part of the global food system, providing a significant source of protein for people around the world. A major factor restraining productivity and compromising animal wellbeing in the pork industry is disease outbreaks in pigs throughout the production process: widespread outbreaks can lead to losses as high as 10% of the U.S. pig population in extreme years. In this study, we present a machine learning model to predict the emergence of infection in swine production systems throughout the production process on a daily basis, a potential precursor to outbreaks whose detection is vital for disease prevention and mitigation. We determine features that provide the most value in predicting infection, which include nearby farm density, historical test rates, piglet inventory, feed consumption during the gestation period, and wind speed and direction. We utilize these features to produce a generalizable machine learning model, evaluate the model’s ability to predict outbreaks both seven and 30 days in advance, allowing for early warning of disease infection, and evaluate our model on two swine production systems and analyze the effects of data availability and data granularity in the context of our two swine systems with different volumes of data. Our results demonstrate good ability to predict infection in both systems with a balanced accuracy of 85.3% on any disease in the first system and balanced accuracies (average prediction accuracy on positive and negative samples) of 58.5%, 58.7%, 72.8% and 74.8% on porcine reproductive and respiratory syndrome, porcine epidemic diarrhea virus, influenza A virus, and Mycoplasma hyopneumoniae in the second system, respectively, using the six most important predictors in all cases. These models provide daily infection probabilities that can be used by veterinarians and other stakeholders as a benchmark to more timely support preventive and control strategies on farms.

2. Predicting antimicrobial resistance of bacterial pathogens using time series analysis

   Jeonghoon Kim, Ruwini Rupasinghe, Avishai Halev, and 6 more authors

   Frontiers in Microbiology, May 2023

   Abs PDF

   Antimicrobial resistance (AMR) is arguably one of the major health and economic challenges in our society. A key aspect of tackling AMR is rapid and accurate detection of the emergence and spread of AMR in food animal production, which requires routine AMR surveillance. However, AMR detection can be expensive and time-consuming considering the growth rate of the bacteria and the most commonly used analytical procedures, such as Minimum Inhibitory Concentration (MIC) testing. To mitigate this issue, we utilized machine learning to predict the future AMR burden of bacterial pathogens. We collected pathogen and antimicrobial data from \textgreater600 farms in the United States from 2010 to 2021 to generate AMR time series data. Our prediction focused on five bacterial pathogens (Escherichia coli, Streptococcus suis, Salmonella sp., Pasteurella multocida, and Bordetella bronchiseptica). We found that Seasonal Auto-Regressive Integrated Moving Average (SARIMA) outperformed five baselines, including Auto-Regressive Moving Average (ARMA) and Auto-Regressive Integrated Moving Average (ARIMA). We hope this study provides valuable tools to predict the AMR burden not only of the pathogens assessed in this study but also of other bacterial pathogens.

2021

1. Policy Learning with Constraints in Model-free Reinforcement Learning: A Survey

   Yongshuai Liu, Avishai Halev, and Xin Liu

   In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, Aug 2021

   Abs PDF

   Reinforcement Learning (RL) algorithms have had tremendous success in simulated domains. These algorithms, however, often cannot be directly applied to physical systems, especially in cases where there are constraints to satisfy (e.g. to ensure safety or limit resource consumption). In standard RL, the agent is incentivized to explore any policy with the sole goal of maximizing reward; in the real world, however, ensuring satisfaction of certain constraints in the process is also necessary and essential. In this article, we overview existing approaches addressing constraints in model-free reinforcement learning. We model the problem of learning with constraints as a Constrained Markov Decision Process and consider two main types of constraints: cumulative and instantaneous. We summarize existing approaches and discuss their pros and cons. To evaluate policy performance under constraints, we introduce a set of standard benchmarks and metrics. We also summarize limitations of current methods and present open questions for future research.

2020

1. pymgrid: An Open-Source Python Microgrid Simulator for Applied Artificial Intelligence Research

   Gonzague Henri, Tanguy Levent, Avishai Halev, and 2 more authors

   In NeurIPS 2020 Workshop on Tackling Climate Change with Machine Learning, Aug 2020

   Abs PDF

   Microgrids – self-contained electrical grids that are capable of disconnecting from the main grid – hold potential in both tackling climate change mitigation via reducing CO2 emissions and adaptation by increasing infrastructure resiliency. Due to their distributed nature, microgrids are often idiosyncratic; as a result, control of these systems is nontrivial. While microgrid simulators exist, many are limited in scope and in the variety of microgrids they can simulate. We propose pymgrid, an open-source Python package to generate and simulate a large number of microgrids, and the first open-source tool that can generate more than 600 different microgrids. pymgrid abstracts most of the domain expertise, allowing users to focus on control algorithms. In particular, pymgrid is built to be a reinforcement learning (RL) platform, and includes the ability to model microgrids as Markov decision processes. pymgrid also introduces two pre-computed list of microgrids, intended to allow for research reproducibility in the microgrid setting.

2018

1. 

   Bouncing ball on a vibrating periodic surface

   Avishai Halev, and Daniel M. Harris

   Chaos: An Interdisciplinary Journal of Nonlinear Science, Sep 2018

   Abs PDF

   We present an investigation of a partially elastic ball bouncing on a vertically vibrated sinusoidal surface. Following the work of McBennett and Harris [Chaos 26, 093105 (2016)], we begin by demonstrating that simple periodic vertical bouncing at a local minimum of the surface becomes unstable when the local curvature exceeds a critical value. The resulting instability gives rise to a period doubling cascade and results in persistent horizontal motion of the ball. Following this transition to horizontal motion, periodic “walking” states—where the ball bounces one wavelength over each vibration cycle—are possible and manifest for a range of parameters. Furthermore, we show that net horizontal motion in a preferred direction can be induced by breaking the left-right symmetry of the periodic topography.