A Reinforcement Learning Approach to Lightning Network Fee Policy

Abstract

We employ Reinforcement Learning (RL) to learn a profitable fee policy in payment channel networks. This is a well-known problem in the Bitcoin Lightning Network (LN) with no effective solutions. Evidently, suitable mechanisms for profitable fee policy can incentivize users to join the network and significantly enhance its functionality. So far, no such learning-based algorithms have been studied in this domain and current deployments rely only on heuristics. Hence, LN users who apply these methods do not earn sufficient income, negatively impacting the cost of opportunity. In this paper, we proposed LeVIN, a simulator based on real-world LN data. Accordingly, we applied on-policy RL methods to learn an effective fee policy in the real-world configuration of LN. We overcame fundamental challenges of this environment such as partial observability, reward sparsity, non-stationarity, and multi-objectiveness. Our experiments illustrate that RL agents significantly outperform the existing algorithms for the task of fee-setting in payment channel networks.