Reinforcement Learning in Energy Trading Game among Smart Microgrids

Huiwei Wang, Tingwen Huang, Xiaofeng Liao, Haitham Abu-Rub, Guo Chen

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

Reinforcement learning (RL) is essential for the computation of game equilibria and the estimation of payoffs under incomplete information. However, it has been a challenge to apply RL-based algorithms in the energy trading game among smart microgrids where no information concerning the distribution of payoffs is a priori available and the strategy chosen by each microgrid is private to opponents, even trading partners. This paper proposes a new energy trading framework based on the repeated game that enables each microgrid to individually and randomly choose a strategy with probability to trade the energy in an independent market so as to maximize his/her average revenue. By establishing the relationship between the average utility maximization and the best strategy, two learning-automaton-based algorithms are developed for seeking the Nash equilibria to accommodate the variety of situations. The novelty of the proposed algorithms is related to the incorporation of a normalization procedure into the classical linear reward-inaction scheme to provide a possibility to operate any bounded utility of a stochastic character. Finally, a numerical example is given to demonstrate the effectiveness of the algorithms.

Original languageEnglish
Article number7460097
Pages (from-to)5109-5119
Number of pages11
JournalIEEE Transactions on Industrial Electronics
Volume63
Issue number8
DOIs
Publication statusPublished - 1 Aug 2016

Fingerprint

Reinforcement learning

Keywords

  • Energy trading game
  • incomplete information
  • reinforcement learning (RL)
  • smart microgrids (MGs)

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Reinforcement Learning in Energy Trading Game among Smart Microgrids. / Wang, Huiwei; Huang, Tingwen; Liao, Xiaofeng; Abu-Rub, Haitham; Chen, Guo.

In: IEEE Transactions on Industrial Electronics, Vol. 63, No. 8, 7460097, 01.08.2016, p. 5109-5119.

Research output: Contribution to journalArticle

Wang, Huiwei ; Huang, Tingwen ; Liao, Xiaofeng ; Abu-Rub, Haitham ; Chen, Guo. / Reinforcement Learning in Energy Trading Game among Smart Microgrids. In: IEEE Transactions on Industrial Electronics. 2016 ; Vol. 63, No. 8. pp. 5109-5119.
@article{d3192b9411e5478caa842831f2eaeb5e,
title = "Reinforcement Learning in Energy Trading Game among Smart Microgrids",
abstract = "Reinforcement learning (RL) is essential for the computation of game equilibria and the estimation of payoffs under incomplete information. However, it has been a challenge to apply RL-based algorithms in the energy trading game among smart microgrids where no information concerning the distribution of payoffs is a priori available and the strategy chosen by each microgrid is private to opponents, even trading partners. This paper proposes a new energy trading framework based on the repeated game that enables each microgrid to individually and randomly choose a strategy with probability to trade the energy in an independent market so as to maximize his/her average revenue. By establishing the relationship between the average utility maximization and the best strategy, two learning-automaton-based algorithms are developed for seeking the Nash equilibria to accommodate the variety of situations. The novelty of the proposed algorithms is related to the incorporation of a normalization procedure into the classical linear reward-inaction scheme to provide a possibility to operate any bounded utility of a stochastic character. Finally, a numerical example is given to demonstrate the effectiveness of the algorithms.",
keywords = "Energy trading game, incomplete information, reinforcement learning (RL), smart microgrids (MGs)",
author = "Huiwei Wang and Tingwen Huang and Xiaofeng Liao and Haitham Abu-Rub and Guo Chen",
year = "2016",
month = "8",
day = "1",
doi = "10.1109/TIE.2016.2554079",
language = "English",
volume = "63",
pages = "5109--5119",
journal = "IEEE Transactions on Industrial Electronics",
issn = "0278-0046",
publisher = "IEEE Industrial Electronics Society",
number = "8",

}

TY - JOUR

T1 - Reinforcement Learning in Energy Trading Game among Smart Microgrids

AU - Wang, Huiwei

AU - Huang, Tingwen

AU - Liao, Xiaofeng

AU - Abu-Rub, Haitham

AU - Chen, Guo

PY - 2016/8/1

Y1 - 2016/8/1

N2 - Reinforcement learning (RL) is essential for the computation of game equilibria and the estimation of payoffs under incomplete information. However, it has been a challenge to apply RL-based algorithms in the energy trading game among smart microgrids where no information concerning the distribution of payoffs is a priori available and the strategy chosen by each microgrid is private to opponents, even trading partners. This paper proposes a new energy trading framework based on the repeated game that enables each microgrid to individually and randomly choose a strategy with probability to trade the energy in an independent market so as to maximize his/her average revenue. By establishing the relationship between the average utility maximization and the best strategy, two learning-automaton-based algorithms are developed for seeking the Nash equilibria to accommodate the variety of situations. The novelty of the proposed algorithms is related to the incorporation of a normalization procedure into the classical linear reward-inaction scheme to provide a possibility to operate any bounded utility of a stochastic character. Finally, a numerical example is given to demonstrate the effectiveness of the algorithms.

AB - Reinforcement learning (RL) is essential for the computation of game equilibria and the estimation of payoffs under incomplete information. However, it has been a challenge to apply RL-based algorithms in the energy trading game among smart microgrids where no information concerning the distribution of payoffs is a priori available and the strategy chosen by each microgrid is private to opponents, even trading partners. This paper proposes a new energy trading framework based on the repeated game that enables each microgrid to individually and randomly choose a strategy with probability to trade the energy in an independent market so as to maximize his/her average revenue. By establishing the relationship between the average utility maximization and the best strategy, two learning-automaton-based algorithms are developed for seeking the Nash equilibria to accommodate the variety of situations. The novelty of the proposed algorithms is related to the incorporation of a normalization procedure into the classical linear reward-inaction scheme to provide a possibility to operate any bounded utility of a stochastic character. Finally, a numerical example is given to demonstrate the effectiveness of the algorithms.

KW - Energy trading game

KW - incomplete information

KW - reinforcement learning (RL)

KW - smart microgrids (MGs)

UR - http://www.scopus.com/inward/record.url?scp=84978880192&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84978880192&partnerID=8YFLogxK

U2 - 10.1109/TIE.2016.2554079

DO - 10.1109/TIE.2016.2554079

M3 - Article

VL - 63

SP - 5109

EP - 5119

JO - IEEE Transactions on Industrial Electronics

JF - IEEE Transactions on Industrial Electronics

SN - 0278-0046

IS - 8

M1 - 7460097

ER -