print page

Master's Thesis

Reinforcement Learning with Artificial Neural Network to control Water Heaters for Demand Response


In electricity networks produced and consumed power is the same at any point in time. Therefore, traditionally, power plants are regulated to produce the amount of consumed power. But now, the increasing share of renewable sources, produce according to the weather not as needed. The approach demand response (DR) contributes to the issue by changing the consumption of devices in time. Suppliers encourage their customers to change their consumption by providing time-varying prices. With day-ahead real-time-pricing, suppliers fix these prices once every day for the next day.

Smart devices may directly receive price information and autonomously optimize their electricity costs. In this work a domestic electric water heater (DEWH) is considered as smart device, which may reduce its operating costs by preheating to higher temperatures at lower prices. However, the water temperature has to remain in certain limits to maintain the consumer’s comfort.

Cost optimal heating profiles may lead to load synchro- nization and large peak demands. Such behaviors may be counter-productive for the energy supplier. Thus an objective is to create a control method, which finds a trade-off between cost minimization, consumer’s comfort and supplier friendly behavior.

Task Description

In this work a reinforcement learning (RL) approach with an artificial neural network (ANN) is considered to function as a DR-controller for a DEWH. The ANN has to learn beneficial heating decisions depending on the received price information, current water and environment temperature, the expected future water demand of the consumer, and the feedback for previous decisions. It shall be analyzed whether this approach allows to reduce consumer’s costs while also maintaining the comfort.

In the next step, the resulting behavior is rated from the supplier’s perspective and can be integrated as another feedback to the RL. However, for privacy concerns, the supplier should only know the aggregated load profile of many devices instead of all individual profiles. Thus, it shall be analyzed if a global feedback can improve the aggregated load profile of many devices.

Start date 24. January 2018
End date 31. August 2018
Documents Flyer
Supervisor Dr.-Ing. Tobias Lübkert