Managing caching strategies for stream reasoning with reinforcement learning

Efficient decision-making over continuously changing data is essential for\nmany application domains such as cyber-physical systems, industry\ndigitalization, etc. Modern stream reasoning frameworks allow one to model and\nsolve various real-world problems using incremental and continuous evaluation\nof programs as new data arrives in the stream. Applied techniques use, e.g.,\nDatalog-like materialization or truth maintenance algorithms to avoid costly\nre-computations, thus ensuring low latency and high throughput of a stream\nreasoner. However, the expressiveness of existing approaches is quite limited\nand, e.g., they cannot be used to encode problems with constraints, which often\nappear in practice. In this paper, we suggest a novel approach that uses the\nConflict-Driven Constraint Learning (CDCL) to efficiently update legacy\nsolutions by using intelligent management of learned constraints. In\nparticular, we study the applicability of reinforcement learning to\ncontinuously assess the utility of learned constraints computed in previous\ninvocations of the solving algorithm for the current one. Evaluations conducted\non real-world reconfiguration problems show that providing a CDCL algorithm\nwith relevant learned constraints from previous iterations results in\nsignificant performance improvements of the algorithm in stream reasoning\nscenarios.\n Under consideration for acceptance in TPLP.\n