In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminates

Inverse reinforcement learning (IRL) aims to learn a reward function and a corresponding policy that best fit the demonstrated trajectories of an expert. However, current IRL works cannot learn incrementally from an ongoing trajectory because they have to wait to collect at least one complete trajectory to learn. To bridge the gap, this paper considers the problem of learning a reward function and a corresponding policy while observing the initial state-action pair of an ongoing trajectory and keeping updating the learned reward and policy when new state-action pairs of the ongoing trajectory are observed. We formulate this problem as an online bi-level optimization problem where the upper level dynamically adjusts the learned reward according to the newly observed state-action pairs with the help of a meta-regularization term, and the lower level learns the corresponding policy. We propose a novel algorithm to solve this problem and guarantee that the algorithm achieves sub-linear local regret $O(\sqrt{T}+\log T+\sqrt{T}\log T)$. If the reward function is linear, we prove that the proposed algorithm achieves sub-linear regret $O(\log T)$. Experiments are used to validate the proposed algorithm.

Paper

References (100)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer L1zi4/10 · confidence 4/52024-06-18

Summary

In this paper, a novel framework for performing Inverse Reinforcement Learning (IRL) from an ongoing trajectory, i.e., to learn a reward function that induces an optimal policy that best explains the expert's demonstrations sequentially, without waiting to having observed the expert's trajectory entirely. Authors propose an online learning algorithm for solving the problem, and this algorithm has the added feature of meta-regularization. Both a theoretical and empirical analysis of the algorithm are provided to validate the results.

Strengths

- The framework of learning from ongoing trajectories is interesting and finds some applications.

Weaknesses

- There is no clear discussion and intuition of what kind of reward function is extracted by the algorithm and what it can be used for. For instance, various works about identifiability or the feasible set concern what we can do with the extracted rewards, because in most cases we cannot just extract a single reward explaining the demonstrations. However, this work does not analyse this fact. - The presentation of the contents and contributions is poor.

Questions

- What reward are you extracting among all the rewards that explain the observed demonstrations? What can it be used for? - Since in RL what matters is the cumulative sum of rewards, how can you be sure that you extract a meaningful reward even though you have not observed the expert's policy for the entire horizon?

Rating

4

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

The authors have adequately addressed the limitations and potential negative societal impact of their work.

Reviewer cmJ68/10 · confidence 4/52024-07-12

Summary

The paper "Learn Incrementally from An Ongoing Trajectory: A Provable In-Trajectory Inverse Reinforcement Learning Framework" proposes an innovative approach to inverse reinforcement learning. The authors introduce an online learning algorithm to address the IRL problem with incomplete expert demonstrations. A thorough theoretical analysis guarantees convergence, and a comprehensive experimental section validates the approach.

Strengths

1.Very interesting problem and strong theoretical analysis. 2.The paper introduces a practical algorithm that appears easy to reproduce and yields good results.

Weaknesses

1.In the experiments, I don’t understand why different algorithms have different initial points? When t=0, are you using a same random reward function or the prior-meta reward function to train your policy?

Questions

1. Can the author explain why comparing the suffixes \(\{s_i', a_i'\}\) and \(\{s_i'', a_i''\}\) encourages the reward function to predict the future? Since you are completing the incomplete trajectory using the same policy, can reward function can learn something even if both are generated by the same policy. 2. Just wondering how you complete the incomplete trajectories in MuJoCo using saved states, since the state in MuJoCo only includes partial information about the simulator.

Rating

8

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

The authors address the limitations of this work.

Reviewer MWk17/10 · confidence 4/52024-07-12

Summary

The authors consider a new problem setting, called in-trajectory IRL, where a reward function and a corresponding policy need to be learned from an ongoing trajectory. The authors propose a novel reward update mechanism specially designed for this scenario and incorporate a meta-regularization strategy to embed prior knowledge and avoid overfitting. Theoretical analysis is provided for the convergence of the algorithm towards stationary points (and global under linear reward parametrization). MuJoCo benchmark and a real-world stock market example are used for experiments.

Strengths

1. The topic is novel to IRL community. The new problem setting of learning the reward function and policy on the fly aligns with many real-world scenarios. In this new situation, standard IRL algorithms are not suitable anymore. 2. The special reward update design of predicting the future trajectory is reasonable and interesting for this ongoing trajectory setting. 3. The meta-regularization is novel to IRL, which helps to better find a reward from limited data. 4. The theoretical analysis is solid and tackles the issue of non i.i.d. input data. The theoretical distinction from literature is well clarified. The experiment includes real-world stock market data.

Weaknesses

1. Assumption 1 assumes that the parameterized reward is smooth, which can be a strong assumption, especially for neural networks with non-smooth activation functions such as ReLU. 2. The meta-regularization part seems to be important to the performance. However, the meta-learning needs to pre-collect data for many related tasks, which can be difficult in real world.

Questions

How the authors collect data for the meta-regularization in the experiment? Other questions please refer to the Weaknesses.

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have discussed the limitations in Appendix F.

Reviewer cmJ62024-08-08

I thank the authors for responding to my comments and providing these clarifications. I am happy to increase my score, assuming you can incorporate the global clarifications into the text.

Reviewer MWk12024-08-08

Thanks for the detailed response. The authors have done a great job answering my questions. I have no concerns nor questions about the assumption and the meta-regularization part now. I thus increase my rating. The meta-regularization is novel to IRL and the in-trajectory learning is an intereting new problem setting. I recommend acceptance of this paper.

Reviewer L1zi2024-08-09

I thank the authors for the time taken to answer to my questions. However, I am still convinced of the score that I assigned to this paper, because I do not understand why it should be useful to learn a reward function incrementally, given that this increases the usual identifiability problems of IRL. In addition, the problem setting is written in a not clear manner, that does not permit to fully understand the paper.

Authorsrebuttal2024-08-10

Dear reviewer, Thanks for mentioning (1) why it should be useful to learn a reward function incrementally given that this increases the reward unidentifiability issue, and (2) the writing of the problem setting. We address these two comments below. **For the comment (1)**, we answer from two aspects: (i) the connection between learning a useful reward and reward identifiability, and (ii) why the incrementally learned reward is useful. (i) The fundamental usefulness of the learned reward is to explain the expert demonstrations. As mentioned in the first paragraph in our previous response, solving the reward identifiability issue is sufficient but not necessary to learn a reward function to explain the demonstrations. We agree that learning a reward incrementally can make it harder to solve reward unidentifiability issue. However, our algorithm can still learn a reward that explains the expert demonstrations despite the learned reward function may be different from the ground truth. In fact, shown in Table 1 in our paper, the optimal policy corresponding to the learned reward can generate trajectories whose cumulative ground truth reward is close to the expert cumulative ground truth reward, i.e., the learned policy generates trajectories close to the expert trajectories. (ii) Moreover, compared to imitation learning that directly learns an imitating policy, our learned reward can be used for conterfactual analysis [C8], e.g., instantly estimating the policies under different environment dynamics. Take the stock market as an example, the reward function captures the investor's risk tolerance [C11]. In our experiment, we learn the investor's risk tolerance incrementally in the stock market of Dow Jones Industrial Average (DJIA). The incrementally learned reward can instantly update the risk tolerance of the target investor. The instantly updated reward has two benefits. First, we can use the latest reward to recommend stocks as soon as possible, which gives us advantages to win this customer before our competitors. Second, we can use the latest reward to estimate the investment behaviors preferred by the target investor in some other stock markets, e.g., CCXT (cryptocurrency) and QuantConnect (US securities) [C12], so that we can not only recommend stocks in the stock market where we collect data, i.e., DJIA, but also recommend stocks in other stock markets, e.g., CCXT and QuantConnect. If we use imitation learning, we can definitely learn an instant imitating policy and use this imitating policy to recommend stocks of DJIA as soon as possible. However, the imitating policy learned by imitation learning cannot be used to recommend stocks in CCXT and QuantConnect because they have different dynamics. In contrast, we can still apply the learned reward to these two new markets to learn the corresponding optimal policies and use the corresponding optimal policies to recommend stocks, because the investor's risk tolerance does not change even if the stock markets are different. We include an additional experiment result below where we use the data collected in DJIA to learn the reward function of the target investor and then apply the learned reward to CCXT and QuantConnect. We compare the cumulative reward between the optimal policy under the learned reward and the expert in these two different markets. | | CCXT | QuantConnect | |:---:|:---:|:--:| | Learned policy under the learned reward | 562.18 ± 34.02 | 491.72 ± 18.25 | | Expert | 571.83 ± 27.16 | 502.82 ± 14.26 | From the results above, we can see that even if in new stock markets with different dynamics, the learned reward from DJIA can still be used to generate policies in the new stock markets and the generated policies can still achieve similar cumulative ground truth reward with the expert in the new stock markets. This shows that the learned reward successfully captures the investor's risk tolerance. **For the comment (2)**, we will add more explanations about the problem formulation (2)-(3) as follows: (i) we will highlight that the method we use to learn a reward function is maximum likelihood IRL (ML-IRL) and provide the intuition of ML-IRL where we aim to learn a reward function that makes the demonstration most likely; (ii) we will show our connection to the standard IRL where our problem formulation (2)-(3) will reduce to the formulation of ML-IRL when the complete trajectory can be observed; (iii) we will discuss the motivation to learn a reward function incrementally. [C11] Qizhou Sun, X. Gong, and Y.-W. Si, "Transaction-aware inverse reinforcement learning for trading in stock markets," Applied Intelligence, 2023. [C12] Xiaoyang Liu, et al. "FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning." NeurIPS, 2022.

Reviewer L1zi2024-08-12

Dear authors, I had already read your comments. Although I appreciated the additional comments, I have decided to keep the score I assigned.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC