Data-Dependent Bounds for Online Portfolio Selection Without Lipschitzness and Smoothness

This work introduces the first small-loss and gradual-variation regret bounds for online portfolio selection, marking the first instances of data-dependent bounds for online convex optimization with non-Lipschitz, non-smooth losses. The algorithms we propose exhibit sublinear regret rates in the worst cases and achieve logarithmic regrets when the data is"easy,"with per-iteration time almost linear in the number of investment alternatives. The regret bounds are derived using novel smoothness characterizations of the logarithmic loss, a local norm-based analysis of following the regularized leader (FTRL) with self-concordant regularizers, which are not necessarily barriers, and an implicit variant of optimistic FTRL with the log-barrier.

Paper

References (41)

Scroll for more · 29 remaining

Similar papers

Peer review

Reviewer Xwzr7/10 · confidence 2/52023-07-04

Summary

The paper considers the online portfolio selection problem. In this problem, one must allocate funds between d possible investment choices, with the goal of maximizing the total amount. In each round, the "success" of each choice is revealed, in the form of a ratio between new and old price, called price relative; these price relatives are adversarially chosen. The goal is to have low regret, in terms of the logarithm of total wealth, against any constant, fractional allocation of funds between the investment choices. In general, where the number of rounds is T, existing work for this problem exhibits either polynomial running time in T and poly-logarithmic regret in T, or running time independent of T and regret polynomial in T (specifically, square root). The paper considers a different approach, providing data-dependent bounds for the problem. The paper presents three bounds w.r.t. the input: small-loss bound, w.r.t. the total loss of the optimal allocation; gradual-variation bound, w.r.t. the volatility of the gradients of the loss functions; and a second-order bound, w.r.t. some second-order statistics of the loss function. The first two appear in the body of the paper. The paper presents an algorithm based on optimistic FTRL w.r.t. a log-barrier regularizer; the optimism refers to using an estimate for the upcoming loss function in choosing the allocation for the next time step. To my understanding, the algorithmic contribution of the paper is in the method for choosing the loss estimate. Comments: Line 33: Maybe "lower" instead of "faster" Line 283: In the displayed equation, should be a_t instead of a

Strengths

The online portfolio selection is an interesting problem, and obtaining data-dependent bounds for this problem seems natural. In that sense, the paper does a comprehensive job and provides bounds w.r.t. three parameters.

Weaknesses

I have a concern regarding the small-loss bound, addressed in the questions for rebuttal.

Questions

Regarding Assumption 1, the paper correctly states that this could be ensured by normalizing the price-relatives vector, without affecting regret. However, this does affect the total loss of the adversary, which comes into play in the small-loss bound in Theorem 6.2. If I understand correctly, the assumption is thus nontrivial to make, which weakens Theorem 6.2. Is this the case?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

yes

Reviewer q9Qg6/10 · confidence 4/52023-07-05

Summary

This work studies online portfolio selection (OPS) problem and establishes regret bounds that is square-root dependent on some data-dependent quantities, namely, the cumulative loss of the best action or the variation of the gradients respectively, without lipschitzness or smoothness assumption. Although previous studies obtain regret bounds with logarithmic dependence on the same data-dependent factors, they require the lipschitzness assumptions, which this work circumvents by investigating the problem carefully and proposing alternative local-norm lemmas as substitutes for lipschitzness and smoothness conditions. Furthermore, this work designs a novel optimistic FTRL algorithm with a self-concordant function to cooperate with the local-norm techniques and validates that the prediction can be resolved in $\tilde{O}(d)$ time pre round.

Strengths

1. This paper leverages the inherent structure of the problem and derives the first data-dependent regret bounds without lipschitzness or smoothness assumptions for the OPS setting. 2. The proposed optimistic FTRL algorithm is novel and interesting, which facilitates the application of local-norm techniques with optimism for this problem. 3. This paper is clearly written and presented well.

Weaknesses

The OPS problem is fundamentally challenging, and I am glad to witness advancements in this field even if the results have not yet reached an optimal level. Below are some questions that emerged when I am reviewing the paper, and I would be appreciated if the authors can provide some feedback. 1. Ordinarily, one might anticipate that data-dependent bounds would demonstrate a marked advantage over minimax optimal results in the best-case scenarios, such as when $V_T = 0$ or $L_T^\star = 0$. But, by Theorem 6.1, when $ V_T = 0$, the algorithm can only assure $O(d\log T)$ regret bound, just matching the minimax optimal bound. By Theorem 6.2, $L_T^\star = 0$ will imply an even less optimal bound than the minimax result. However, these results do attain Pareto frontier optimality when balancing the optimality and efficiency under the best cases. I advise the authors to elaborate further on the motivation and highlight the efficiency of the proposed algorithms to make the contributions clearer. 2. As the authors noted in line 242, the term "implicit" might potentially mislead readers since it is commonly associated with another class of algorithms [Kulis and Bartlett, 2010]. It may be beneficial to consider a different nomenclature for the algorithm. 3. I’m interested in the relationship between two data-dependent bounds. Indeed, some earlier study (see Zhao et al., 2021, Theorem 6) posits that a gradual variation dynamic regret can imply a small-loss dynamic regret in the analysis, for general convex, non-negative, and smooth functions. This result is also true for static regret. A natural question that arises from this is whether it's feasible for one algorithm to concurrently secure guarantees for both the gradual-variation bound and small-loss bound. References: Brian Kulis and Peter L. Bartlett. Implicit Online Learning. ICML 2010. Peng Zhao, Yu-Jie Zhang, Lijun Zhang, and Zhi-Hua Zhou. Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimization. ArXiv:2112.14368, 2021.

Questions

See comments above.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Area Chair 66Gx2023-08-16

Please acknowledge rebuttal

Dear reviewer, Please acknowledge that you have read the rebuttal and indicate whether it adequately addresses your comments. The author-reviewer discussion period ends Aug 21; please engage with the authors before that if needed. Thanks, AC

Reviewer cmFV7/10 · confidence 2/52023-07-06

Summary

The paper presents beyond-the-worst-case regret bounds for Online Portfolio Selection (OPS). In general online learning, beyond-the-worst-case bounds are established using structural assumptions on the loss functions, such as Lipschitzness and smoothness, but the loss functions in OPS are neither Lipschitz nor smooth, which is the main technical difficulty the paper addresses. To this end, local norm analogues of Lipschitzness and smoothness are established for OPS, and a generic optimistic FTRL algorithm with the log-barrier regularizer is proposed. Specializations of this algorithm achieve a gradient-variation-dependent bound and a $L^*$ bound, which are the first without the additional no junk bond assumption. In the worst case, these bounds match a type of classical regret-computation tradeoff. In better cases, these bounds are logarithmic in $T$, which is a substantial acceleration.

Strengths

- Online portfolio selection is an iconic problem in online learning, with plenty of recent progresses in its worst case characterization. The paper introduces classical types of data dependent bounds to this problem, which has a very clear and natural motivation. - The challenge of non-Lipschitzness and non-smoothness is technically nontrivial. The solution relies on establishing local counterparts of these properties, which is interesting, and could be of broader applicability. - The specific data-dependent bounds are novel without the additional no junk bond assumption. - Related works are discussed in depth, which gives the new contributions a nice context.

Weaknesses

Overall this is a good paper, and there isn't any major criticism I'd like to make. On less important issues, - The technical presentation of the paper could be improved. Currently there are quite a few of typos and unclear notations. For example, there's Algorithm 1 in the main paper, and an algorithm 1 in the appendix. $\omega$ in Theorem 3.2 is not defined in the main paper. The notation $x(i)$ is used in line 201 but defined in line 227, ... - I think compared to data-dependent bounds in general online learning, the benefit of data-dependent bounds in OPS is a little less clear, particularly due to the existence of a computationally inefficient logarithmic regret algorithm. I would appreciate more discussions on computation, since the paper essentially considers a particular regret computation tradeoff. Is $\tilde O(d)$ runtime and $\tilde O (\sqrt{dT})$ regret pareto-optimal for OPS in some sense? A numerical example would also be helpful, as it will make the computational savings over universal portfolio (and line 33 to 36) more clear.

Questions

- Could the authors comment a bit more on the novelty of the smoothness characterizations (Lemma 4.6 and 4.7)? - I'm also generally wondering about the regret computation tradeoff in OPS. Are there lower bounds?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The limitations are adequately addressed.

Reviewer DgZU6/10 · confidence 2/52023-07-11

Summary

The paper studies follow the regularized leader algorithm (FTRL) on the online portfolio selection problem without the assumption of no junk bonds. This makes the resulting loss function non-Lipschitz and non-smooth and makes analyzing the regularized follow the leader algorithm hard to analyze. The paper proposes using a self-concordant regularizer for the FTRL algorithm and proves regret bounds for the algorithm for a generic online convex optimization problem. They then use the result to propose two novel algorithms that have small-loss and gradual-variation regret bounds, respectively, for the online portfolio selection problem, which to the best of the author’s knowledge are the first regret bounds for non-Lipschitz and non-smooth losses.

Strengths

The paper seems significant as it provides a new result towards bounding regret for non-Lipschitz and non-smooth losses. The main contribution seems to originate from being able to prove that FTRL with a self-concordant regularizer without the barrier requirement obtains a regret bound similar to the setting with the barrier requirement. They then show how to apply the result to the online portfolio selection problem which is a canonical online convex optimization problem. The paper provides clear background for the problem to help motivate the goal and significance of the result.

Weaknesses

The paper lacks clarity in how exactly Theorem 3.2 is applied to the online portfolio selection problem. Specifically, it is unclear what conditions are needed to apply Theorem 3.2 as Algorithm 1 is applied to a very specific online convex optimization problem. I believe Section 4 tries to highlight these conditions via Lemma 4.4, 4.5, 4.6, and 4.7, but without looking at the appendix, it is unclear where the conditions factor into proving the regret bounds. Moreover, in Section 5 where Theorem 3.2 is applied to the online portfolio selection problem, it is very unclear why the implicit optimistic LB-FTRL Algorithm 2 solves it. The first challenge in understanding how Algorithm 2 maps the OPS problem to Algorithm 1 is in the limited explanation in the mapping. From my reading, it seems that the only explanation in the main body is in the second sentence of the first paragraph in Section 5, “By the convexity of the loss functions, OPS can be reduced to an online linear optimization problem described in Section 3 with $v_t = g_t$ and $\mathcal{X}$ being the probability simplex $\Delta$.” Writing the optimization problem explicitly would be more clear.

Questions

Questions 1. Is it possible to state Theorem 3.2 for a generic loss that satisfies some set of specific conditions? This might streamline sections 3 and 4 by highlighting the necessary conditions ahead of time. Additionally, it might reduce the repetition in redefining the problem since fundamentally the only thing that changed was the loss function. 2. Is there intuition how the newly proposed algorithms get around the non-Lipschitz loss? My intuition is that it depends on choosing the learning rate properly so that when the Lipschitz constant is large, the correct learning rate mitigates it. Is the intuition correct? This seems to imply the learning rate should change depending on where the location of the data point for FTL type algorithms.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The paper is generally more theory-oriented and authors imply it is unclear how the analysis can be generalized to other settings, thus adequately addressing the limitations.

Reviewer mCNh7/10 · confidence 4/52023-07-26

Summary

This paper studies how to achieve adaptive regret bound, including gradient-variation bound and small-loss bound, for the online portfolio management problem, without the classical no-junk bund assumption. The authors successfully achieve this goal by observing a new kind of smoothness for the function -log wx.

Strengths

The finding of a kind of smoothness of -log<wx> is the major contribution of the paper. I think this is a novel and significant observation, and lead to multiple new conclusions. Specially, for the small-loss bound, one can easily show a upper bound related to root{sum of gradients at x_t} by using a step size inversely proportional to root{sum of norm of gradients}. However, to get a small loss, bound, the next step is to assume smoothness to create a relationship between root{sum of gradients at x_t} and root{sum of function values at x_t} (using the classical Lemma 2.1 of [29]). In this paper, the authors show that, in the OPS problem, for the loss -log<wx>, we do not need smooth to construct this relationship. The key observation, to me, is Lemma 4.7, which shows a similar result to Lemma 2.1 of [29] with a “surrogate gradient”. In this way, the paper achieves the first small-loss bound for non-smooth functions, which is novel and interesting. For the gradient-variation bound, similar arguments also hold. Note that, with this observation, one can directly combine it with existing adaptive methods to achieve adaptive bounds. However, it does not mean the proposed methods are less novel. This paper is generally well-written and easy to follow. The key finding, section 4.3, is easy to understand.

Weaknesses

For exp-concave functions with bounded gradients (e.g., with no-junk bund assumption), algorithms such as variant of ONS () can get a O(log L^*) bound, while the algorithm in this paper yields a O(root{L^*} + log T) bound, which is sub-optimal. It would be great if the authors can consider how to get a best-of-both-world bound, which enjoys O(root{L^*} + log T) for non-smooth functions, and O(log L^*) for smooth functions. To obtain he optimal gradient-variation bound, the proposed algorithms need to know V_T in advance. In the appendix, the authors propose an algorithm that can learn the step size adaptively. However, it somewhat lost the good property of the gradient-variation bound when V_T is small.

Questions

N/A

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

N/A

Reviewer cmFV2023-08-11

I appreciate your rebuttal. It adequately addressed my comments. Although the benefit compared to the computationally inefficient log regret algorithm is still a bit murky, I agree that further studies are beyond the scope of this paper. Within the computationally tractable paradigm, the results are good contributions to the field.

Authorsrebuttal2023-08-14

Thank you

Thank you for the reply and appreciation of our work.

Reviewer mCNh2023-08-11

Thanks for the reponse

Thank you for the detailed response, and I do not have further questions. For the prior knowledge of V_T, I made a mistake and thought that the step size of Theorem 6.1 is related V_T (but its actually related to V_t). It would be great if the authors can emphasize this point in the revised version after Theorem 6.1. For the first point, I would like to clarify that my **main** point is to discuss the limitation of the current results, instead of suggesting a new direction, so I think it is related to the topic of this paper.

Authorsrebuttal2023-08-14

Thank you for the response

Thank you for the reply. We will emphasize that the step size in Theorem 6.1 does not need $V_T$ and discuss the possibility of obtaining a $O ( \log L_T^\star )$ regret bound in the revision. If we still have any misunderstandings, then please let us know. Thank you again for your appreciation of this work!

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC