We give an effective solution to the regularized optimization problem <inline-formula><tex-math notation="LaTeX">$g (\boldsymbol{x}) + h (\boldsymbol{x})$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>g</mml:mi><mml:mo>(</mml:mo><mml:mi mathvariant="bold">x</mml:mi><mml:mo>)</mml:mo><mml:mo>+</mml:mo><mml:mi>h</mml:mi><mml:mo>(</mml:mo><mml:mi mathvariant="bold">x</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="bai-ieq1-3215914.gif"/></alternatives></inline-formula>, where <inline-formula><tex-math notation="LaTeX">$\boldsymbol{x}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">x</mml:mi></mml:math><inline-graphic xlink:href="bai-ieq2-3215914.gif"/></alternatives></inline-formula> is constrained on the unit sphere <inline-formula><tex-math notation="LaTeX">$\Vert \boldsymbol{x} \Vert _{2} = 1$</tex-math><alternatives><mml:math><mml:mrow><mml:msub><mml:mrow><mml:mo>∥</mml:mo><mml:mi mathvariant="bold">x</mml:mi><mml:mo>∥</mml:mo></mml:mrow><mml:mn>2</mml:mn></mml:msub><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:math><inline-graphic xlink:href="bai-ieq3-3215914.gif"/></alternatives></inline-formula>. Here <inline-formula><tex-math notation="LaTeX">$g (\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>g</mml:mi><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="bai-ieq4-3215914.gif"/></alternatives></inline-formula> is a smooth cost with Lipschitz continuous gradient within the unit ball <inline-formula><tex-math notation="LaTeX">$\lbrace \boldsymbol{x} : \Vert \boldsymbol{x} \Vert _{2} \leq 1 \rbrace$</tex-math><alternatives><mml:math><mml:mrow><mml:mo>{</mml:mo><mml:mi mathvariant="bold">x</mml:mi><mml:mo>:</mml:mo><mml:mo>∥</mml:mo><mml:mi mathvariant="bold">x</mml:mi><mml:msub><mml:mo>∥</mml:mo><mml:mn>2</mml:mn></mml:msub><mml:mo>≤</mml:mo><mml:mn>1</mml:mn><mml:mo>}</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="bai-ieq5-3215914.gif"/></alternatives></inline-formula> whereas <inline-formula><tex-math notation="LaTeX">$h (\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>h</mml:mi><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="bai-ieq6-3215914.gif"/></alternatives></inline-formula> is typically non-smooth but convex and absolutely homogeneous, e.g., norm regularizers and their combinations. Our solution is based on the Riemannian proximal gradient, using an idea we call <italic>proxy step-size</italic> – a scalar variable which we prove is monotone with respect to the actual step-size within an interval. The proxy step-size exists ubiquitously for convex and absolutely homogeneous <inline-formula><tex-math notation="LaTeX">$h(\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>h</mml:mi><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="bai-ieq7-3215914.gif"/></alternatives></inline-formula>, and decides the actual step-size and the tangent update in closed-form, thus the complete proximal gradient iteration. Based on these insights, we design a Riemannian proximal gradient method using the proxy step-size. We prove that our method converges to a critical point, guided by a line-search technique based on the <inline-formula><tex-math notation="LaTeX">$g(\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>g</mml:mi><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="bai-ieq8-3215914.gif"/></alternatives></inline-formula> cost only. The proposed method can be implemented in a couple of lines of code. We show its usefulness by applying nuclear norm, <inline-formula><tex-math notation="LaTeX">$\ell _{1}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="bai-ieq9-3215914.gif"/></alternatives></inline-formula> norm, and nuclear-spectral norm regularization to three classical computer vision problems. The improvements are consistent and backed by numerical experiments. available at <uri>https://bitbucket.org/FangBai/proxystepsize-pgs</uri>.