Planning safe trajectories under uncertain and dynamic conditions makes the\nautonomous driving problem significantly complex. Current sampling-based\nmethods such as Rapidly Exploring Random Trees (RRTs) are not ideal for this\nproblem because of the high computational cost. Supervised learning methods\nsuch as Imitation Learning lack generalization and safety guarantees. To\naddress these problems and in order to ensure a robust framework, we propose a\nHierarchical Reinforcement Learning (HRL) structure combined with a\nProportional-Integral-Derivative (PID) controller for trajectory planning. HRL\nhelps divide the task of autonomous vehicle driving into sub-goals and supports\nthe network to learn policies for both high-level options and low-level\ntrajectory planner choices. The introduction of sub-goals decreases convergence\ntime and enables the policies learned to be reused for other scenarios. In\naddition, the proposed planner is made robust by guaranteeing smooth\ntrajectories and by handling the noisy perception system of the ego-car. The\nPID controller is used for tracking the waypoints, which ensures smooth\ntrajectories and reduces jerk. The problem of incomplete observations is\nhandled by using a Long-Short-Term-Memory (LSTM) layer in the network. Results\nfrom the high-fidelity CARLA simulator indicate that the proposed method\nreduces convergence time, generates smoother trajectories, and is able to\nhandle dynamic surroundings and noisy observations.\n