JOINT DESIGN OF USER ASSOCIATION AND HYBRID BEAMFORMING METHOD AND SYSTEM FOR SUB-THz UDN USING MULTI-AGENT DEEP REINFORCEMENT LEARNING

Disclosed is a method and system for interference control and hybrid beamforming using multi-agent deep reinforcement learning for multiple users. The disclosed method includes performing multi-agent reinforcement learning using interference and antenna gain information that is expected based on a gain table designed using channel state information (CSI) of all user equipments; searching for an analog beamforming matrix pair corresponding to a link that maximizes antenna gain and minimizes interference between user equipments through multi-agent reinforcement learning; applying a signal-to-leakage plus noise ratio (SLNR) maximization technique that minimizes the interference between the user equipments based on the link; and optimizing transmission (Tx) power for each link based on iterative water-filling.