MODEL PARAMETER SHARING BETWEEN INFERENCE APPLICATION INSTANCES IN PROCESSING UNIT OF INFORMATION PROCESSING SYSTEM

Techniques for model parameter sharing between inference model instances are disclosed. For example, a method performed by a first process obtains a representation of an inference model for which multiple instances of the inference model are to be executed on at least one processing unit. The method determines, from the representation of the inference model, one or more model parameters that are a pre-trained type of model parameter. The method allocates a shared memory for storing the one or more model parameters that are the pre-trained type of model parameter. The method stores the one or more model parameters that are the pre-trained type of model parameter in the shared memory for access by the multiple instances of the inference model to be executed on the at least one processing unit.