Collaborative inference enables resource-constrained edge devices to make\ninferences by uploading inputs (e.g., images) to a server (i.e., cloud) where\nthe heavy deep learning models run. While this setup works cost-effectively for\nsuccessful inferences, it severely underperforms when the model faces input\nsamples on which the model was not trained (known as Out-of-Distribution (OOD)\nsamples). If the edge devices could, at least, detect that an input sample is\nan OOD, that could potentially save communication and computation resources by\nnot uploading those inputs to the server for inference workload. In this paper,\nwe propose a novel lightweight OOD detection approach that mines important\nfeatures from the shallow layers of a pretrained CNN model and detects an input\nsample as ID (In-Distribution) or OOD based on a distance function defined on\nthe reduced feature space. Our technique (a) works on pretrained models without\nany retraining of those models, and (b) does not expose itself to any OOD\ndataset (all detection parameters are obtained from the ID training dataset).\nTo this end, we develop EARLIN (EARLy OOD detection for Collaborative\nINference) that takes a pretrained model and partitions the model at the OOD\ndetection layer and deploys the considerably small OOD part on an edge device\nand the rest on the cloud. By experimenting using real datasets and a prototype\nimplementation, we show that our technique achieves better results than other\napproaches in terms of overall accuracy and cost when tested against popular\nOOD datasets on top of popular deep learning models pretrained on benchmark\ndatasets.\n