We introduce a novel and generic convolutional unit, <monospace>DiCE</monospace> unit, that is built using dimension-wise convolutions and dimension-wise fusion. The dimension-wise convolutions apply light-weight convolutional filtering across each dimension of the input tensor while dimension-wise fusion efficiently combines these dimension-wise representations; allowing the <monospace>DiCE</monospace> unit to efficiently encode spatial and channel-wise information contained in the input tensor. The <monospace>DiCE</monospace> unit is simple and can be seamlessly integrated with any architecture to improve its efficiency and performance. Compared to depth-wise separable convolutions, the <monospace>DiCE</monospace> unit shows significant improvements across different architectures. When <monospace>DiCE</monospace> units are stacked to build the <monospace>DiCENet</monospace> model, we observe significant improvements over state-of-the-art models across various computer vision tasks including image classification, object detection, and semantic segmentation. On the ImageNet dataset, the <monospace>DiCENet</monospace> delivers 2-4 percent higher accuracy than state-of-the-art manually designed models (e.g., MobileNetv2 and ShuffleNetv2). Also, <monospace>DiCENet</monospace> generalizes better to tasks (e.g., object detection) that are often used in resource-constrained devices in comparison to state-of-the-art separable convolution-based efficient networks, including neural search-based methods (e.g., MobileNetv3 and MixNet).