Passive flow control and heat transfer enhancement technique has become an attractive method for device internal cooling with low resistance penalty. In the present paper, the flow and heat transfer characteristics in the small scale rectangular channel with different groove–protrusions are investigated numerically. Furthermore, the combination effect with ribs is studied. The numerical results show that on the groove side, the flow separation mainly occurs at the leading edge, and the reattachment mainly occurs at the trailing edge in accordance with the local Nusselt number distribution. On the protrusion side, the separation mainly occurs at the protrusion back porch and enhances the heat transfer at the leading edge of the downstream adjacent groove. The rectangle case provides the highest dimensionless heat transfer enhancement coefficient Nu/Nu0, dimensionless resistance coefficient f/f0, and thermal performance (TP) with the highest sensitivity of Re. When ribs are employed, the separation bubble sizes prominently decrease, especially inside the second and third grooves. The Nu/Nu0 values significantly increase when ribs are arranged, and the one-row case provides the highest heat transfer enhancement by ribs. Besides, the two-row case provides the highest Nu/Nu0 value without ribs, and the three-row case shows the lowest Nu/Nu0 value whether ribs are arranged or not.