Electroosmosis is the main mechanism for flow generation in lab-on-a-chip (LOC) devices. The temperature rise due to the Joule heating phenomenon, associated with the electroosmosis, may be detrimental for samples being considered in LOCs. Hence, a complete understanding of the heat transfer physics associated with the electroosmotic flow is of high importance in design and active control of LOCs. The objective of the present study is to estimate the temperature rise and the thermal entry length in electroosmotic flow through rectangular microchannels, having potential applications in LOC devices. Along this line, the power-law rheological model is used to account for non-Newtonian behavior of the common biofluids encountered in these devices. A mixed type of thermal boundary condition is employed at the channel surface, instead of routinely presumed constant wall heat flux or constant wall temperature conditions. A finite difference-based numerical method is employed for solving the governing equations in dimensionless form. An approximate solution, based on the premise of a uniform temperature field throughout the channel cross section, is also obtained for the bulk mean temperature, which is found to be of high accuracy. This, accompanied by the assessments of the temperature profile, reveals that the temperature variations in the channel cross section are negligible, and as a result, the bulk mean temperature can be used as a very precise estimate of the maximum temperature in an LOC device. Moreover, the evaluation of the entry length shows that a thermally fully developed flow is hardly achieved in practical applications because of small length scales involved. Accordingly, the maximum temperature rise may significantly be smaller than what is calculated based on a thermally fully developed flow assumption.