When a current flows through a resistor, the resistance causes power to be dissipated. This power is I*I*R and will cause the resistor to get warm. So, to answer your question, yes, it should get warm.
As to whether its "too warm", well a resistor has a power rating, 0.6W, 1W, 2W are common...