The photo looks fine, so doesn't give any clues. There is no inbuilt convector, it is only a standard storage heater. So as the right hand consumer unit to which it should be connected is turned off, it should not be working at any time. So, it must be connected to the left hand one. So as you have discovered it will work anytime but will be using full price electricity.
This is how it should work:
The consumer unit on the left is powered permanently and your lights, sockets, etc will all be connected to it. Turn that off, they will all go off. If you want to try it and see if the storage heater stops working too, you could, and it would prove that the heater is connected to it. But be careful if you do. The storage heater is full of 'bricks' which store up the heat and once warm they can stay warm for hours after it is actually switched off. Hence why they are called storage heaters. You would have to try this when it had cooled completely. Bear in mind through that your fridge / freezer etc., will also be off. You could check to see if any of the circuit breakers make mention of the 'storage heater' but it's unlikely.
The consumer unit on the right should only receive power charged at a cheaper rate during the 'off-peak' times when and it is there specifically for the storage heater (as the label above the circuit breaker confirms) but you say that this consumer unit has never been switched on. However, you tell us the heater is coming 'on' when you turn on the wall switch, so it is obvious that the storage heater is connected to somewhere else. Whether you have economy 7 or not is irrelevant at the moment and makes no difference, as the consumer unit is permanently 'off' anyway.
My guess is that someone has moved the storage heater wiring from the 'off-peak' consumer unit to the 'normal rate' consumer unit, which would mean it will heat up at anytime.
