Thanks all. Yeah I've just noticed there is a earth metal lug behind that connector.
So long story short, everything looks fine? The light being slightly on when it's off is probably a good thing anyway, call it a night light feature.
Edit: what is that yellow wire, if not earth? And why is it...