Thanks all for the advice.
I'll try fixing a nut to the back of the lug.
I turned the power off, connected the plate and then turned the power back on. You don't get any shock.
It was still sparking when I touched the back box with the face plate, but it was instant, it didn't keep...