Some work with a grinder and that back box would be out in 5 minutes, okay there's plaster damage and redecoration to consider, but the best way would be to replace it with a correct sized box.
As long as you are not needing an earth via the conduit and will have circuit protection elsewhere (RCD / RCBO) I pretty sure the old conduit doesn't need connection to the new box.
Another alternative would be to have a shallow
surface back box fitted over the old back box. You could get some self tapping 30mm screws, drill 2 holes in the old steel back box and fix the new box to the old and use the back box centre for cable access.
Not ideal, but the new box footprint would cover the old box- I wouldn't do it for a client, but other than going down the road of chopping and cutting the old box out, I can't see any better alternative.
Opps, maybe just one. If you measure the centre of the switch screw holes and the gap between the two, then measure between the edges of the existing box- Is the box gap wider?
If so you could tap thread the back box and use long face plate screws to fix on to the cut thread holes.