I'm in a similar position though not because I gut it too big.
When changing the lamp a load of the plasterboard crumbled and fell away leaving the unit hole that's too big, not perfectly round and has a ragged edge of plasterboard and the paper (?) that was left behind.
Any suggestions...