Cheers for that John,
I'm going to try filling it once more, as it can't do any harm, and it may work, as it's not the main weight bearing bracket, the outer brackets are screwed in tight.
If it pulls out, then, yes I guess I'll have to raise the pole up the wall, going to look a bit daft, but...