concrete brick post expansion

I have a 175 mm square wooden post that I am setting in a hole as shown in the really not to scale picture below!! The hole width is 175 mm from each side of the post except for the side that faces the brick pillar. The width of the hole here is 20mm. Can I just pour concrete into the hole...