I am assuming you are a young chap? it helps to look fit and strong, with hardened hands that show you are not afraid of work. If you can keep busy with jobs like gardening, laying paths, clearing rubble - even if only for your own family - it marks you out as the right sort of person. Dress the part as if you were already doing that kind of job.
As well as approaching people that don't know you, on site or around the builders merchant, see who you know in the building trade. Uncles, friends of your parents, neighbours, people down the pub or the church, depending on your lifestyle. It will mostly be people of an older generation that can help, and you may not usually spend much time with them, but provided they think you are a reliable, honest, hard-working person who can get out of bed in the morning, they will be willing to help with suggestions, and probably also put in a good word for you. People like to lend a hand, and an older tradesman will feel good if he sees that his experience and contacts are being valued.
be aware though that if they think you are not, they will not want to damage their own reputation and relationships with people they know, by recommending you, and then have you turn out to be useless.
I don't know about labouring jobs in particular, but that's how it works when you're starting out. Personal recommendation, and looking the right sort, is best.
