remove grout

I tried to re grout some tiles so I scraped of some of the old grout using a screw driver. Unknown to me the new grout I used was past it's shelf life so its ended up drying like chalk. I've tried to remove this grout but it's a real pain. I can't seem to get rid of the residue from this...