I decided to use a syringe to fill the largest of the holes. Bust didn't work. Next I broke out my viberator sander. It is an old sander that the pad is not even good on. It really vibrates.
I used a plastic feed sack to protect the sander. Mixed grout to thin peanut butter. Smeared it like a thick sandwich. Laying the bag over and started vibrating. As it dried out I misted it with water to keep it soupy. I vibrate it over and over for maybe 5 minutes.