How do I avoid these?
I get those too! For me, it was caused when I did a v cut using an end mill to clear out the larger areas. The 1/4 and 1/8 end mills I use (Amana) do not have flat ends. Because of this, if there is an area to clear where the end mill does not move laterally, it leaves these bumps (or nipples as my wife calls them).
The shape left matches the shape of the end mill’s tip. To avoid this, use a smaller end mill for clearance. If that is not possible, do not use a clearance and let the v bit do all the work.
I’ll have to pick up a flat end 1/8. V carve would take too long. Thanks for the insight I was hoping someone had some magic saw dust I could sprinkle to have them go away. 
It is not clear what the “this” is - the side walls look like chatter to me (though the picture is quite blurry) - slow down your feed rate or decrease your DOC. If you are talking about the dimples in the corners, not sure. Maybe the work piece is not secured properly or flat to the work surface?
-Tom
What about ramping moves added to the toolpath?