Take a sharpened metal scraper or razor blade and hold it at a 45° angle. Pull the razor using short, overlapping strokes (about a half inch at a time) to remove all material that is protruding above the base of the snowboard. Use whatever direction works best.

The razor is sharp and can cut you if you are not careful.

You can also wrap sandpaper around a block and rub the area to flatten it completely.