Safety is #1. You can never be too safe with firearms. I would definitely move that berm so that you can shoot in a safer direction with a good backstop.

Last year at thansgiving, we had a family gathering on our 40 acres...long story short, a cousin was taking pictures by the pond and was hit by a stray bullet in the arm. They said it was a small caliber...most likely .22. Passed all the way through. Never found out where it came from.