The presence of your badges shows that you are not an accidental visitor to the resource, which limited himself to one question, received an answer and left forever - you are an active member of the community
You don’t have to take care of the badges - they simply accumulate over time, signal your activity and usefulness, add credibility to your answers (you must admit that the answer from the veteran looks more convincing than from the beginner).
And yes - your history can be viewed. Does it bother you? Did you do something unworthy, which should be ashamed?