Problem With Disc Caliper

Have a pulsar150 UG3 2 yrs old. The disc calipers which are mounted on the aluminum disc have a little play. So whenever the bike goes on a small bump the calipers hit the fork and make a sound. this sound is getting pretty irritating. I gave the bike to castrol for servicing and they said that the play is due to a rubber cap on the screw mounting of the disc caliper and thats the reason why the calipers are not rigid. He told me that its not a big problem but honestly the sound is very irritating on the ears when u are riding.

On close inspection of the disc calipers I realized that Bajaj has completely redesigned the calipers for the 150 UG3 with the black silencer model which almost eliminates this problem. now should i go in and change the entire disc brake? or just live with this noise?