@oliver @shel say that your Asian friends say it's racist.
Also explain that often people of colour have to concede things in social settings in order to fit in sometimes ("Oh yeah, it's totally not racist Maykayleighlough. I'm not saying this to avoid making a scene or anything")