But, you need to make a stance, and tell him that it's disrespetful for YOU for him to hint of your sex life to other men.. It's your private world with him and you'd like him to stop it.
Be wary but also be mindful that if he's in-secure, then he may use words in play, meaning the exact opposite to try to pose as being "tough" the bad guy not wanting to let his true feelings be known in case you see him as weak and leave him, given he slipped the marriage word out
CW
Comment