[kaldodson]

I feel like if I was in your situation I would be feeling the same thing you would.
I would also probably make the same decision you are thinking about making.
The only thing is don't make the decision to leave him, until you talk to him!
Maybe he has no clue how you feel. Let him know what is up. Then make a rational decision after that.