You can't really do anything about that. I'd suggest maybe hanging a stand 10' on your side and hunting it just as often as he hunts his. Come walking in after daylight a couple times on him and he'll get out. Just lock your stands. You may also give him a great opportunity to steal something of yours if he's that kind of person.