I didn't think the OP was saying go out and steal the clothes. You could borrow and return...which, I agree isn't really that great because A) I wouldn't like someone borrowing my clothes without permission and B) I might be looking for something that you were still borrowing and C) I may wonder why my clothes are stretched or fit differently.
Also, I have let my SO borrow quite a few of my clothes...which, turned out to be me giving him the clothes because he can't afford to spend a lot of money right now and just giving the clothes to him was a lot easier when we no longer were able to live together (school reasons).
But, I mean, when he first started out...what else could he do? He was broke. He didn't have any other female friends or relative to borrow from...so, I mean where is the harm in that case?