Why worry? Do you expect the interview will held in the buff? As Kitten said, "It's all about discretion." Don't get a tattoo on the back of your hand or your face and expect to get a job as a Stock Broker or a sales job in Ferrari. Otherwise purs...
Thinking about it..... and wanting it badly enough to do the work are two different things. Maybe he's just not into the idea as much as he thought or you'd like him to be. Let it go. It'll either happen or it won't.