Wow, that's some story and thanks for sharing.
Be careful that you aren't being ‘played' here though. There are some guys that want it all i.e. more than one woman they can turn to when one woman ‘goes cold'. He sounds a little that way from what you describe but I be mistaken.
The real point of all this is that neither of you have been clear on your emotional state and how you feel. That very situation is only going to confuse and muddy the relationship, if there actually is one.
At the end of the day, you need to be clear of where you stand, what you feel and how you'd like things to move AND you need to understand the same things from him. If he is still elusive, take heed.
Get clear so you have some foundational information to make a judgement of what and where you go next. Anything less is will keep you where you are now – confused.