Hmm. I very much disagree with you here. I see what you're saying, and I think the privilege thing is an interesting and valuable thing to talk about, but I do not believe at all that this would (or does) make the sub-genre inherently sexist, particularly because the "time tourism" sub-genre has escapism and suspension of disbelief as part of its nature. I think the reason we don't have female-led stories is just because no one's written them yet.
I wrote a lot more in response, but it ended up about 3,000 words too long for a comment :), so I posted it as a blog entry. If you're interested:
http://www.slhuang.com/blog/2013/09/12/the-reason-escapist-time-travel-doesnt-star-women-is-that-we-havent-written-it-yet/
]]>