Yes, it's made out of people, but I don't mean the food - I mean the society in which any SF novel takes place. (I could've quoted Margaret Thatcher instead, about there being no such thing as society, but only to negate her sentiment.) It's something like the Gestalt notion of perception, of seeing a foreground object only against its background, content in its context - characters in their culture.
Of course a novel's scientific extrapolation, whether wild or logically reasoned, may already drive the fictional culture in a particular direction. Even so, you want something twisty and interesting, rather than straightforward. For me, much of the background is a natural part of the tapestry - in other words, as flashes of scenes come to me, the people's interactions are driven by how they are related. Those relationship types might be peculiar to the context: officer/private, master/slave.
I'm not really a political writer, I don't think, but my most overtly political world-building belongs to my books set furthest in the future (a 35th-century colony world, isolated for 1200 years) and those set closest to the present day - somewhere between 10 and 30 years from now.