(Yes, I'm working today ...)
One of the unusual features of SF and fantasy genre literature is that they rely on worldbuilding to an unusual degree. (I note in passing that the SF Encyclopedia doesn't have an entry for worldbuilding yet, hence the wikipedia definition.)
When we write fiction in the realist mode, set in the present day, we can rely on the author and reader's shared experience of the world they live in to sketch in a lot of background details. There's no need for a mainstream author to work out the constitutional structure of the British government, for example, unless they're writing a thriller about British constitutional law; likewise, they can refer to a Ford Transit in the knowledge that readers will know what one of those is. (Except they can't — the brand name applies to two hilariously different families of vehicles in Europe and the United States. But that's a localization issue.)
This consensus reality breaks down under certain conditions. American readers need background details if they're reading books set in Europe, and vice versa. Western culture readers again need more orientation cues if reading a book set in, say, Afghanistan or Botswana. Names, places, the appearance of everyday things from roads to government buildings to cooking pots — the physical trappings of human cultural life — may warrant description that can be elided if the setting is familiar.
If you leave out the background details, a reader unfamiliar with the culture being depicted may have problems. When I was a pre-teen I didn't know that in America, people drove on the right; consequently, any number of action scenes in books didn't make sense to me. I lacked an essential piece of consensus knowledge. This isn't to say that it's necessary to state explicitly which side of the road your protagonist is driving in every work of fiction — but when I write I always have the ghost of a notional target reader hovering behind my shoulder, and occasionally they ask, "what's that?" And I have to go back and describe something or other that previously I had merely named, and which they are unfamiliar with.
And that's just the physical stuff: what's inside human heads is much stranger. People who speak different languages, adhere to (or grew up with) different religions or belief structures, or who simply come from parts of the world where the flow of history is different (for example, where there has been a civil war or military coup or invasion in their own lifetime) will have different attitudes and outlooks to people in the relatively rich and peaceful nations of the EU and North America. Do people in this culture trust their government to protect them, or fear it as an imposition? Do they trust strangers in the street to share their own culture and assumptions, or do they remember the bad time 25 years ago when fascists with clubs were battling students with bricks in the street and anyone might be an informer who could have you dragged away and tortured in the Colonels' prisons on the basis of a rumour?
If you're writing a mainstream novel for, say, a British or US audience that is set in 1970s Greece or Argentina, you're going to have to deal with the fact that the British and American experiences of government in that decade, turbulent and confrontational though they were, vanish into insignificance compared to tanks in the streets and Dirty War disappearances and mass graves. And your audience are going to need a whistle-stop tour of those aspects of the setting, insofar as they impact your story.
We haven't got to SF and fantasy yet. Nowhere near, in fact.
Historical fiction is the first genre we come to where world-building is essential to the reader's comprehension. Not only are the physical trappings of life (clothing, transport, houses) significantly different in appearance and design, but many of the conveniences of modern life are absent — requiring, in turn, awareness on the author's part of the in-conveniences they freed us from. And that's before we get into the characters' heads.
The past is another country — go back just 1-2 human lifespans (150 years) and it's an alarming one. Around 50% of newborn infants died before the age of 5, so merely achieving a stable population required a TFR over 4.0. Life expectancy once people were past the terrible toddler years was better than many imagine, but nevertheless infectious diseases killed up to 30% (a figure today associated with cancer in the developed world). Energy sources other than muscle power were crude, inefficient, and rare: much labour was hard, grinding manual work. And today's basic social security structures didn't exist anywhere prior to the 1870s. In combination, these factors imposed severe economic and personal constraints on how people lived; today's Islamic Republic of Iran is, in many ways, less alien to us than Great Britain in the 1850s. That's without considering the other aspects of "1850s Britain" as a world-building exercise that an author trying to depict it to modern eyes would have to take into account. And it only gets worse the further back and further away you go.
So what of the literature of the fantastic?
I propose that worldbuilding is the primary distinguishing characteristic of SF and fantasy (at least at a superficial level). Get the worldbuilding wrong, and your readers won't be able to get a grip on the story line or the motivation of your characters. Or worse — they'll get a grip, and realize that your story is, at best, a western or an age-of-sail yarn with the serial numbers filed off: that the trappings of the fantastic are only there to add a spurious sense of exoticism to an everyday tale.
Worldbuilding in classical or high fantasy usually creates societies that in structure and trappings resemble ones drawn from history; indeed, if you take an historical novel and add homeopathic doses of magic you get something like Harry Turtledove's Videssos books, and if you take a fantasy novel and strip out the magic you might end up with a blow-by-blow re-enactment of the Wars of the Roses.
Worldbuilding in urban fantasy ... well, there usually isn't very much of it: urban fantasy is for the most part our own inhabited world, with added supernatural fauna.
Science fiction usually attempts to extrapolate in the opposite direction. But we have no cognitive models for societies that have access to technologies that don't exist yet, so our visualization of future cultures is frequently cloudy. A Victorian novelist might come up with the idea of antibiotics — drugs that magically banish infectious diseases — as a conceit for their great novel set in the 1990s, but they're unlikely to anticipate the resultant demographic transition, the rise of nursing homes as warehouses for the elderly, or second wave feminism. (All of which, arguably, are among the second-order effects of the huge reduction in infant mortality between 1900 and 1950.) Similarly, we can look at the emergence of 3D printers and Maker culture in the west as a side-effect of the outsourcing of bulk mass production to global centres of excellence and the hollowing-out of our own local industrial base ... but how do we guess what it's going to lead to, or what it's going to make possible? What the political and social consequences are going to be?
(I'm going to get round to that in the next part of this ongoing brain dump, when I get round to writing it.)