Not too long ago, someone in the twittersphere asked, "Whatever happened to psi? It used to be all the rage in science fiction."
The answer, essentially, was that John Campbell died and nobody believes in that crap any more. And anyway, it's fantasy.
Now here's the thing. If you accept Clarke's Third Law, which boils down in the common wisdom to "Any sufficiently advanced technology is indistinguishable from magic," you kind of have to ask, "Do we believe psi is crap because it really is crap, or do we just not have the technology to detect or manipulate it?"
Yes, of course, that way lies madness. But with quantum physicists messing around with teleportation, and computer engineers inching toward a technological form of telepathy, are we really that far off from making at least part of the Campbellian weirdness a reality?
And if that's the case, where did the psi go? It's no more improbable than the ftl drive that's a staple of the space-opera canon. Why is ftl still a thing, but psi is now subsumed under "Magic, Fantasy, Tropes of"?
Maybe because science fiction is about the hardware, and fantasy is about the wetware? Faster-than-light travel may be presumed to need some form of machine to happen. Psi, by contrast, is an organic phenomenon. Generally it's considered to originate from some form of human or alien (or, since it's fantasy now, magical or elven or similarly fantasy-focused creature) brain.
When I was a very young writer, a baby for a fact, I sent my first novel--all 987 space-and-a-half 10-point-typed pages of it--to the late, great Lester del Rey. He sent it back with a three-page letter, kindly and reasonably rejecting it, but encouraging me to keep writing, because There Was Hope. The line I remember most clearly from that letter was the one that defined his main reason for passing on the submission: "Fantasy readers seem to be tolerant of science fiction in their fantasy, but science-fiction readers will not stand for fantasy in their science fiction."
This was when Anne McCaffrey's dragons were still mostly considered science fiction, because alien planet and genetic engineering and John Campbell, and Darkover was in full swing and Andre Norton was mixing hardcore nastytech with her Witches. But the lines were already hardening, and the categories were just beginning to set in cement--not least through the efforts of the Del Reys, who were just getting rolling with the fantasy boom of the Eighties. By the time the Nineties rolled in, McCaffrey was fantasy because dragons, and Bradley and Norton were in the middle somewhere but "Boys Write SF, Girls Write Fantasy," and Bradley had done The Mists of Avalon, so there we all were. With Fantasy now a major category of its own, and Science Fiction sticking to its own shelves in the bookstores.
It's interesting that even while the categories separated for ever and aye, or at least until the Publishing Apocalypse changed everything, the writers stood up and said, "HEY! We still want to be together!" And the Science Fiction Writers of America became the Science Fiction and Fantasy Writers of America, and fantasy started getting nominated for Nebulas (and wasn't that a tempest in the tiny teapot), though science fiction couldn't (and still can't) be nominated for the World Fantasy Award. But horror can, and is, so there's some cross-fertilization there, too.
What happened here was that what used to be all one column had become, for marketing purposes, Column A and Column B. Column A: Future, technology (usually high), time travel (if by mechanical means), alien planets, space travel, and so on. Column B: Past or secondary worlds, low tech (or at least lower than the present day, though there's also urban fantasy, which hits most of the other checkboxes), dragons and elves and other mythical beings, time travel or portal travel (if by magical/nontechnological means), magic--and, as a subset thereof, the mind powers known in earlier science fiction as psi, etc., etc.
So Pern's lost Earth colony was labeled fantasy, between the dragons and the psi. Darkover? Um, yeah. Fantasy. Low tech (albeit voluntary) and psi, despite the central theme of conflict between high and low tech in a spacefaring future. (And yet Dune is still science fiction in spite of the psi and the weirdness. Higher ratio of spacefaring culture to low-tech planet? Post-technological vibe? Male author?)
What this did to younger writers was lock in the categories and make it difficult to impossible to sell work that crossed the lines. Female writers were pushed to heighten the romance and emphasize the fantasy elements, and many were actively discouraged from venturing into science fiction. The freewheeling nature of the old, smaller, still evolving field had both hugely expanded in numbers and sales reach, and distinctly contracted in the range of what was allowable in worldbuilding and storytelling. Categories solidified, and to some extent ossified.
I wonder if Steampunk is in some ways a reaction to this. The closer modern technology gets to Clarke's threshold, the more alluring it can be to focus on gears and levers and automata. They're accessible; they don't spin off into quantum bizarrerie. And god forbid, they don't disappear into the Singularity.
Still, there's psi on the fantasy side of the divide, with the "MAGIC" label slapped over it. The author who gets psi in her space adventure (notwithstanding the Force or the Betazoids) may meet with fastidious flinching and "we can't sell this." The categories are firm, and while there's "interstitial" and "intergenre," those are narrowly defined and equally specific. You can have a science-fiction mystery or a cyberpunk space opera, but a trope from fantasy Column B in your science-fiction Column A? Not so much. Especially if it also mixes up the age groups (YA? Adult? Both? Neither?).
The ebook boom and the rise of independent publishing--now well on its way to respectability--has been a serious game-changer for authors who can't or won't color correctly inside the lines. Marketing categories still prevail, but there's much more choice and far fewer restrictions. If even a few readers will read it, pretty much anything goes. Even science fiction with fantasy cooties. Or technology that's crossed the line into magic. Or psi powers. With or without the help of technology.
So maybe psi in science fiction will come back. We're seeing so many different variations on the genre now, and so much exuberance, and a good amount of crossing over and some work that isn't even categorizable (Martha Wells' Raksura, anyone?). Why not a new vogue for mind powers in our science-fictional worlds?