So this week the usual folks have been all over China's proposal to use big data techniques to assign every citizen a Citizen Score. And while a tiny ethics-free part of my soul weeps for joy (hey, I never expected parts of Glasshouse to come true!) the rest of me shudders and can't help thinking how much worse it could get.
So, let's start by synopsizing the Privacy Online News report. It's basically a state-run universal credit score, where you're measured on a scale from 350 to 950. But it's not just about your financial planning ability; it also reflects your political opinions. On the financial side, if you buy products the government approves of your credit score increases: wastes of time (such as video games) cost you points. China's main social networks feed data into it and you can lose points big-time by expressing political opinions without prior permission, talking about history (where it diverges from the official version—e.g. the events of 1989 in Tiananmen Square—hey, I just earned myself a negative credit score there!), or saying anything that's politically embarrassing.
The special social network magic comes into play when you learn that if your friends do this, your score also suffers. You can see what they just did to you: are you angry yet? Social pressure is a pervasive force and it's going to be exerted on participants whether they like it or not, by friends looking for the goodies that come from having a high citizen score: goodies like instant loans for online shopping, car rentals without needing a deposit, or fast-track access to foreign travel visas. Also, everyone's credit score is visible online, making it easy to ditch those embarrassingly ranty cocktail-party friends who insist on harshing your government credit karma by not conforming.
The gamification of social conformity, overseen by an authoritarian government and mediated by nudge theory, is a thing of beauty and horror; who needs cops with nightsticks to beat up dissidents when their friends and family will give them a tongue-lashing on behalf of the government for the price of a discount off a new fridge?
But don't worry, I could make it a whole lot worse.
The first notable point about this system is that it's an oppressive system that runs at a profit. Consider the instant no-colateral loans for online shopping: the Chinese system only grants these to folks who are a good credit bet. The debt will be repayed. Meanwhile it goes into providing a Keynsian stimulus for the productive side of the economy. And it rewards people for political right-thinking. What's not to like?
Governments love nudge theory because it offers a cheap shortcut to enforcing social policy, even when the social policy in question is utterly broken. Paying a cop costs money—not just their salary and the cost of their uniform, but the station they work out of, the support personnel who keep the police force operating (janitors, human resources, vehicle maintenance), and the far less tangible political cost of being seen to wield a big stick and force people not to do what they want to do (or to do things that you want them to). Using big data to give folks a credit score, then paying them bright and shiny but essentially cost-free bonuses if they do what you want? That's priceless. You may not be able to track folks who like to toke up directly (if it's illegal in your jurisdiction), but you can penalize them for hanging out with known cannabis users and buying paraphernalia. More to the point, you can socially isolate users and get their family to give them grief without the unpalatable excesses (and negative headlines) of no-knock raids and cops kicking down the wrong door and shooting children by mistake. One may ask whether the medical marijuana movement and decriminalization pressure would have got off the ground in the United States if a citizenship scoring system with downvotes for pot users was in place. Or whether emancipatory rights movements could exist at all in a society that indirectly penalizes people for "wrong lifestyle choices" rather than relying on imperfectly applied but very visible and hateful boots and nightsticks.
Let's look at some other pooled-risk areas.
Take car insurance. Traditionally your premium is reduced if you don't make any claims against it, and if you avoid racking up any tickets for speeding or bad driving that happens to be spotted by a traffic cop. Historically, claims might be made for having a window broken, or being in a fender bender. Insurance is supposed to pool risk, but some market segmentation is permitted—otherwise those of us who drive responsibly would be forced to carry the irresponsible minority.
More recently, we've met insurance policies that give us a discount for good driving, as monitored by in-car black boxes with GPS and accelerometers that determine whether we're accelerating too hard or breaking speed limits. These systems are always on—they don't depend on you misbehaving in front of a traffic officer. Rather than rewarding good behaviour directly coupled to the system (you don't make any claims on your insurance so you can buy it at a discount) they actively punish bad behaviour—even if you don't make an insurance claim or get ticketed for speeding your premium goes up if you habitually drive too fast. Being a safe, fast, driver is not measurable, so we settle for measuring the trait that usually correlates with risk and extrapolate from there.
Now take health insurance. (Or, if you live, like me, in a country with a national healthcare system that has a single comprehensive payer, the health system.) There are periodic suggestions that we should punish bad behaviour, behaviour that increases medical costs: Scotland has an alcoholism problem so we get the Alcohol (Minimum Pricing)(Scotland) Act, 2012. Obesity comes with its own health risks, and where resource scarcity exists (for example, in surgical procedures), some English CCGs are denying patients treatment for some conditions if they are overweight.
It should be argued that these are really stupid strategies, likely to make things worse. Minimum alcohol pricing is regressive and affects the poor far more than the middle-class: it may cause poor alcoholics to turn the same petty criminality observed among drug addicts, to fund their habit. And denying hip replacements to overweight people isn't exactly going to make it easier for them to exercise and improve their health. But because we can measure the price of alcohol, or plot someone's height/weight ratio on a BMI chart, these are what will be measured.
It's the classic sylogism of the state: something must be controlled, we can measure one of its paramenters, therefore we will control that parameter (and ignore anything we can't measure directly).
Now, what else can governments do with this tech?
First a micro-example: The Chinese government could conceivably to abolish it's Great Firewall once the citizen score is enacted. Instead, it could require ISPs to log all outgoing internet connections; the UK's GCHQ already does this via the KARMA POLICE program (and that name could be a big hint about where this is going). By monitoring what people are looking at, you can then reward or punish their habits. The 50 Cent Party demonstrates that they've got the human resources to actively track internet activities; members could be rewarded for identifying hostile foreign web sites, and non-members could then earn penalty points on their citizen scores for looking at those sites. By rendering the firewall transparent they could paradoxically improve enforcement: looking at dodgy sites on the internet would get you shunned by family, friends, and workmates out of self-interest.
So a committed government program could apply deep social pressure towards conformity while giving the appearance of lightening up on oppression and encouraging transparency.
But maybe you don't need a travel visa or a cheap loan for a new iPad. What leverage does our system retain over you?
Back to healthcare, because it's a solid lever—we all need it sooner or later. So here's another micro-example: It's believed that owning a pet improves happiness and life expectancy, but some pets may have deleterious side-effects. Buying products indicative of a pregnancy at the same time as feline care items (cat litter, cat food) could therefore get you a big negative citizenship score, or a visit from the local community nurse and some advice about re-homing your pet. Buying too much chocolate and too little kale? Ditto, only with advice about healthy eating and a warning about your access to healthcare being cut back if you don't comply.
Healthcare so obviously stands beside Mom and Apple Pie in the good citizenship stakes that tying it into your citizenship score is a no-brainer. And it can be fed not only from your medical records and the costs you have incurred in healthcare provisioning, but by using data from your cloud-mediated smart lifestyle monitors. your Fitbit could snitch on you to the Party (or, if you want to sugar-free-sweeten the pill, reward you with a point on your citizenship score every time you hit your recommended daily activity target).
Pollution is not only a social evil, but a personal and direct threat to your health, and your health is a pillar of your good citizenship. So I think it's inevitable that smart electricity meters and the internet of things will be deployed in an anti-pollution/consumption mode: it would be trivial to punish folks for leaving light bulbs or heaters on in rooms that they don't occupy. Less obviously, they can be deployed to enforce personal hygeine—or at least to cost you karma for not showering daily (or for slothfully lazing around in the bath).
But so far I've only considered the prospects for authoritarian but relatively modern regimes that are trying to enforce the sort of behaviour we don't really disagree with.
It'd be interesting (in a gruesome sort of way) to see what Da'esh (or the government of Saudi Arabia) could do with a citizen score. Currently enforcement of public morality in hardcore Salafi muslim states is carried out by the Committee for the Promotion of Virtue and the Prevention of Vice in Saudi Arabia, and other religious police in other states. As with all police forces, there is a cost associated with putting boots on the ground. If you have, for example, a modest dress code, you could go some way towards enforcement by feeding purchases of garments into the citizen's score. (Buy too much of the wrong kind of underwear and you could be singled out for an in-person check by the mutaween. And heaven forbid they catch you streaming music from a western cloud service.) Signs of non-conformity could be punished indirectly: it's a lot harder to resist ubiquitous peer pressure than it is to dodge external resource-limited law enforcement.
In The Handmaid's Tale, Margaret Atwood's Republic of Gilead subordinates women rapidly by taking control over the financial system. But that's a comparatively crude mechanism. The more data you've got, the more tightly you can constrain your reward/punish metrics and the more accurately you can focus your oppression—and micro-focussed oppression minimizes the risk of generating wide-scale resistance. Everybody's experience is different, isolated, locked inside an invisible cell with asymmetric walls that their neighbors can't see. And if you can't see the invisible walls locking your neighbours in, you can't establish solidarity and exert collective pressure against them.
We are heading towards a situation where we all carry smartphones, all the time; where we need them to call a cab, or check a bus timetable, or unlock our cars, or pay for something. Your smartphone knows who you are, knows where you've been, reads all your correspondence, and hears everything you say. The discrete activity of placing a voice phone call is in the process of replaced by barking "phone, put me through to Sandy in Sales", followed by rapid connectivity (unless Sandy is in do-not-disturb mode or talking to someone else, in which case their phone will take a message for you). With always-on recognition, your phone (without which you can't really exist in an internet-of-things world) will track your mood and your pulse rate and possibly award you citizenship points or penalties if you respond to the wrong stimuli.
But that's the nightmarish, dystopian grim-meathook-future version of citizenship scoring: a system that facilitates the pervasive enforcement of mandated behavioural standards and punishes quantifiable expressions of individuality. Nobody would vote for (or buy into) that! So it's going to be even more gamified, to make it fun. You can see your score in real time, get helpful tips on what to do (or not to do) to grind for points, and if you're thinking about doing something a bit naughty a handy app will give you a chance to exercise second thoughts and erase your sin before it is recorded. But that's not all. Obviously you didn't really want to date that manic pixie dream girl (she'll murder your citizenship score with her quirky and unpredictable fun transgressions) but we can apply the magic of Affinity Analysis to look for someone more suitable for you—similar preferences, similar tastes, and most importantly a similar attitude to social improvement and good citizenship.
Now eat your greens; your phone says you haven't been getting your five a day this week and if you keep it up we're going to have to dock you a point.