Big Brother is Alive and Well in China
 
Imagine a world where many of your daily activities are constantly monitored and evaluated: what you buy at the shops and online; where you are at any given time; who your friends are and how you interact with them; how many hours you spend watching content or playing video games; and what bills and taxes you pay or don’t pay. Its not hard to picture, because most of that already happens, thanks to Google, Facebook, Instagram, Fitbit and others. But now imagine a system where all these behaviors are rated as either positive or negative and distilled into a single number, according to rules set by the government. That would create your Citizen Score and it would tell everyone whether you were trustworthy, or not. Plus, your rating would be publicly ranked against that of the entire population and used to determine your eligibility for a mortgage or a job, where your children could go to school, or even your chances of getting a date.
 
Is this a fantasy about of “Big Brother” getting out of control?  No, according to the new book, Who Can You Trust? How Technology Brought Us Together and Why It Might Drive Us Apart by Rachel Botsman, its already getting underway in China, where the government is developing the Social Credit System (or SCS) to rate the trustworthiness of its 1.3 billion citizens. The Chinese government is pitching the system as a desirable way to measure and enhance "trust" nationwide and to build a culture of "sincerity."  As the policy states, "It will forge a public opinion environment where keeping trust is glorious.  It will strengthen sincerity in government affairs, commercial sincerity, social sincerity and the construction of judicial credibility."

For now, participating in Chinas Citizen Scores is voluntary. But by 2020 it will be mandatory. The behavior of every single citizen and legal person (which includes every company or other entity) in China will be rated and ranked, whether they like it or not.

Prior to its national roll-out in 2020, the Chinese government is taking a watch-and-learn approach. In this marriage between communist “oversight” and capitalist “can-do,” the government will license eight private companies to come up with systems and algorithms for social credit scores. Predictably, data giants currently run two of the best-known projects.

The first is with China Rapid Finance, a partner of social-network Tencent and developer of the messaging app WeChat with more than 850 million active users.  The other, Sesame Credit, is run by the Ant Financial Services Group, an affiliate company of Alibaba. Ant Financial sells insurance products and provides loans to small- to medium-sized businesses. However, the real star of Ant is AliPay, its payments arm that people use not only to buy things online, but also for restaurants, taxis, school fees, cinema tickets and even to transfer money to each other.

Sesame Credit has also teamed up with other data-generating platforms, such as Didi Chuxing, the ride-hailing company that was Ubers main competitor in China before it acquired the American companys Chinese operations in 2016, and Baihe, the countrys largest online matchmaking service. Its not hard to see how that all adds up to gargantuan amounts of big data that Sesame Credit can tap into to assess how people behave and rate them accordingly.

So, just how are people rated? Individuals on Sesame Credit are measured by a score ranging between 350 and 950 points. Alibaba does not divulge the "complex algorithm" it uses to calculate the number, but they do reveal the five factors taken into account. The first is credit history. For example, does the citizen pay their electricity or phone bill on time?  Next is fulfilment capacity, which it defines in its guidelines as "a users ability to fulfil his/her contract obligations."  The third factor is personal characteristics, verifying personal information such as someones mobile phone number and address.  But the fourth category, behavior and preference, is where it gets interesting.
 
Under this system, something as innocuous as a persons shopping habits become a measure of character. Alibaba admits it judges people by the types of products they buy. "Someone who plays video games for ten hours a day, for example, would be considered an idle person," says Sesames Technology Director. "Someone who frequently buys diapers would be considered as probably a parent, who on balance is more likely to have a sense of responsibility." So, the system not only investigates behavior - it shapes it. It "nudges" citizens away from purchases and behaviors the government does not like.

Friends matter, too. The fifth category is interpersonal relationships. What does their choice of online friends and their interactions say about the person being assessed? Sharing what Sesame Credit refers to as "positive energy" online, nice messages about the government or how well the countrys economy is doing, will make your score go up.

You can imagine how this might play out when the governments own citizen score system officially launches in 2020. Even though there is no suggestion yet that any of the eight private companies involved in the ongoing pilot scheme will be ultimately responsible for running the governments own system, its hard to believe that the government will not want to extract the maximum amount of data for its SCS, from the pilots. If that happens, and continues as the new normal under the governments own SCS it will result in private platforms acting essentially as “spy agencies” for the government.

Posting dissenting political opinions or links mentioning Tiananmen Square has never been wise in China, but now it could directly hurt a citizens rating. But heres the real kicker: a persons own score will also be affected by what their online friends say and do, beyond their own contact with them. If someone they are connected to online posts a negative comment, their own score will also be dragged down.

So why have millions of people already signed up to what amounts to a trial run for a publicly endorsed government surveillance system? There may be darker, unstated reasons - fear of reprisals, for instance, for those who dont put their hand up.  However, there is also a lure, in the form of rewards and "special privileges" for those citizens who prove themselves to be "trustworthy" on Sesame Credit.

If their score reaches 600, they can take out a Just Spend loan of up to 5,000 yuan (around $750) to use to shop online, as long as its on an Alibaba site. Reach 650 points, and they may rent a car without leaving a deposit. They are also entitled to faster check-in at hotels and use of the VIP check-in at Beijing Capital International Airport. Those with more than 666 points can get a cash loan of up to 50,000 yuan (or $7500), obviously from Ant Financial Services. Get above 700 and they can apply for Singapore travel without supporting documents such as an employee letter. And at 750, they get fast-tracked application to a coveted pan-European Schengen visa.

Higher scores have already become a status symbol, with almost 100,000 people bragging about their scores on Weibo (the Chinese equivalent of Twitter) within months of launch. A citizens score can even affect their odds of getting a date, or a marriage partner, because the higher their Sesame rating, the more prominent their dating profile is on Baihe.  In a country with an extreme gender imbalance, that’s quite a perk.

Sesame Credit already offers tips to help individuals improve their ranking, including warning about the downsides of friending someone who has a low score. This might lead to the rise of score advisers, who will share tips on how to gain points, or reputation consultants willing to offer expert advice on how to strategically improve a ranking or get off the trust-breaking blacklist.

Indeed, the governments Social Credit System is basically a gamified, big data version of the Communist Partys old surveillance system: dangan.  Under this system, the regime keeps a dossier on every individual that tracks political and personal transgressions. A citizens dangan follows them for life, from schools to jobs.  Under that system, people started reporting on friends and even family members, raising suspicion and lowering social trust in China. The same thing will happen with digital dossiers. People will have an incentive to say to their friends and family, "Dont post that. I dont want you to hurt your score, but I also dont want you to hurt mine."

Currently, Sesame Credit does not directly penalize people for being "untrustworthy" because its been found more effective to lock people in with treats for good behavior.  But the system is designed so that "untrustworthy people cant rent a car, cant borrow money or even cant find a job."   Sesame Credit has also approached Chinas Education Bureau about sharing a list of its students who cheated on national examinations, in order to make them pay into the future for their dishonesty.

Notably, penalties are set to change dramatically when the government system becomes mandatory in 2020.  Indeed, on September 25, 2016, the State Council General Office updated its policy entitled "Warning and Punishment Mechanisms for Persons Subject to Enforcement for Trust-Breaking". The overriding principle is simple: "If trust is broken in one place, restrictions are imposed everywhere."

For instance, people with low ratings will have slower internet speeds; restricted access to restaurants, nightclubs or golf courses; and the removal of the right to travel freely abroad with, "restrictive control on consumption within holiday areas or travel businesses."  Scores will influence a persons rental applications, their ability to get insurance or a loan and even social-security benefits. Citizens with low scores will not be hired by certain employers and will be forbidden from obtaining some jobs, including in the civil service, journalism and legal fields, where of course you must be deemed trustworthy. Low-rating citizens will also be restricted when it comes to enrolling themselves or their children in high-paying private schools.  Botsman is not fabricating this list of punishments. Its the new reality Chinese citizens will face.

As the government document states, the social credit system will "allow the trustworthy to roam everywhere under heaven, while making it hard for the discredited to take a single step".

As Botsman observes, “Where these systems really descend into nightmarish territory is that the trust algorithms, as currently designed, are unfairly reductive. They dont take into account context. For instance, one person might miss paying a bill or a fine because they were in hospital; another may simply be a freeloader.  And therein lies the challenge facing all of us in the digital world, and not just the Chinese. If life-determining algorithms are here to stay, we need to figure out how they can embrace the nuances, inconsistencies and contradictions inherent in human beings and how they can reflect real life.”

The government also argues that the system is a way to bring in those people left out of traditional credit systems, such as students and low-income households. Professor Wang Shuqin from the Office of Philosophy and Social Science at Capital Normal University in China recently won the bid to help the government develop the system that she refers to as "Chinas Social Faithful System". Without such a mechanism, doing business in China is risky, she stresses, as about half of the signed contracts are not kept. "Given the speed of the digital economy its crucial that people can quickly verify each others credit worthiness," she says. "The behaviour of the majority is determined by their world of thoughts. A person who believes in socialist core values is behaving more decently." She regards the "moral standards" the system assesses, in addition to financial data, as a bonus.

Given this trend, we offer the following forecasts for your consideration.

First, implementation of the Chinese governments Social Credit System will reinforce China’s trend toward collectivism.
As explained in the November 2017 issue, China is alone in being the world’s only large nation with rising wealth that has seen diminishing individualistic values. That’s largely a function of government efforts to minimize conflicts arising from its increasing wealth gap.  And, once the mandatory Social Credit System is in place, the Communist Party will have a great deal more leverage over individual behavior. This could undermine religious freedom and create a de facto “caste system” based on rankings.

Second, trends in technology, demography, and human behavior will converge in ways that pave the way for “social credit systems” around the world.

The prevalence of electronic transaction processing, social media, location-based services, and surveillance cameras combine with big data and AI to make cost-effective systems available globally.  The growing security risks created by migration into the EU and elsewhere creates a reasonable rationale for 24/7 surveillance.  And the actions of terrorists in places as diverse as Paris, New York and Mumbai, means that most nations will want to monitor social deviancy.  The need to balance stability and freedom will lead to serious policy debates in unexpected places.  And,

Third, the rise of “social credit systems” in places like China will trigger an anti-surveillance backlash in other places. 

The realization of what is possible with current technology will set off alarms among civil liberties advocates around the world.  The result will be compromises that support national security without infringing upon civil rights.  

 
References

1. Wired, 21 October 2017. RACHEL BOTSMAN. Big data meets Big Brother as China moves to rate its citizens.

http://www.wired.co.uk/article/chinese-government-social-credit-score-privacy-invasion

2. Rachel Botsman, October 4, 2017. Who Can You Trust? How Technology Brought Us Together and Why It Might Drive Us Apart(Penguin Portfolio).

https://www.amazon.com/Who-Can-You-Trust-Relationships/dp/1541773675 

3. com. December 2, 2017. Dom Galeon and Brad Bergan.  China’s “Social Credit System” Will Rate How Valuable You Are as a Human.

https://futurism.com/china-social-credit-system-rate-human-value/

4. Nov. 10, 2017. Amulya Shankar. Whats your citizen trust score? China moves to rate its 1.3 billion citizens.

https://www.usatoday.com/story/news/world/2017/11/10/whats-your-citizen-trust-score-china-moves-rate-its-1-3-billion-citizens/851365001/

5. Wall Street Journal. Nov. 28, 2016. Josh Chin and Gillian Wong.  China’s New Tool for Social Control: A Credit Rating for Everything.

https://www.wsj.com/articles/chinas-new-tool-for-social-control-a-credit-rating-for-everything-1480351590