Seventy years ago, George Orwell in his dystopian novel 1984 depicted a dark future where the elites use technologies to control society. The idea is not too far removed from the truth today
Facial recognition cameras equipped with Artificial Intelligence (AI)-enabled body temperature detection technology have been installed in cities across China to detect people who may be infected with COVID-19 to prevent them from traveling. The facial recognition technology (FR) is probably developed with good intentions, but experts argue this intrusive surveillance could be potentially used by totalitarian regimes against the population. Therefore, many pro-social technology experts, human rights activists, and politicians have been calling for a ban. The law enforcement agencies and other technologists, however, argue FR has the potential to offer huge benefits to society and are instead a calling for proper regulation.
Seventy years ago, George Orwell in his dystopian novel 1984 depicted a dark future where the elites use the technologies popular among the general public to control society. The idea considered too far-fetched for almost half a century is now not too far removed from the truth, especially with advances in big data, FR, AI and the Internet of Things (IoT).
FR has received huge backlash recently because the empirical studies showed it was being used for mass and discriminatory surveillance. Fifty prominent universities including Columbia, MIT, and Brown, have already confirmed that they will not use FR on campus. San Francisco was the first city in the US to ban the use of FR. Other American cities have also followed suit. Oakland and Somerville have also banned law enforcement agencies from using technology.
Culture of Schadenfreude
The good thing is that governments and tech-giants are on the same page on their stance on properly regulating FR. Alphabet and Google CEO Sundar Pichai last year said, “repressive uses of facial recognition” are of great concern at the moment and asked government agencies to refrain from making rules and regulations in an ad-hoc manner, but take adequate amount of time to create something meaningful. Last year, US Senators Roy Blunt and Brian Schatz, members of the Senate Committee on Commerce, Science, & Transportation, introduced “the Commercial Facial Recognition Privacy Act of 2019” to enhance consumer protections by “prohibiting commercial users of FR from collecting and re-sharing data for identifying or tracking consumers without their consent.”
The bill requires corporations to notify consumers if FR is being used and also “requires third-party testing and human review of technologies before their implementation, to address accuracy and bias issues in the technology and avoid use cases that may result in harm to consumers.” Additionally, the bill restricts “organizations from redistributing or disseminating data to third-party entities without the end-users’ consent and makes it mandatory for FR providers to meet data security, minimization and retention standards as determined by the Federal Trade Commission and the National Institute of Standards and Technology.”
Some politicians are in favor of completely halting the government’s use of FR stating that it is still primitive and unreliable, thus just an unnecessary infringement on people’s privacy and liberty. In reality, FR has become highly sophisticated and is already being used to unlock iPhones, scan flight passenger’s faces, screen people attending musical concerts, and monitor crowds on busy city streets and airports. At the same time, authoritarian regimes have been abusing FR for mass and discriminatory surveillance infringing people’s fundamental rights generating negative news coverage for FR.
The people who support FR are pushing for proper regulations stating the technology has the potential to help solve criminal cases, suspicious terrorist activities and locate missing children. In short, they maintain that technology’s benefits to society outweigh the negatives aspects. Also, according to the Global Opportunity Analysis and Industry Forecast (2015-2022), the global facial recognition market is expected to grow significantly reaching $9.6 billion by 2022. Therefore, there is a need for governments to come up with regulations that allow enough space to find the correct balance between privacy, innovation and competitiveness.
Keeping in mind the market potential and societal benefits, the European Union (EU) has scrapped the possibility of a ban on facial recognition technology in public spaces. According to the latest draft of the EU’s Artificial Intelligence strategy, “the union is no longer interested in introducing a ban on facial recognition in public spaces, instead, planning to develop ‘clear criteria’ for the mass-scale deployment of biometric identification systems deployed inside the EU.”
To help engineers and policymakers, the World Economic Forum recently released the first framework for the safe and trustworthy use of facial recognition technology. In a nutshell, the ‘Framework for Responsible Limits on Facial Recognition’ is structured around four steps: defining what constitutes responsible use of FR, design best practices, assess to what extent the system being designed is responsible, and validate compliance with the principle for action through the design of an audit framework by a trusted third party (AFNOR certification for the policy pilot).
At a time when the ethical concerns and limitations of using the highly sophisticated surveillance technology are being debated around the world, the framework released by the World Economic Forum could be a timely guide for technologists and policymakers. Still, a multi-stakeholder collaboration involving academics, scientists, engineers, technology solution providers, users, policymakers, lawyers and citizens is necessary to formulate a sound global policy framework on the appropriate use of FR. Sadly, the political discourse in Nepal has not moved beyond CCTV cameras. The recent developments in the area of innovative technologies are never discussed. It is high time for the Government of Nepal to work with the industry, academia, civil society, and activist to formulate progressive laws to regulate FR.
It is stupid to blame the technology. It only mirrors everyday biases among humans. More importantly, the technology will be harmful to society only when people who are on the driver’s seat use it with bad intentions of harming the public. Many authoritarian regimes have banned Orwell’s 1984 but the book still gets pirated, digitally downloaded, and printed into false covers. Recently, the sales have surged in stable democracies as well because mass surveillance is quickly becoming the new normal.
Rai is the Deputy Director of Studies at Nepal Administrative Staff College and has a PhD from IIT-Bombay. Shah is a Policy Wonk