Unveiling the AI Privacy Paradox

The EU AI Act, released by the European Union, represents the first comprehensive set of laws specifically tailored to regulate high-risk AI systems.

The latest wave of AI regulations has a dual focus: ethical usage and safeguarding privacy. These regulations, as exemplified by the European Union's General Data Protection Regulation (GDPR), are designed to establish stringent safeguards for personal data while imposing restrictions on potentially harmful automated decisions made by artificial intelligence (AI) systems. One of the fundamental principles of the GDPR is the prohibition of individuals facing impactful choices solely determined by AI, although there are certain exceptions to this rule. The rationale behind these regulations is to harness the advantages of AI technology while ensuring proper oversight to protect individuals' rights and prevent unintended consequences. The GDPR, for instance, introduces the concept of mandatory Data Protection Impact Assessments. These assessments are a crucial component of GDPR compliance for organizations embarking on new projects, especially those involving AI.

Data Protection Impact Assessments play a pivotal role in evaluating the potential risks associated with AI systems. They require organizations to meticulously document their automated decision-making processes, assess the necessity and proportionality of such decisions, and develop strategies for mitigating potential risks. This robust evaluation process ensures that high-risk AI systems are subject to the necessary levels of regulation and oversight, minimizing the potential for misuse and harm. In contrast, the United States adopts a sectoral approach to AI regulation. Instead of a comprehensive framework like the GDPR as US regulations are dispersed across various sectors. For example, the Fair Credit Reporting Act mandates that credit scoring algorithms must be transparent and offer consumers access to their credit data. Similarly, the Equal Credit Opportunity Act serves as a safeguard against credit discrimination. Moreover, the Federal Trade Commission (FTC) has taken an active role in shaping AI accountability through guidelines that emphasize transparency, explainability, fairness, and security. These guidelines aim to ensure that AI systems used in various industries adhere to principles that protect consumer interests and maintain trust in the technology. While the European Union's GDPR sets a high bar for AI ethics and data protection, the United States employs a more diversified approach, with regulations tailored to specific industries and the guidance of agencies like the FTC. Both approaches share the common goal of balancing the benefits of AI with the need for oversight and protection of individuals' rights.

In 2017, New York City made a groundbreaking move by enacting a law that mandated audits to uncover biases in algorithms utilized by city agencies. This municipal regulation emerged as a pioneering effort to hold automated decision systems accountable. Its impetus arose from concerns surrounding the use of personality tests in the hiring process for teachers. Subsequently, California became the first US state to embrace similar legislation, signaling a growing trend toward algorithmic transparency and fairness. Fast forward to 2021, and the European Commission introduced what can be considered a landmark development in AI regulation. The EU AI Act, released by the European Union, represents the first comprehensive set of laws specifically tailored to regulate high-risk AI systems. Within this framework, the EU AI Act categorizes AI systems by their level of risk and imposes obligations on developers that vary according to the intended purpose and potential harm of the AI system in question. While critics argue that these regulations may pose burdens for innovators, regulators contend that responsible oversight is essential for safety and to prevent inadvertent discrimination, as exemplified by cases such as Apple's credit limit algorithm showing gender-based bias. One of the primary challenges in regulating AI lies in the inherent opacity of advanced AI systems, often referred to as "black boxes." These systems operate in a manner that is difficult to scrutinize due to their complex, non-transparent nature, making it arduous to identify and rectify biases.

Beyond formal regulations, the AI research community has also taken proactive steps to establish guidelines and frameworks for the responsible development of artificial intelligence. Notably, the Asilomar AI Principles, crafted during a 2017 conference of AI experts, have exerted considerable influence. These principles offer high-level guidance, aligning AI development with human values and ethical considerations. Regulations, on the other hand, institute obligatory standards that must be followed. Numerous technology companies and organizations have joined the quest for ethical AI by adopting their own sets of principles. For instance, Google has committed to developing AI that is not only beneficial but also accountable and socially aware, while Microsoft and Facebook have also endorsed similar AI guidelines. Additionally, the Organization for Economic Cooperation and Development (OECD) has established internationally agreed-upon AI principles that serve as essential policy benchmarks. Critics of these principles argue that they alone may not suffice to restrain the potential harms of AI and advocate for more robust oversight measures. However, well-designed and effectively implemented guidelines can prove highly effective for internal governance within organizations, fostering responsible AI development and use.

Nepal can glean valuable insights from the global landscape of artificial intelligence (AI) regulation and ethics to shape its own path towards responsible AI development and data protection. Observing the proactive measures taken by jurisdictions such as New York City, Nepal should prioritize algorithmic accountability in both public and private sector AI applications, particularly within government agencies. Implementing mandatory audits to identify biases in algorithms can enhance transparency and fairness. While Nepal may not replicate the European Union's comprehensive AI regulations, it can adapt elements of their approach, such as categorizing AI systems by risk and imposing developer obligations. Collaborating with its AI research community to craft context-specific guidelines, akin to the Asilomar AI Principles, is crucial for aligning AI development with the nation's values. Encouraging technology companies and organizations within Nepal to adopt AI ethical principles ensures that locally developed AI systems uphold ethical standards. Additionally, referencing international benchmarks, like the OECD AI principles, can help Nepal establish a robust regulatory framework. By incorporating these lessons and principles, Nepal can pave the way for responsible AI development, safeguard individual rights, and position itself as an ethical player in the global AI landscape.