Time to regulate AI

Artificial Intelligence is the most transformative revolutionary technology of our era that is shaping every area of human activity, but its immense power is also saddled with immense risks. The opacity of its algorithms enhances this danger – there are numerous instances where it has caused grievous harm to society. These harms can propagate at lightning speed giving little scope for correction or course reversal.

The risks also extend to systemic instability, as evidenced by AI-driven financial “flash crashes” – on 6 May 2010, the Dow Jones fell more than 1,000 points in just 10 minutes, erasing about $1 trillion in equity, though 70 per cent of the losses were recovered by the end of the day. Left unregulated, these risks can undermine trust in institutions and destabilize markets. But much more sinister is their ability to inflict physical violence. In 2020, a UN report indicated that Turkish-made Kargu-2 drones, powered by AI-based image recognition, may have attacked human combatants without direct human oversight, marking what may be the first recorded incident of autonomous lethal force.

Advertisement

Reports from Gaza in 2023–24 suggested that Israel used an AI system known as “Lavender” to automatically generate target lists for bombing campaigns. Such automated decision-making may have caused unintended civilian deaths by lowering the strike threshold, mocking the morality, legality and accountability in using violence against civilians. AI systems are prone to inheriting their creators’ biases and frequently reflect hidden biases within their training data, thereby replicating and intensifying human prejudices while feigning impartiality.

AI’s scale, invisibility, and speed make it more dangerous; once embedded in automated decision-making systems, biased outcomes can affect millions of people together. Such abuses erode trust, deepen inequalities, and perpetuate systemic injustice. A case in point is the COMPAS algorithm used in US courts to predict reoffending risk. The system unfairly labelled Black defendants as “high risk” compared to white defendants even when the latter had worse criminal histories, affecting bail and sentencing. Amazon’s AI hiring tool was scrapped after it was found to discriminate against female applicants.

Apple’s credit card algorithm came under fire in 2019 when women, including high-profile applicants, received significantly lower credit limits than men with identical financial credentials. Another serious concern is that a handful of large US corporations control most AI resources and computing power. This creates significant geopolitical risks, entrenching monopolies and deepening the digital divide. It is impossible to overlook how Facebook’s algorithm contributed to the Rohingya genocide in Myanmar’s Rakhine state in 2016-17.

Programmed to enhance user engagement to generate more advertising revenue for Facebook, the self-learning algorithm quickly learnt a common human trait – that propagating hatred and sharing provocative content were more efficient in enhancing user engagement. It then started normalising hateful content and dehumanizing Rohingyas – thousands of them were murdered and over 700,000 fled to Bangladesh. A 2018 UN Fact-Finding Mission highlighted the role of Facebook in inciting violence against Rohingyas, accusing the company of being “slow and ineffective” in responding to the crisis.

For big companies, profit is more important than safety, ethics, or accountability. AI is a tool with limitless potential to advance human welfare, but equally capable of causing limitless harm. The release of ChatGPT in 2022 was a wake-up call – its potential for spreading misinformation and hatred in society, and even undermining the democratic process, prompted global demands for AI-regulation. The European Union spearheaded this initiative by passing the EU Artificial Intelligence Act in August 2024. This August marks the implementation of its key features, with complete functionality expected by August 2026.

At its core, the Act follows a “risk-based” approach, categorising AI applications according to their risks from minimal to unacceptable, and prescribes obligations accordingly. It aims to ensure that AI systems are safe, transparent and non- discriminatory, and are overseen by humans to prevent harmful outcomes. Those with unacceptable risks will be outright prohibited in the EU – they include cognitive behavioural manipulation of people or specific vulnerable groups, social scoring or profiling of people by governments or any manipulative biometric surveillance including real-time biometric identification systems like facial recognition in public spaces.

As per this Act, high risk AI systems capable of breaching the safety or fundamental rights of citizens will need continuous monitoring. While Generative AI, like ChatGPT, will not be classified as high-risk, it will have to comply with transparency requirements by disclosing AI-generated content and to prevent it from generating illegal or harmful content. Content generated or modified with the help of AI – images, audio or video files (like deep-fakes) – must be labelled as AI generated. The Act seeks to protect citizens from discrimination, privacy breaches, and opaque algorithmic decision-making.

Any company offering AI products in the EU must comply with these requirements, and its influence is likely to extend beyond Europe, shaping global norms in the governance of emerging technologies, much like with the General Data Protection Regulation (GDPR), EU’s flagship law on data protection and privacy since 2018. The Act also prescribes a Code of Practice which has been signed by major companies (Google, Microsoft, OpenAI, Anthropic, etc.). Meta has refused, but will still be required to comply. The Act has created a new governance body, the European Artificial Intelligence Board (EAIB), to coordinate and oversee the consistency of its implementation across the EU. Critics argue these rules may stifle innovation and impose high compliance costs on start-ups, potentially giving an advantage to large firms.

The Trump administration wants the EU to scrap some of its rules which it considers barriers to trade, and the EU’s response will test whether it can chart an independent course in governing AI. Hitherto, the US approach has been focussed on safety, standards, and agency-driven oversight rather than sweeping regulation. But President Trump’s Executive Order of January 2025 revoked the existing policies, emphasising instead American leadership in AI. At the Paris AI Summit in February 2025, the US declined to sign a global AI governance declaration, contending that the EU Act will hinder AI investments, reduce competition, deter innovation, encourage a shift of AI-related jobs and capital and favour large players. In April 2025, it conveyed to the EU that its Code of Practice was excessively burdensome and beyond the regulation’s scope. In July 2025, the Trump administration initiated a series of aggressive deregulation measures, demonstrating a clear divergence from EU’s regulatory approach. Notwithstanding Trump, the world thinks that AI needs regulation.

At the 2025 World AI Conference in Shanghai, China unveiled its Global AI Governance Action Plan, promoting international cooperation, safety regulation, and UN involvement in global AI governance. This plan also emphasised oversight and collaboration – even though the transparency of Chinese AI development systems remains questionable. Brazil is also close to enacting its own risk-based AI regulation bill. India has no dedicated AI law in place. NITI Aayog’s 2018 National AI Strategy and 2021 paper on Principles of Responsible AI emphasised the need for frameworks for responsible AI adoption to ensure transparency, fairness, and accountability in AI systems and to balance innovation with protection of citizens’ rights and data privacy.

In January 2025, the government launched the IndiaAI Safety Institute, aimed at promoting a safe, ethical, and socially grounded development of AI. The Coalition for Responsible Evolution of AI (CoRE-AI) was formed in July 2024, pulling together 30+ stakeholders – including Google, Microsoft, Infosys, IIM Bangalore, and key AI start-ups – to shape voluntary guidelines and governance frameworks. The Ministry of Electronics and Information Technology released the Draft AI Regulation Bill in early 2025 for public consultation – it has provisions similar to the EU Act that focus on risk-based approach, mandatory registration for high-risk systems, human oversight, data privacy, etc.

The EU Act leaves a gap between safety compliance and accountability when damage occurs. The AI Liability Directive, which was originally intended to streamline claims, was withdrawn. It is also heavy on process but light on outcomes. India should use these weaknesses as guidance to create a fair regulatory system. To fix this, there must be clear rules about who pays if AI messes up, so victims can get compensation. Instead of depending on compute-based thresholds, we should also prioritize context-driven criteria, incorporating risk evaluation with deployment circumstances. India could surpass the EU through the clarification of copyright and data usage regulations, along with creating a one-stop national AI office like the EAIB.

(The writer is a commentator, author and academic. Opinions expressed are personal)