​Manufacturing justice: On the top court, AI use observations

But for its serious implications, a judge relying on Artificial Intelligence (AI) hallucinations to arrive at a judicial determination would have been comical. Given the serious miscarriage of justice that such hallucinations can lead to, if used in the judicial processes, the Supreme Court of India has likened it to methyl isocyanate, the poisonous gas that led to the Bhopal gas tragedy in 1984 — “invisible, insidious, and catastrophic by the time anyone notices”. The Court made these observations while setting aside the orders of the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT) in an insolvency case after finding that the NCLT had relied on fictitious AI-generated legal citations, a lapse overlooked by the appellate tribunal. This is the latest in a series of interventions by the Court this year, cautioning against the use of AI-generated fictitious judicial precedents in court proceedings. Through its rulings and oral observations, the Court has consistently adopted a strict and cautionary approach to the deployment of AI in the justice delivery system. On February 27, the same Bench of Justices P.S. Narasimha and Alok Aradhe took cognisance of a trial court relying on AI-generated fictitious case laws and underlined that it was not merely “an error in decision-making” but amounted to judicial “misconduct”.

The Court has made clear that AI may serve as an assistive tool to improve efficiency, but it can never replace independent human reasoning, judicial discretion or professional accountability. In AI, humanity is encountering what many experts fear might well be an existential question. AI disruption is a known unknown — everyone knows that it is happening but nobody quite knows its extent or implications. There is, however, enough evidence of a combination of human stupidity and deliberate design leading to dangerous social outcomes being derived from AI. Human oversight is the essential counter to these dangers. The Court has said that presenting fabricated, machine-generated judgments to a court constitutes professional misconduct for advocates and a serious lapse of duty for judges; and any judgment influenced by an iota of fake or hallucinated AI material is “no decision in the eyes of law”. The draft ‘Regulations for Use of Artificial Intelligence (AI) in Courts, 2026’ prohibits AI in the making of judicial outcomes such as the function of adjudication, sentencing, or deciding bail eligibility and evaluating the credibility of parties or witnesses. Even as the draft is open for public consultation, the Court has directed the Bar Council of India (BCI) to set up a dedicated committee to formulate strict norms and define disciplinary actions for lawyers who cite unverified AI material. Justice must be done and seen; not hallucinated.

E