SC quashes tribunal orders for using AI-hallucinated citations, orders rehearing

The Supreme Court on Thursday set aside orders passed by the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT), holding that both forums had relied on non-existent, AI-generated 'hallucinated' judicial precedents while deciding an insolvency dispute.

In a significant ruling on the use of artificial intelligence in the legal system, a bench of Justice PS Narasimha and Justice Alok Aradhe also directed the Bar Council of India to constitute an expert committee to examine the challenges arising from the use of AI in adjudication and legal practice.

"We have therefore directed the Bar Council of India also to constitute a committee and examine these issues in detail," livelaw.in quoted the Supreme Court as saying.

The bench underlined that while artificial intelligence can assist judges and lawyers, it cannot replace human reasoning. It stressed that adjudication must remain under the "total and absolute control" of humans at every stage.

"There is again a case where the tribunal relied on non-existing fake and hallucinated material generated through artificial intelligence as if they were precedents in support of its judgment. For the reasons to follow, we have set aside the judgment of NCLT as well as the judgment in appeal to affirm and maintain the integrity of the adjudication and its processes," the court observed.

The judges said the case offered an opportunity to define the judiciary's approach to AI beyond merely correcting an erroneous judgment.

"More than the inevitable consequence of setting aside such judgment, what is significant for our decision making is our resolve to adopt artificial intelligence technology in aid of adjudication while at the same time asserting and declaring total and absolute control over adjudications with a human in the loop at every stage," the bench said.

The appeal stemmed from insolvency proceedings initiated by Jammu and Kashmir Bank Ltd. under Section 7 of the Insolvency and Bankruptcy Code against Essel Infraprojects Ltd. (EIL), which had furnished a corporate guarantee for credit facilities extended to Pan India Utilities Distribution Company Ltd. The NCLT, Mumbai, admitted the insolvency plea on August 28, 2024, recording a default of 87.43 crore. The NCLAT upheld that decision on September 11, 2025.

Before the Supreme Court, Senior Advocate Madhavi Divan, appearing for suspended director Pooja Ramesh Singh, argued that the tribunals had relied on six judicial decisions that either did not exist or did not support the legal propositions attributed to them.

The orders cited purported precedents, including State Bank of India v. Shree Ram Urban Infrastructure (2020 SCC OnLine SC 341), Everest Kento Cylinders v. Union of India ((2015) 2 SCC 1), and ICICI Bank v. Urban Infrastructure Real Estate ((2019) 16 SCC 528). However, an affidavit submitted before the apex court confirmed that these authorities could not be traced in any recognised legal database, establishing that they were fabricated or AI-generated 'hallucinated' citations.

Accepting the submissions, the Supreme Court ruled that judicial orders founded on non-existent precedents cannot be sustained and set aside the decisions of both the NCLT and the NCLAT. The matter has now been remanded to the NCLT for fresh adjudication without relying on the fabricated AI-generated citations.

The bench, according to livelaw.in, also delivered a strong warning on the dangers posed by AI hallucinations in the justice delivery system. It noted that one of the defining characteristics of present-day AI systems is "a tendency to generate non-existent fake and hallucinated results" while responding to prompts.

"We are neither concerned with the cause nor the process of resolving such hallucinations. It is for the engineers and scientists to deal with them," the court said, adding that the legal system cannot permit fabricated AI-generated material to influence judicial reasoning.

Drawing a striking comparison, the bench observed: "For us, that is, for those who are in the province of adjudication and determination of disputes, this by-product of AI, that is the production of fake, non-existing and hallucinated material and its utilisation as precedence in law, is like the release of methyl isocyanide in the province of law and justice, invisibly insidious and catastrophic by the time anyone notices."

"So far as the decision on facts is concerned, we set aside the judgment of the NCLT as well as NCLAT and ask the tribunals to decide them on the facts," the court held.

- Ends