The Supreme Court of India has raised serious concerns over the unchecked use of artificial intelligence in judicial proceedings, particularly warning that unverified or AI-generated content can severely damage the integrity of the justice system.

The court made these observations while hearing an appeal related to the Essel Infraprojects insolvency case, where it found that the National Company Law Tribunal (NCLT) had relied on non-existent and AI-generated hallucinated legal precedents while passing its order.

The Supreme Court, comprising Justices PS Narasimha and Alok Aradhe, set aside both the NCLT and National Company Law Appellate Tribunal (NCLAT) decisions and directed a fresh hearing of the matter.

The apex court emphasised that while artificial intelligence may assist in legal research and adjudication, it cannot replace human reasoning or judicial decision-making.It stressed that human oversight must remain central in all stages of judicial processes.

The court also directed the Bar Council of India (BCI) to constitute a committee to study and regulate the use of AI in legal practice and court proceedings, highlighting the urgent need for regulatory safeguards.

Describing the use of fabricated AI-generated precedents as “invisible, insidious and catastrophic”, the court warned advocates and judges against relying on such material without verification.

It further noted that citing fake or hallucinated judgments should be treated with zero tolerance, as it undermines the credibility of the legal system.

In the Essel Infraprojects case, the dispute stemmed from insolvency proceedings initiated by Jammu and Kashmir Bank over an alleged default linked to a large credit facility.

The Supreme Court found that the lower tribunals’ reliance on false precedents compromised the fairness of adjudication, necessitating a complete reconsideration of the case.