The Supreme Court has expressed deep concern over the increasing misuse of artificial intelligence tools by legal professionals, particularly in citing judgments that do not exist. During a recent hearing, the bench observed that the practice of relying on AI-generated content without proper verification is emerging as a serious threat to the integrity of judicial proceedings. The court termed it a growing “menace” that could undermine trust in the legal system.

Highlight: Fabricated Case Laws Detected in Submissions

The issue came to light after instances where lawyers submitted case references and legal precedents that were later found to be entirely fabricated. These citations, often generated by AI chatbots, appeared convincing but had no basis in actual court records. Judges noted that such errors not only waste judicial time but also mislead proceedings and could impact case outcomes.

Highlight: Judges Stress Responsibility of Legal Professionals

The bench emphasized that advocates have a duty to ensure the accuracy and authenticity of the material they present before the court. It made clear that reliance on AI tools does not absolve lawyers of their professional responsibility. The court warned that any negligence in verifying sources could invite strict consequences, including penalties or disciplinary action.

Highlight: AI Tools Under Scrutiny for Hallucination Risks

The court’s remarks also highlighted the broader issue of “hallucinations” in AI systems—where tools generate plausible but incorrect or entirely false information. While acknowledging the usefulness of AI in research and drafting, the judges cautioned that such tools must be used carefully and supplemented with human verification, especially in sensitive fields like law.

Highlight: Call for Guidelines and Ethical Usage

Legal experts believe the Supreme Court’s observations could pave the way for clearer guidelines on the use of AI in legal practice. There is a growing demand for regulatory frameworks and ethical standards to ensure that technology enhances efficiency without compromising accuracy or accountability.

Highlight: Global Concern Mirrors Indian Scenario

The issue is not limited to India. Courts in other jurisdictions, including the United States, have also dealt with cases where lawyers cited fictitious judgments generated by AI. The Supreme Court’s warning aligns with a global push to address the risks posed by unchecked reliance on generative AI in professional environments.