When Technology Begins to Decide the Law

There was a time when the greatest threat to justice came from delayed trials, missing evidence, or procedural irregularities. Today, a new challenge has quietly entered courtrooms—not in the form of an overburdened docket or a hostile witness, but through lines of computer-generated text capable of fabricating legal authorities that never existed.

The Indian judiciary has now reached a defining moment.

In a landmark development, the Supreme Court has sounded one of the strongest judicial warnings yet against the unregulated use of Artificial Intelligence in legal proceedings. The Court went beyond expressing caution—it set aside decisions of the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT) after finding reliance on AI-generated, non-existent judicial precedents. The Court described such material as a direct threat to the integrity of the justice delivery system and emphasized that unregulated AI could become “catastrophic” for the rule of law. It also called upon the Bar Council of India to examine safeguards and professional standards for the use of AI in legal practice.

This is not merely another judgment. It is the beginning of India’s constitutional conversation on Artificial Intelligence and justice.

The Rise of Artificial Intelligence in the Legal Profession

Within only a few years, Artificial Intelligence has transformed legal practice.

Today AI systems can:

• Draft pleadings within minutes.
• Summarize lengthy judgments.
• Compare statutory provisions.
• Translate legal documents.
• Predict litigation trends.
• Assist with legal research.

For advocates handling hundreds of files, AI promises unprecedented efficiency.

For courts burdened with millions of pending cases, AI offers opportunities to improve case management, translation, scheduling, transcription, and administrative functions.

Technology itself is not the problem.

The real question is whether the legal profession has developed the ethical discipline necessary to use it responsibly.

When Artificial Intelligence Hallucinates the Law

Unlike traditional legal databases, generative AI systems do not merely retrieve information.

They generate language.

Sometimes they generate legal authorities that never existed.

This phenomenon—popularly known as “AI hallucination”—occurs when an AI confidently produces fictional judgments, fabricated quotations, incorrect statutory provisions, or imaginary precedents.

In ordinary conversations this may be harmless.

Inside a courtroom, it is dangerous.

A single fake precedent cited before a court can distort judicial reasoning, mislead lawyers, waste judicial time, and ultimately affect the rights and liberties of citizens.

That is precisely why the Supreme Court viewed the issue with exceptional seriousness.

The Court held that judicial decisions based on fabricated precedents cannot be sustained because they strike at the very foundation of adjudication. It stressed that courts must adopt a zero-tolerance approach to fake or hallucinated AI-generated precedents and that advocates remain responsible for verifying every authority they place before a court.

Technology Can Assist. It Cannot Replace Judicial Reasoning.

The Supreme Court’s observations do not reject Artificial Intelligence.

Instead, they establish a constitutional principle:

Machines may assist. Human beings must decide.

This distinction is fundamental.

AI may efficiently perform tasks such as:

• legal research,
• document classification,
• translation,
• transcription,
• scheduling,
• document comparison,
• case analytics.

However, adjudication involves much more than processing information.

A judge evaluates credibility.

A judge balances competing constitutional values.

A judge appreciates human conduct.

A judge interprets justice within the framework of law.

None of these functions can presently be delegated to an algorithm.

Justice requires conscience.

Artificial Intelligence has none.

Professional Responsibility Cannot Be Outsourced

Perhaps the most significant implication of the judgment concerns advocates.

The Court has effectively reminded the Bar that professional responsibility remains entirely human.

Whether a pleading is drafted manually or generated through AI, the advocate signing it assumes complete responsibility for every fact, every citation, and every legal proposition contained within it.

Artificial Intelligence cannot become a defence against professional negligence.

If a lawyer cites a fictitious judgment generated by software, the misconduct remains the lawyer’s—not the machine’s.

The legal profession has always rested upon one principle:

Trust.

Without that trust, the justice system cannot function.

India’s Draft AI Regulations Show the Way Forward

The Supreme Court’s warning comes at a time when its proposed Draft Regulations for Use of Artificial Intelligence in Courts, 2026 already outline a structured governance framework.

The draft regulations permit AI for assistive and administrative functions such as legal research, translation, transcription, case management, accessibility tools, and document verification. At the same time, they prohibit AI-only adjudication, AI-driven sentencing, opaque risk-scoring for bail, predictive profiling, and any use that compromises judicial independence or due process. Human primacy, transparency, accountability, cybersecurity, and data protection remain the guiding principles.

This balanced approach recognizes a simple truth:

Technology should strengthen judicial institutions—not replace them.

Global Lessons for India

India is not alone.

Across the world:

• Courts in the United States have sanctioned lawyers for citing AI-generated fake cases.
• European regulators are building AI governance around transparency and accountability.
• Singapore and the United Kingdom increasingly encourage responsible AI adoption under human supervision.

The global consensus is becoming clear.

Artificial Intelligence belongs inside the courtroom only when accompanied by meaningful human oversight.

The Next Frontier: Ethical Artificial Intelligence

India now requires more than technological adoption.

It requires legal ethics for the AI age.

Every advocate should ask:

• Have I verified every citation?
• Have I personally read every judgment I rely upon?
• Have I independently confirmed every statutory reference?
• Can I defend every statement made in my pleading?

If the answer to any of these questions is “No,” then Artificial Intelligence has already become a liability.

The Rex Juris View

Artificial Intelligence is neither the hero nor the villain of modern legal practice.

It is a powerful instrument.

Like every instrument of law, its value depends entirely upon the integrity of the person using it.

The Supreme Court has delivered a timely reminder that justice cannot be automated.

Algorithms may identify patterns.

Machines may summarize precedents.

Software may translate languages.

But no algorithm can understand compassion.

No machine can evaluate conscience.

No software can replace judicial wisdom.

The future Indian courtroom will undoubtedly become more digital.

Yet one constitutional principle must remain unchanged:

Justice shall always remain a human responsibility.

The legal profession should therefore welcome Artificial Intelligence—but only under a framework that preserves judicial independence, professional accountability, procedural fairness, and public confidence.

Innovation is essential.

But innovation without regulation is not progress.

It is risk.

And when that risk enters a courtroom, it threatens not merely technology—but the Rule of Law itself.

Key Takeaways

• AI is a valuable assistive tool but cannot replace judicial reasoning.
• Advocates remain personally responsible for every citation and legal submission.
• Courts must adopt zero tolerance for AI-generated fake precedents.
• Human oversight should remain mandatory in every stage of judicial decision-making.
• India’s Draft AI Regulations for Courts provide a strong foundation for responsible innovation but will require effective implementation and continuous oversight.

Editorial Board
Rex Juris
Law • Justice • Wisdom
www.rexjuris.in

About The Author

Rex

See author's posts