Judiciary, chatbots and hallucinations: As Gujarat High Court draws a red line on AI, why human judgment is non-negotiable

In a landmark decision, the Gujarat High Court has firmly established that human judgment remains the cornerstone of judicial proceedings, rejecting the integration of artificial intelligence (AI) in critical aspects of the legal process. On April 4th, the court unveiled a comprehensive policy that not only regulates AI but imposes stringent restrictions on its use in judicial functions. This move comes as a response to growing concerns over the misuse of AI tools, which have increasingly led to fabricated citations and unreliable outputs in court proceedings.

The policy sets a clear boundary by prohibiting the use of AI in judicial reasoning, order drafting, judgment preparation, bail or sentencing considerations, and any substantive judicial process. These restrictions extend beyond direct use, barring even indirect influence of AI on findings of fact, findings of law, or operative orders. The court emphasizes that adjudication cannot be assisted, influenced, or shaped by machines in any form. This stance is a direct challenge to global trends, where AI is increasingly being employed in various judicial tasks.

Moreover, the policy extends its restrictions to the use of AI for sorting evidence, classifying documents, organizing evidentiary material, assessing credibility, filtering relevance, or summarizing depositions and testimony. The court asserts that any task involving evaluation or categorization of proof must remain exclusively within the human domain. This is a significant departure from the prevailing trend of leveraging AI for such functions, underscoring the court's commitment to preserving the integrity of the judicial process.

The Gujarat High Court has also placed strict limits on the data that can be shared with AI tools. According to the policy, no confidential or sensitive information—such as names of parties, witness details, case records, legal strategies, or personal data like health, financial, biometric, or caste-related information—can be provided to public AI tools. Even if an enterprise AI tool is approved by the High Court, the use of such data is heavily restricted. This reflects the court's concern over privacy protection and the potential leakage of sensitive information into external systems.

Additionally, the policy prohibits the reliance on AI-generated citations or legal references without human verification. This measure aims to prevent the introduction of inaccurate or misleading legal references into court proceedings, ensuring that all citations and references are vetted by human judges to maintain the reliability and accuracy of legal arguments.

The Gujarat High Court's policy is a bold statement about the non-negotiable role of human judgment in the legal system. While AI has the potential to enhance efficiency and accuracy in certain areas, the court's restrictions underscore the importance of safeguarding the fundamental principles of justice and fairness. By drawing a clear line on the use of AI in judicial processes, the court is not only protecting sensitive information but also ensuring that the judiciary remains accountable and transparent. This decision sends a strong message to the legal community and the broader tech industry about the necessity of human oversight in the administration of justice. As the world continues to explore the integration of AI in various sectors, the Gujarat High Court's approach serves as a reminder of the enduring value of human judgment in the most critical of decisions.