AI Tool Boosts Case Resolution by Over 6% in Pakistani Trial Courts

A large-scale field trial in Pakistan's justice system revealed that the integration of a generative artificial intelligence assistant, named JudgeGPT, led to a significant increase in case resolution. The study, published on Tuesday and titled “Courts of Tomorrow: Evidence from a Nationwide Rollout of Generative AI,” found that AI use resulted in an additional 1,848 cases being resolved annually, a rise of over 6.3% compared to the average. This improvement did not appear to compromise the quality of judicial output. JudgeGPT, a chatbot based on OpenAI's GPT-4 models, was customized for the Pakistani context and extensively beta-tested with the Federal Judicial Academy before its deployment to 1,559 judges across 118 courts. The trial's effectiveness was most pronounced when the AI assistant was paired with targeted training for judges on its proper use. Judges receiving both AI access and specific training were more likely to adopt, use intensively, and continue using the tool, reporting increased productivity. Targeted training also guided the AI's application towards tasks like text improvement, where language models are highly effective, and away from complex legal queries requiring extensive verification. The trial involved approximately half of the country's trial judges and 80% of district courts, with participants randomly assigned to groups receiving AI with targeted training, AI with generic training, or only generic training. Post-trial analysis of judicial opinions showed little evidence of deterioration in writing quality, with some indications of a positive effect on quality assessment. Furthermore, the study found minimal evidence of systematic changes in gender or religious bias within judicial language. Judges primarily utilized the tool for legal research and writing support, with training directing its use towards tasks like summarization and text refinement rather than full-text generation, preserving judicial agency. The research concluded that while AI access increases usage, sustained engagement hinges on targeted training, emphasizing that AI serves as a tool to enhance judicial capacity rather than a replacement for judges, particularly for judiciaries facing case backlogs. In April, Pakistan's National Judicial Policy Making Committee issued national guidelines for AI in judicial institutions, advocating a human-centric approach where AI assists but does not replace judicial decision-making, ensuring ethical, transparent, and bias-mitigated use with strong safeguards for accountability.