Developing effective AI and law applications: a methodological proposal for developing natural language processing applications based on transformers, pre-trained models and transfer learning

Abstract

CONTEXT: The intersection between Artificial Intelligence (AI) and Law has been explored since the early days of AI research. In recent years, both judicial institutions and legal professionals have increasingly adopted AI technologies to streamline processes, support legal decision-making, automate repetitive tasks, and extract structured information from legal texts. These applications are predominantly powered by Natural Language Processing (NLP), given the textual nature of legal proceedings. Advances in deep learning, particularly transformer-based architectures and pre-trained language models, have significantly reshaped the development of NLP systems and achieved state-of-the-art results across many downstream tasks. PROBLEM: Despite the potential of these advances, there remains a lack of a unified, domain-specific methodology for developing AI applications in the legal field. This gap limits the effective adoption of NLP-based AI solutions in real-world legal contexts. Additionally, most cutting-edge resources are available primarily in English, creating a barrier for Portuguese-speaking legal systems. SOLUTION: This thesis proposes a structured methodology for developing AI and Law applications, grounded in modern NLP paradigms such as transformers, transfer learning, and domain adaptation. The methodology addresses both technical and domain-specific challenges and includes practical components, such as datasets, fine-tuned models, and evaluation tools, tailored to the Portuguese legal context. PROPOSED METHOD: The proposed methodology is composed of four main steps and emphasizes interdisciplinary collaboration between legal and technical teams. It defines a clear development flow, from problem definition to deployment, incorporating iterative validation, the creation of applicationlevel datasets, and the use of Explainable AI mechanisms where applicable. RESULTS: The methodology was validated through multiple experiments, including the development of Legal Named Entity Recognition (NER) models that achieved new state-of-the-art results on the LeNER-Br dataset, alongside experiments with context-specific and parameter-efficient finetuning techniques to enhance model performance and adaptability. A real-world application for AI-generated judgment reports was implemented, supported by a novel evaluation framework combining automated metrics and human assessment. This work also provides practical resources in Portuguese and insights into the correlation between human and automated evaluations of AI outputs in the legal domain, demonstrating the feasibility and benefits of a structured, domain-adapted approach to AI development in legal applications.