Artificial Case: ChatGPT Helps HSE Lawyer Write Article for Academic Journal

© iStock

Roman Yankovskiy, head of the Centre for Legal Education Transformation at the HSE Faculty of Law, academic supervisor of the Digital Lawyer programme, has written an article in a legal journal using ChatGPT. The experiment was agreed upon with the journal’s editorial board. The author explains why we need such an experience, whether artificial intelligence (AI) is up to the task, and whether we can prohibit its use in scientific work.

‘Can Artificial Intelligence Write an Article in a Law Journal?’ by Roman Yankovskiy was published in the March issue of the Zakon (‘Law’) journal. The author analysed the advantages and disadvantages of using AI in the field of jurisprudence, including its ability to adapt to complex terms, changes in legislation and the subtleties of argumentation.

‘I mainly focused on potential errors that may occur when generating AI text, their causes and consequences. In addition, it covers copyright issues for works created with the help of AI and possible ways to solve them,’ states the article abstract.

Roman Yankovskiy

‘Initially, I wanted to test the capabilities of the fourth version of ChatGPT, since the previous one (3.5) referred to fictional sources and generally coped worse with specialised texts. However, this did not prevent a student of the Russian State University for the Humanities from writing a diploma [with this version]. I really liked the fourth version, including its source searching capacities, so in order to show its capabilities to my colleagues, I decided to generate a full-fledged scientific article. Besides, I've always liked recursive articles, and now I finally got the opportunity to write one myself. I couldn't pass it up.’

Yankovskiy adds that the editorial board of the journal, which had previously published his articles, took some time to think it over. As a result, they quickly agreed to the publication with a disclaimer and even, as an exception, accepted the article out of turn—otherwise it would not have been published before autumn. ‘I think the fact that the editorial board of the journal has recently been taking more relevant topics also played a role. I know many digital lawyers who publish works in it. In a more conservative journal, they might just think it was nuts,’ says Yankovskiy.

The first part of the article, the theoretical section, is completely generated by AI. An appropriate disclaimer is given. The second (practical) part explains how the author worked with the AI and whether the result was successful.

In the conclusions to the theoretical part of the article, the author notes that AIs such as ChatGPT are already able to generate texts on legal topics, but the quality of these texts still leaves much to be desired.

‘Existing AI technologies have encountered several problems: misunderstanding the context, incorrectly interpreting legal terms, misunderstanding other nuances. With the development of AI technologies and more accurate training of models on legal data, the quality of generated texts may improve,’ explains Roman Yankovskiy.

Texts written by AIs cannot yet replace the work of human intelligence and experience, expertise, creativity and deep understanding of the material. At the same time, the author concludes that AI can be a useful tool for lawyers, ‘helping them to write draft versions of texts or providing basic information that can be refined and clarified.’

In the practical part of the article, Roman Yankovskiy spoke about his experience with ChatGPT. He started with general questions, such as ‘is an artificial intelligence capable of writing an article for a lawyer in a law journal?’. Then he asked the AI to clarify certain points—for example, whether artificial intelligence can perform creative tasks, particularly writing articles on current topics. The author also asked for examples of specific mistakes made by AI in terms of incorrectly understanding legal terms.

The entire first part of the article was compiled from ChatGPT’s answers, which Roman Yankovskiy edited over a few hours: he added headings and adjusted leads to particular sections of the article, made stylistic edits (such as replacing ‘artificial intelligence’ with ‘AI’), edited some long sentences, reduced enumerations, edited links, etc.

‘Thus, the conducted experiment showed that artificial intelligence (ChatGPT 4.0) is sufficiently developed to write an article suitable for publication in a legal journal. However, this requires a competent author who will ask the correct questions and then double-check the result. One should choose a general topic, far from doctrinal nuances, because current language models are very limited in dealing with such issues. Of course, the article will have to be edited, and the proposed sources should be rechecked,’ says Roman Yankovskiy.

When asked whether it is necessary to prohibit the use of AI while writing scientific articles, Roman Yankovskiy replied that it is hardly possible anymore. After some editing, a text written by an AI cannot be distinguished from a human one. In addition, it is possible in the latest versions of ChatGPT to train the AI to understand tasks even better and produce even more consistent results. ‘On the other hand, articles generated by AI have a very conditional scientific novelty, and if editors carefully monitor this criterion and involve reviewers in checking articles, the quality of articles will not fall,’ he believes.

Yankovskiy adds that AI is primarily a new tool that greatly expands the capabilities of people in many areas. ‘I don't know Python, but now I've started writing simple programs using ChatGPT—for example, to conduct quantitative analysis of legal texts or create macros for complex actions in Excel. It's great!’ concludes the researcher.