Kapittel 5.3Initiated: Facilitate the use of language models adapted to Norwegian conditions

Background

The Ministry of Health and Care Services (HOD) has tasked the Norwegian Directorate of Health with the following: as part of further work, the government agencies shall also identify measures to ensure that the health and care services have access to one or more language models that are adapted to Norwegian conditions [64].

The description of this measure is based on the recommendation concerning access to language models adapted to Norwegian conditions (5.2) in the knowledge base on the use of large language models [65].

What is the problem?

It is uncertain whether the available language models give sufficient consideration to the Norwegian language (both Bokmål and Nynorsk), professional terminology, culture, ethics and practice in the healthcare sector. There is no Norwegian test suite that can be used to determine whether this is the case.

What do we want to achieve?

The objective is to ensure that language models and/or applications based on large language models in the health sector function sufficiently well to perform tasks in a Norwegian context while ensuring that international knowledge is safeguarded. Texts for the general public should be written using clear and patient-friendly language so that they can be read by patients, users, relatives and citizens in general. Texts for health professionals should be consistent with Norwegian professional terminology. Nynorsk should work as well as Bokmål. Language models which are to be used in the administrative area for the Sámi languages must function in Sámi.

What is happening within the area?

As of 1 March 2024, the following stakeholders have initiatives in place relating to the development of large language models in Norwegian:

• NorwAI (NTNU and others) is currently based on Norwegian language models
• NORA (UiO, NB and others) is currently based on Norwegian language models
• Western Norway Regional Health Authority ICT is fine-tuning the base model NorBERT for the clinical domain: Clinical NorBERT (statics.teams.cdn.office.net)

The Norwegian Directorate of Health (and others) are testing adaptations of international models through what is known as Retrieval Augmented Generation (RAG) in, for example, DigiUNG and Helsesvar.

A number of enterprises in both the private and public sectors have developed guidelines for the use of generative AI [66]

How to make it happen

A number of measures could be used to secure access to language models adapted to Norwegian conditions. More experience will be needed to identify appropriate measures. Initially, it will be important to monitor the results of the ongoing initiatives.

A report should be prepared to identify measures which ensure that the health and care services have access to language model(s) that are adapted to Norwegian conditions. This can be achieved by:

• Monitoring and participating in national and/or Nordic projects that develop large Norwegian language models that can be used in the health and care sector
• Analysing the possible need for a national medical language model based on results from ongoing development projects, such as Clinical NorBERT, NorwAI and NORA.
• Monitoring projects that adapt open, international models to Norwegian conditions.
• Assessing what Norwegian medical training data is relevant and the extent to which it is available for the development, training and fine-tuning of language models, including copyright and privacy 
• Developing Norwegian test suites that can be used to determine whether language models are adapted to Norwegian conditions, and developing Norwegian benchmarking
• Making available common semantic resources, such as knowledge graphs, for use in language models
• Analysing possible management regimes for one or more national medical language models
• Coordinating and compiling relevant guidelines for the use of generative AI development and, where appropriate, encouraging the development of guidelines in selected areas of the health and care services
• Establishing an overview of national and international data sets that are available for AI research.
• Facilitating the enrichment of language models using terminologies and classifications translated into Norwegian

Who is responsible?

The Norwegian Directorate of Health

Who collaborates?

The specialist and R&D community within the fields of AI and language technology (including the National Library of Norway), central administrative bodies (including the Norwegian Digitalisation Agency and the Norwegian Data Protection Authority), the health and care services and the business community provide input in the work.