Recommended topic: Expertise concerning use, development and fine-tuning
Section 1.2 describes a series of seminars on the integrated use of AI in the health and care services. This section describes the topics that have emerged linked to the use of large language models during the work on a joint AI plan which should be considered further in the seminar series.
Recommended topic: Expertise concerning use, development and fine-tuning
Background
Section 6.7 of the final report for the coordination project states the following recommendation:
- that government agencies continue to build expertise relating to language models.
The description of this measure is based on the recommendation to strengthen capacity and expertise (5.1) in the knowledge base on the use of large language models.
What is the problem/challenge/barriers?
There is a lack of expertise concerning the use, development and fine-tuning of large language models at different levels within the health and care sector. There is also insufficient expertise available concerning change management linked to the introduction of language models. There is a shortage of funding for the necessary training within the service.
What we want to achieve
The Norwegian Directorate of Health and the entire health and care service should strengthen its expertise concerning the appropriate use of large language models. This expertise must be adapted to the different needs and levels that exist within the service. Future strategic decisions will be more closely linked to digital transformation and technological development. Basic competence will therefore also be needed among managers relating to the risks and benefits of using language models in the health and care service.
A report is also being prepared which will set out recommendations to ensure that the necessary expertise is available for digital transformation based on AI in the health and care services (Section 6.2).
What is happening within the area?
Some universities offer courses on large language models, including:
- Topic under the subject area IN1140 Introduction to language technology (University of Oslo)
- Topic under the subject area DIGI117 Language technology (University of Bergen)
- Topic under the subject area HSI415 Understanding technology (University of Agder)
Language models are likely to be included in AI study programmes, e.g. at the University of Inland Norway and UiT The Arctic University of Norway. There are also numerous seminars and online courses, including those run by Digital Norway, for example.
AI and language models are not a major part of the curriculum of health informatics courses.
How to make it happen
The following measures are relevant:
- Clarify the need for a dedicated board or network of experts in language models and health sciences from both the health and care sector and the university and college sector
- Promote key projects through an annual AI award for the sector
- Ensure that expertise concerning language models forms part of course curricula, not just as a separate study programme, but also as an integral part of health education in particular. This is seen in the context of the report on competence (section 6.2).
- Identify measures to enhance leadership skills in the sector concerning AI and language models. This is seen in the context of the report on competence (section 6.2).
- Obtain experience from Nordic and other countries
- Determine what should constitute user competence for AI and language models
Work should be commenced through the series of seminars proposed in section 1.2.
Current cooperation
The Norwegian Directorate of Health, the health and care services, the higher education sector and the R&D sector.
Recommended: Applications for large language models
Background
Section 6.7 of the final report for the coordination project states the following recommendation:
- that the government agencies, together with the health and care services, assess the applications where such [language] models may be useful and appropriate.
The description of this measure is based on the recommendation to promote key applications (5.2) in the knowledge base on the use of large language models.
What is the problem?
It is still uncertaiprep in which areas of application language models may be useful and suitable in the health service. This uncertainty could hinder development and use, even in cases where large language models can be both safe to use and offer savings and/or better healthcare. The distinction between pure language models and other AI models, such as image models, is likely to diminish as multimodal models become more prevalent.
What we want to achieve
The use of language models in areas where they are suitable, i.e. where they can offer benefits in the form of more efficient work processes or better quality health services in Norway. Applications entailing less risk may be faster to implement, yet also offer good value.
What is happening within the area?
None implemented, but testing/research is underway, e.g.:
- ClinCode: machine-supported ICD-10 coding (National Center for E-Health Research in collaboration with several other organisations)
- Citizen-facing chatbot (Norwegian Directorate of Health)
- Structuring of free text (Norwegian Health Archives)
- Risk prediction (Sørlandet Hospital). This project has been concluded.
How to make it happen
The following measures are relevant:
- Identify suitable applications and documentation of efficacy and benefits, in addition to the applications that should be introduced first (e.g. low risk, low cost and good efficacy, e.g. relating to administrative tasks) [67]
- Carry out a needs analysis to ensure applications originate from and are anchored among health professionals, administrative staff and patient representatives
- Highlight promising Norwegian initiatives and facilitate national use
- Monitor national, Nordic and international initiatives to disseminate knowledge and experiences
- Facilitate pilot projects and experimentation. This work can be carried out in collaboration with the business community
- Provide funding through a separate funding scheme for the most promising applications or use existing funding schemes, such as the Norwegian Medical Association’s Funds for quality improvement and patient safety (see also Chapter 4)
The work should be initiated through the seminar series proposed in section 1.2.
Current cooperation
The health and care services, the Norwegian Directorate of Health, parties that have adopted large language models in other sectors?
[67] One possible useful application that was highlighted in the input is interaction with foreign language-speaking patients. High-quality language models may lower the threshold for translating good patient information into multiple languages. Appointment letters will also be easier for patients to understand in their own language. This could reduce the risk of non-attendance at appointments. Translating patient information is a major task and very resource-intensive. In this area, there is considerable potential for savings to be made and excellent opportunities to improve the quality of equitable healthcare.