Kapittel 4.1Recommended topics for the seminar series on a framework for use

Section 1.2 describes a series of seminars on the integrated use of AI in the health and care services. This section describes the topics that have emerged in the work on a joint AI plan relating to a framework for use which should be considered further in the seminar series.

Recommended topic: validation and use

What is the problem?

The appropriate use of AI in the health and care services often presupposes that the AI system has been validated and/or tested for use in Norway, on the Norwegian population, is adapted to language use and culture, and is consistent with our ethical values. The testing, validation and introduction of AI within healthcare enterprises is challenging and resource-intensive, and there are few common ways of doing this. It is difficult to acquire sufficient personnel resources, such as clinicians who have the time needed to test and validate AI in an already busy working day.

What we want to achieve

There is a need to collaborate, share experiences and re-use methods and also to develop common methods and practices for testing, validation, adoption and management. This will enable the health and care services both to increase the rate at which benefits are realised and to reduce resource usage relating to the introduction and use of AI systems.

Consideration must be given to the fact that not all validation methodology is directly transferable between enterprises, management levels or regional health authorities.

What is happening within the area?

It should be noted that the topic of validation is also discussed in the areas ‘Framework for quality assurance’ (section 3.1) and ‘Further development of health technology assessments’ (section 3.5).

• There are already a number of promising initiatives and collaborations underway in the sector that will facilitate efforts to collate and share knowledge and experience concerning the validation and use of AI systems relating to image diagnostics:
  • Vestre Viken has set up the Regional AI network in image diagnostics for the health trusts in the South-Eastern Norway Regional Health Authority. Among other things, Vestre Viken is assisting with the transfer of experience, sharing and re-use, and supporting project groups in the planning of further work [41]. Vestre Viken is also developing an AI starter kit.
  • The South-Eastern Norway Regional Health Authority plans to establish a national network for image diagnostics. This network will act as an arena for coordinating the AI initiative nationally and sharing experiences linked to various initiatives, including validation of the solutions.
  • The regional health authorities are working together to procure AI platforms for applications within image diagnostics, which all health trusts will be able to order under a framework agreement. The South-Eastern Norway Regional Health Authority is leading this work.
• Validation and quality assurance can be viewed in the context of guidance and approval schemes under the health technology scheme, when AI is part of welfare technology, digital home follow-up or EPJ systems[42].
• Ethical principles for the use of AI can be found in the current national strategy for artificial intelligence [43].

How to do it

The following activities are relevant:

• Establish an overview of validated products, contact details of those who have experience with the product, and any available information on assessments that have been carried out.
• Assess the need and scope to gather, share and re-use experiences and assessments relating to validation, quality assurance, risk assessments and ethical assessments.
• Assess what should be done locally, regionally and nationally and by whom relating to the validation, use and scaling of AI systems.
• Disseminate relevant knowledge and research results concerning the validation, introduction and use of AI in the health and care services, e.g. through articles on the cross-agency information pages on AI.
• Identify the need to develop guidance or national guides and guidelines in selected areas, based on experiences and needs in the sector.
• Identify the need to establish/further develop the existing guidance service with advice on validation, procurement, ethics, introduction, benefit realisation, etc.
• Assess the need to establish an approval scheme to ensure that AI is used safely and effectively, thereby creating trust among health professionals and citizens.
  • One possibility could be to further develop the Health Technology Scheme.

Work should be commenced through the series of seminars proposed in section 1.2.

Current collaboration

Health authorities, regional health authorities, health trusts, KS, municipalities, interest organisations, the health industry and research environments, including KIN, NORA and the National Centre for E-Health Research (NSE).

Recommended topic: benefit potential and realisation

Background

Section 6.6 of the final report for the coordination project states the following:

• The health and care services face significant financial challenges, which limit their ability to adopt new and innovative technology. This has been documented in EU studies as one of a number of inhibiting factors for investing in AI solutions. The strategy should therefore discuss the need to establish national and regional incentive schemes to expedite necessary investments that could yield future returns, such as meeting the need for health professionals.

What is the problem?

Using AI in the health and care services could introduce a risk of making mistakes, as well as uncertainties relating to technological choices, information security, privacy and benefits. As with the introduction of other technology, the benefits of using AI can be extracted in places other than where the investments are actually made, in addition to the fact that it may take time to extract the benefits.

In its R&D report, KS states that artificial intelligence offers considerable potential, and not utilising the opportunities that it offers would represent a substantial societal loss. Yet many municipalities are unsure of the potential benefits and are therefore awaiting further developments [44]. In the statistical publication Arbeidsgivermonitoren 2023, it is stated that realising benefits through digitalisation is perceived as being the biggest challenge, as 80 per cent of municipalities cite this as being very or fairly challenging [45].

What we want to achieve

We want the health and care services to use AI to better or more efficiently perform its tasks, and to accelerate the pace of realising the benefits of adopting AI. These benefits could include both short-term and long-term benefits, and for different target groups.

What is happening within the area?

The following activities are relevant:

• Under the leadership of the South-Eastern Norway Regional Health Authority, the regional health authorities will adopt artificial intelligence solutions that can help to free up time among health professionals and reduce waiting times. The effect of the solutions must be documented to facilitate further introduction [46].
• The Norwegian Digitalisation Agency has written about risk assessments and assessments of the benefits of using artificial intelligence [47].
• Examples of benefits
  • Vestre Viken has introduced bone fracture AI at all hospitals – more than 11,000 patients have been checked (dagensmedisin.no)
  • Ålesund Hospital and St. Olavs Hospital use artificial intelligence in radiotherapy for breast cancer (nrk-no)
  • Haukeland University Hospital uses AI in cancer treatment – saving valuable time (dagensmedisin.no)
  • Examples of opportunities within health and care in the municipal health service (section 4.1.1) (ks.no)

How to do it

The following activities are relevant:

• Assess the need to collect, share and re-use experiences and assessments concerning benefits and the work relating to benefits, both nationally and internationally.
• Identify areas where benefits can be realised, systematise who will benefit and what the benefits consist of. This also applies to non-clinical applications.
• Consider the preparation of guidance and templates for cost estimation and the work relating to realising the benefits of AI in the health and care services.
• Consider the preparation of guidance on strategic and financial considerations regarding the choice of in-house development versus the procurement of AI systems. 
• Assess how the negotiating power of the sector as a whole can be better leveraged, e.g. through the scope to stipulate contractual requirements concerning AI system performance, for example by stipulating contractual requirements concerning AI system performance in operation (prospectively).
• Consider instigating follow-up research in certain areas where AI solutions have been introduced.

Work should be commenced through the series of seminars proposed in section 1.2.

Current collaboration

The Norwegian Directorate of Health, the regional health authorities, the health and care services, the health industry and R&D organisations.

Recommended topic: barriers and funding needs

Background

Section 6.6 of the final report for the coordination project states the following:

• The health and care services face significant financial challenges, which limit their ability to adopt new and innovative technology. This has been documented in EU studies as one of a number of inhibiting factors for investing in AI solutions. The strategy should therefore discuss the need to establish national and regional incentive schemes to expedite necessary investments that could yield future returns, such as meeting the need for health professionals.

What is the problem?

Little is known about barriers and funding needs specifically linked to the introduction and use of AI in the health and care services. The fact that benefits are often extracted in areas other than where the investments are actually made can be perceived as a barrier to the adoption and use of AI systems. Furthermore, the scarcity of internal resources can make it challenging to procure and adopt quality-assured solutions. There may also be barriers that make it unattractive to hand over tasks to other occupational groups, the patients themselves and/or AI systems. There is a need to further investigate such barriers and consider the need for incentives that cut across management levels.

What we want to achieve

Map and clarify any AI-specific barriers and funding needs, so that the healthcare sector can accelerate the rate at which the benefits of adopting AI are reaped. Possible funding solutions can then be investigated based on the outcome of this work. The health and care services are included in this work.

What is happening within the area?

Some existing incentive schemes:

• Schemes such as Diagnosis-related groups (the DRG system) and activity-based funding (the ISF system) in the healthcare sector.
• Grant schemes under the Norwegian Directorate of Health, Health technology assessment scheme
• The co-funding scheme (digdir.no) under the Norwegian Digitalisation Agency.
• Impending schemes for Tryggere helseapper (More secure health apps) and Verktøykatalogen (The tool directory) on helsenorge.no

How to do it

The following activities are relevant:

• Investigate barriers and the need for additional financial incentives or funding that could boost the safe use of AI in the health and care services. In this regard, consideration should be given to whether the current funding solutions are sufficient or hindering the introduction of AI.
• Determine one or more suitable funding solutions if there is a need.
  • An example would be to consider further developing the grant scheme under the Health technology assessment scheme

Work should be commenced through the series of seminars proposed in section 1.2.

Current collaboration

The Norwegian Directorate of Health, the regional health authorities and the health and care services.

Recommended topic: access to data and infrastructures for training, fine-tuning and validation of AI models

Background

Section 6.6 of the final report for the coordination project contains the following recommendation:

• At present, data is often tied to proprietary administrative systems and is difficult to access. As a result, they are difficult to use for the training, validation and fine-tuning of AI models. One solution to this problem could be to establish a suitable secure and robust infrastructure for this. The strategy should revolve around how data and infrastructure in the health enterprises should be facilitated so that data can be re-used more readily and more holistically. Done properly, this could accelerate the pace at which such systems are introduced safely and appropriately.

What is the problem?

Accessing data is cumbersome, time-consuming and resource-intensive for researchers, internal AI development in healthcare enterprises and suppliers looking to develop AI systems. AI development is taking place rapidly, and the need for infrastructure in the health and care services can change rapidly. It is therefore difficult to predict and clarify future needs. At times, there may be a shortage of computing and storage resources in the market, which will also affect prices. In addition, the environmental footprint is large, and a set of environmental accounts must be included in the full set of accounts. Using supercomputing resources can be both complicated and time-consuming for researchers and others. There is also uncertainty regarding access to and the need for the various types of data that are required for training, fine-tuning and validating different types of AI systems, including language models.

Accessing data is cumbersome, time-consuming and resource-intensive for researchers, for internal AI development in healthcare enterprises, and for suppliers seeking to develop AI systems.

What do we want to achieve?

We want to:

• establish effective and flexible ways of training, fine-tuning and validating AI models and contributing to safe solutions for the entire health and care service. The need for such infrastructure applies to different types of AI systems, including language models (Chapter 5). There may be differing needs as regards research, validation and operation respectively. The work must be carried out in a positive dialogue with the health services which have an overview of the opportunities and limitations inherent in today’s infrastructure.
• provide appropriate information on how to access data and sufficient capacity for the processing of applications for access to data for AI by the Norwegian Directorate of Health and Helsedataservice
• provide appropriate support to researchers and others who need to use supercomputing resources, so that they can reduce the amount of time needed and effective use of limited supercomputing resources.
• better enable the Norwegian supply industry to effectively develop AI systems based on Norwegian data for the development, testing and validation of AI-based software, both for new products and for the further development of existing software products.
• strengthen the work relating to information management within the sector at the local, regional and national levels, including the use of common code values and terminology.

What is happening wtihin the area?

Some key initiatives are underway to test, train and/or validate AI models and scale.

• Commissioned reports: The need for supercomputing power for research and artificial intelligence has been assigned to the Research Council of Norway by the Ministry of Education. The Ministry of Health and Care Services has requested the Norwegian Directorate of Health to link in with this work. 
• NOR-X-CHANGE: The regional health authorities have a joint initiative to set up research PACS systems and establish a central support unit that supports common functionality such as data migration from clinical systems and data exchange between regions.
• Sigma2 AS is responsible for providing the national e-infrastructure for computational science in Norway, providing supercomputing services and large-scale data storage for research and education purposes. Sigma2 is owned by the Agency for Shared Services in Education and Research (Sikt). Sigma2 also assists researchers and others in the use of supercomputing resources.
• EuroHPC [48] is a major EU initiative which aims to bring together European resources within supercomputing. As of March 2024, Europe’s most powerful supercomputer is LUMI in Finland. Europe’s first exascale computer (i.e. a computer which can perform 10¹⁸ calculations per second) is scheduled to be in place in Germany in 2024: JUPITER. These supercomputers are becoming important for the further development of artificial intelligence in Europe. At the end of 2023, the University of Oslo was allocated a couple of weeks on LUMI to train three Norwegian large language models [49].
• EHDS (European Health Data Space) is a new regulation for establishing a common European health data space. The aim of EHDS is to promote secure access to and exchange of health data across borders. EHDS covers both the primary use of data (myHealth@EU) and secondary use (HelthData@EU).
• TEF Health [50] (Testing and Experimentation Facility for Health AI and Robotics) will provide certification and quality control standards to simplify processes for bringing responsible AI systems to market. TEF Health centres are being set up with environments that AI system developers can use to demonstrate the interoperability and functionality of their solutions. Norway does not participate in TEF Health as an “associated country”, but Norwegian enterprises are still able to respond to calls for applications published by TEF Health.
• The European Cancer Imaging Initiative [51] is an initiative under Europe’s Beating Cancer Plan. The goal here is to establish a federated European infrastructure for imaging data for cancer. This will link the EU-level and national initiatives and give clinicians, researchers and innovators access to cross-border cancer imaging data.
• The EU Rare Disease Platform [52] will provide researchers, healthcare institutions, patients and decision-makers with tools to improve knowledge, diagnosis and treatment of rare diseases. The platform will make it possible to search registry data at the EU level and will standardise data capture and data exchange.
• Personalised medicine: Among other things, a national strategy has been developed for personalised medicine covering the period 2023-2030 [53], a national genome centre is in the process of being established, and Norway is participating in various European initiatives.
• Grand Challenge in the Netherlands provides access to data for model development/evaluation.
• AIMInd is an established EU project [54] where the collaboration partners are seeking to establish a centre of excellence within innovation (SFI) aimed at creating a secure federated environment for the development and testing of AI models.

How to do it

The following activities are relevant:

• Identifying needs:
  • identify the needs of the specialist health service and assess possible approaches to a common framework for the effective training, fine-tuning and validation of AI models.
  • identify the needs of the municipal sector and assess possible approaches to a common framework for the effective training, fine-tuning and validation of AI models.
  • identify supply industry needs and assess possible approaches to access to health data for the development of commercial AI systems. Among other things, look at perceived legal and organisational barriers and infrastructure challenges for access to data. This applies both to new products and to existing products that are expanded with AI functionality.
• Consider establishing an overview of and collaboration with existing initiatives.
  • Establish one or more structures to monitor infrastructure initiatives relating to training, validation and fine-tuning in the Norwegian health sector, cross-sectorally and in Europe, such as Sigma2, LUMI and Jupiter.
  • consider closer collaboration to link us to relevant European and/or Nordic initiatives and programmes such as testing and experimental platforms for AI (TEF Health).
  • establish closer cooperation with educational and research institutions.
• Depending on needs and existing initiatives, consider, for example:
  • to better facilitate the storage, linking and use of both historical data and prospective data acquisition. This applies to data from medical examinations, such as EEG, neurography or ultrasound, data from patient records, data from registries, etc.
  • the need for and possible organisation of a national infrastructure to make available Norwegian training data for the use of large language models in the health and care services (see also the focus area concerning the use of large language models).
  • establishing a national organisation of federated machine learning, including, for example, common definitions of variables.
  • establishing national testing and validation data sets, which will facilitate the re-use of validations.
  • establishing registries of national and international datasets that are available for AI research.

Work should be commenced through the series of seminars proposed in section 1.2.

This measure is viewed in the context of the assignment entitled "Secure access to language models adapted to Norwegian conditions" (Sikre tilgang på språkmodeller tilpasset norske forhold) and the Ministry of Education and Research’s assignment concerning supercomputing and academia. The measure is also viewed in the context of relevant EU initiatives (see section 1.4).

Current collaboration

The Norwegian Institute of Public Health, the Norwegian Directorate of Health, the regional health authorities, the municipal sector, the health and care services, the research and education sector and the Norwegian Ministry of Trade, Industry and Fisheries (NFD).