Effective Innovation Intermediary for AI adoption
Sengmeng KOO
AI Singapore, skoo@aisingapore.org
Abstract – Many firms rely on innovation intermediaries
to evaluate promising technologies and develop them to a
suitable level of maturity for firms to profit from
adoption.
AI technology helps firms solve many
problems and create new opportunities. Still, only a few
firms have successfully adopted AI into their business
practice. This paper examines the reasons behind the
low adoption, the AI-specific capabilities needed for
successful AI adoption, and what intermediaries need to
do to promote better results and outcomes. A case study
is done on an existing intermediary specializing in AI
who has completed over 40 AI projects in the last five
years. The study synthesized its practices and identified
the configuration of factors behind its project successes.
From the study, a new AI Intermediary framework is
proposed to understand better the capabilities needed for
an intermediary to be effective in AI adoption and how it
can play a more active role in the joint exploration and
creation of AI knowledge and value as demand for AI
continues to grow.
Artificial Intelligence, Innovation Intermediary, Technology
Transfer, National Transformation
Many technology-based firms rely on intermediaries to
evaluate strategic technological opportunities and develop
them to a sufficient level of maturity so that the firms can
commercialize and profit from customer adoption
(Chesbrough 2003, Howells 2006, Alexander & Martin
2013, Clayton et al. 2018). The importance of Artificial
Intelligence (AI) has been well-researched and documented
and governments worldwide also provide many incentives
for firms to adopt AI (OECD.AI).
Given the limited research and evolving nature of the topic,
a case-study approach is used to identify an existing
intermediary specializing in AI (Yin 2009) to identify its
capabilities and key practices that help firms to adopt AI
successfully.
With the results synthesized from the case study, we
propose a new framework for an effective AI intermediary
and suggest how an AI intermediary can take an active
leadership role to promote better results and outcomes for AI
adoption. This study also has broader implications for how
intermediaries can evolve and position themselves to deliver
value for future general-purpose technologies (Lipsey et al.
2005) that come after AI.
TECHNOLOGY ADOPTION WITH INTERMEDIARIES
Firms innovating with technologies typically prefer those
with high maturity levels so that they can commercialize at
speed to outperform competitors (Coyne & Subramaniam
1996). Large firms establish research and development
(R&D) units to identify promising technology, and research
and develop them to a sufficient level of maturity for
commercialization efforts.
Most firms lack such internal capabilities or do not see
internal R&D as cost-effective. Thus, they often turn to an
intermediary to evaluate a promising technology and work
with external R&D to develop the technology to a sufficient
level of maturity that the firms can integrate internally for
commercialization. This approach has been prevalent since
the wave of open innovation (Chesbrough 2003, Howells
2006, Alexander & Martin 2013, Clayton et al. 2018).
Yet very few firms have successfully adopted AI (Zolas
et al. 2020, IBM 2021, Staff, V. 2022) and 7 out of 10
companies reported no value from their AI investments
(Davenport & Zhang 2021). The reason is that
commercializing AI is complex and requires specific
technical capabilities to deploy and maintain AI models in
production. These AI-specific capabilities are rarely present
in innovation intermediary actors (Sculley et al. 2015, Zhou
Y et al. 2020, Studer et al. 2020).
To understand how intermediaries help firms innovate
with technology, we use the technology readiness level
(TRL) to illustrate the interactions and roles between the
typical innovation intermediary actors (the firm, the
intermediary and the R&D performer) at the various TRL
and how the intermediary coordinates between the different
actors and drives the technology adoption from idea to
validation and commercialization.
This traditional
intermediary framework is shown in Table 1.
Therefore, the traditional framework many innovation
intermediaries use becomes ineffective for AI adoption.
We need to understand these AI-specific capabilities and
what type of framework can bring together the right
innovation intermediary actors to increase AI adoption.
In this model, R&D performers (typically public and
private research institutions and technology start-ups)
develop new and novel forms of technology between TRL 1
to 3 and look for commercial adoption to fund further
research. In the case of technology start-ups, they look for
TABLE 1
INTERMEDIARY FRAMEWORK ACTIVITIES
firms to acquire their technologies. An intermediary
regularly reviews technology offerings from these R&D
performers at TRL 3, evaluates commercialization potential
and seizes market opportunities based on their understanding
of the market and business needs of the firms (Lichtenthaler
and Ernst 2008, Tran et al. 2011).
When there is commercial interest from the firm, the
intermediary brokers a collaboration between the R&D
performer and the firm (Hargadon 2002, Agogué et al. 2013)
and manages the technology transfer activities (Nambisan et
al. 2012). The R&D performer further develops the
technology to an agreed level of maturity, which the
intermediary then packages to transfer through licensing
agreements (Shohert and Prevezer 1996) to complete the
technology adoption process.
This framework has been successful in the adoption of
many technologies. The intermediary supports the R&D
performer in developing the technology from an idea (TRL 1
to 3) to a prototype where its performance is validated and
shows clarity of its market potential (TRL 4 to 6). As a
result, the firm obtains a technology of sufficient maturity
that it can commercialize and recoup its technology
investment (TRL 7 to 9) and at the same time, has a
resource-effective way to experiment and innovate with
technology, combining new technologies with existing ones
in new ways (Hargadon 2002), sometimes discovering new
solutions and products (Turpin et al. 1996).
Through this ftramework, the intermediary also
influence the speed of technology diffusion and the creation
of new technological inventions (Howells 2006). The
intermediary generates tacit and codified knowledge from
each collaboration, which helps to improve the
implementation
practices
and
overall
innovation
performance over time (Nielsen 2005, Martín-de Castro
2015). The intermediary also promotes trustworthy
relationships, which address common failure points in
collaborations by promoting information openness (Bruneel
et al. 2010) and reducing risk (Vlaar et al. 2007).
ARTIFICIAL INTELLIGENCE (AI)
In the past ten years, AI has become an essential generalpurpose technology (Lipsey et al. 2005) with a wide range of
socio-economic impacts. When incorporated into smart
products and intelligent services, these AI systems (AIS)
enhance decision-making capabilities, automate processes
and discover new business opportunities offering
tremendous competitive advantages to industry adopters
(Agrawal 2018, Horowitz & Kahn 2021, Nepelski et al.
2020).
OECD – a global intergovernmental economic
organization – has documented over a thousand national
initiatives and funding on AI across sixty-one countries as of
2021, as shown in Table 2.
It can be generalized that firms are actively incentivized
and supported to adopt AIS and for most firms,
intermediaries and R&D performers play key roles in their
AI adoption.
However, studies have shown comparatively low AI
adoption by firms. For example, a US Census Bureau survey
of 583,000 businesses revealed that only 2.8 percent had
adopted AI (Zolas et al. 2020); AI is mostly used in single
pilots by firms and only 8% have adopted AI in core
practices (Fountaine et al. 2019). IBM found only 21% of
5,501 companies to have deployed AI systems across the
business with the rest still “exploring” (IBM 2021). In
addition, 87% of AI models trained are never put into
production (Staff, V. 2022) and an MIT Sloan Management
Review/Boston Consulting Group survey in 2019 found that
7 out of 10 companies reported no value from their AI
investments (Davenport & Zhang 2021).
The low rate of adoption and low return on investment
implies there are complexities specific to AI adoption
differentiating AI from other digital technologies which are
typically easier to deploy (Lokuge et al. 2019). This raises
important questions about these complexities and how they
affect AI adoption. Understanding them will also reveal
what kind of intermediaries can address those complexities.
In the next section, we explain the typical development
cycle of an AI system (AIS) and what successful AI
adoption looks like.
ADOPTING AI TECHNOLOGY
Figure 1 describes how an AI technology goes from its first
journal publication to be integrated into commercial
products and services using the earlier TRL that stretches
from idea conception to commercialization. In the initial
idea phase (TRL 1 to 3), the R&D performer invents or
discovers a novel AI method or a new machine learning
algorithm, publishes its scientific value in outlets like
journals and develops an AI model into an experimental
proof-of-concept to demonstrate its potential application
areas.
In the next phase of prototyping and validation (TRL 4
to 6), the firm or intermediary interested in using the AI
model in commercial applications works with R&D
performers to train the AI model to deliver the specified
performance. Training the AI model is done with data sets
supplied by the firm, usually static data sets extracted from
the firm’s production environment or reference data sets
publicly available that fits the firm’s intended business
outcome.
However, this model appears to be ineffective regarding AI
adoption. Most AI models trained are never put into
production (Staff, V. 2022), suggesting the lack of, or
absence of AI-specific capabilities within the firm to deliver
a successful AI adoption.
The following section will explain what AI-specific
capabilities are.
AI-SPECIFIC CAPABILITIES
ML-Ops is an AI engineering discipline that combines Dev
Ops, Data Engineering and Machine Learning and represents
basic capabilities required by anyone to leverage the
business value of an AI solution. In a real-world enterprise
AI deployment, only a small fraction consists of the AI
model (or machine learning code), as shown by the small
black box in figure 2.
The required surrounding
infrastructure is vast and complex.
These various
infrastructure components embed the AI model into the
existing processes and systems across the firm’s production
environment to deliver the business value of the resulting AI
solution.
ML-Ops are still a relatively new AI discipline with
embryonic commercial activity (Sculley D et al. 2015). In
the case of a large firm, each of the activities represented in
figure 3 would be delegated to separate departments. As for
small and medium enterprises, the activities would be
delegated to an individual or group with extensive crossfunctional technical capabilities.
A successful AI deployment will require the AI model
to be integrated with all the different components and
operationalized within the firm’s business activities. The
“operations” in ML-Ops include the acquisition and cleaning
of the big data that is strategic to the firm’s business
activities; the tracking and the versioning of AI model
training and experiments for improvements; the continuous
deployment and monitoring of the machine learning
pipelines; and scaling the AI deployment to meet changing
business needs of the firm.
In the final stage of validation and commercialization
(TRL 7 to 9), the trained AI model is integrated into the
firm’s product or production environment and validated in
operations before the firm begins to profit from customer
adoption.
Thus, firms must be fluent in DevOps and data
engineering to perform successful ML-Ops. This is why
many well-known AI firms, such as Alibaba, Amazon,
Google, Huawei and Meta, are previously digital-first
companies with DevOps and data engineering as core
organizational capabilities.
Thus, a successful AI adoption by a firm happens when
its investment in AI technology reaches TRL 9, where the
performance of the AI model delivers continuous business
value for its intended application purpose. In the traditional
intermediary framework, the intermediary engages the R&D
performer to develop the technology to TRL 6. At that level,
the AI model is trained to deliver the performance in a test
environment. Then, the firm takes over to execute TRL 7 to
9 to deliver the business value as illustrated in figure 1.
Some of these AI firms offer their infrastructure as
platforms for use by other firms, for example, Amazon Web
Services, Google Cloud Platform and Alibaba Cloud. A
recent study shows that 78% of all enterprise AI
deployments are done on ML-Ops platforms (Sacolick
2021). This is done as a Software-as-a-Service (SaaS)
model which other firms pay to host their AI solutions and
datasets. This approach is different and disconnected from
TABLE 2
NATIONAL AI POLICIES AND STRATEGIES (OECD.AI)
FIGURE 1
DEVELOPMENT OF AI FROM INVENTION AND COMMERCIALIZATION
FIGURE 2
SYSTEMS OF A TYPICAL REAL-WORLD AI DEPLOYMENT (SCULLEY ET AL.
2015)
the traditional intermediary context. The AI firms do not act
as knowledge transfer partners to help firms innovate with
technology. There are also AI startups such as H20.ai and
DataRobot helping companies towards automating the
deployment of AI models and running ML-Ops. Firms
engaging their services must already have a trained AI
model or solution for deployment; thus, it is not the right
match for firms looking for intermediaries to help with
technology innovation.
EFFECTIVE INTERMEDIARY FOR AI ADOPTION
We can see that successful AI adoption happens when a firm
performs ML-Ops to deploy the AI model into production
and continuously maintain and improve the AI solution
(TRL 7 to 9). Unfortunately, the capabilities are not present
in most firms exploring AI innovation for the first time,
accounting for the high rate of AI projects never going into
production and low AI adoption reported (Fountaine et al.
2019, Zolas et al. 2020, IBM 2021, Davenport & Zhang
2021).
When the firm turns to an intermediary to help with AI
adoption, the traditional intermediary model becomes
ineffective as it cannot provide the AI-specific capabilities to
deliver success. The intermediary’s network of R&D
performers are typically university faculty members or
public researchers with experience up to TRL 6 in primary
or translational research and lack practical ML-Ops
experience. ML-Ops platform providers and specialized AI
startups can provide operational support, but it is difficult to
rely solely on them.
As far as the author can tell, there has been limited
research focused on intermediaries specializing in AI and
what type of intermediary is effective in AI adoption.
Nevertheless, firms will continue to view AI as an
imperative enabler for their growth (Agrawal 2018,
Fitzgerald et al. 2014) and how they can be successful
working with intermediaries is important.
Given this subject's nascent and evolving nature, a casestudy approach (Yin 2009) is used to understand what an
effective AI intermediary looks like and what AI-specific
capabilities it must have to increase AI adoption.
To
conduct the case study, we identified an AI intermediary that
has helped more than 40 firms develop deployable AI
models. In the next section, we conduct a comparative
analysis of the projects the AI intermediary has undertaken
to identify the success factors and propose a new
intermediary framework for effective and efficient AI
adoption.
STUDY
AI Singapore (AISG) is a public innovation intermediary
established by the Singapore government in July 2017 using
the classical triple-helix innovation model (Etzkowitz &
Leydesdorff 1995) to develop a vibrant local AI ecosystem
in Singapore and drive AI adoption.
AISG coordinates with six autonomous universities and
public research institutes, tapping on over 200 local AI
researchers to perform use-inspired research and
development, help firms build deployable AI systems (AIS),
and grow the talents necessary to sustain the AI ecosystem,
such as the apprenticeship program to train qualified AI
engineers.
AISG is a good case study candidate as its five years of
accumulated experience provide good examples and study
materials. In addition, its public information repository is
easily referenced to contribute to future research interests.
The study was done in two parts through interviews and
discussions with project managers and AI engineers
involved in AIS project execution. First, we reviewed the
completed projects and presented four exemplars
highlighting the operational issues and challenges they faced
during the execution. Second, we examined the practices
AISG had adopted over five years of experience.
Understanding the practices helped us to identify the key AIspecific capabilities, recognize the interactions between
AISG and the actors in an intermediary mode, and how they
overcome challenges to complete successful AI adoption.
EXECUTING AN AIS PROJECT
AISG adopts an Agile methodology when executing AI
system projects. Agile offers many advantages to delivering
working and effective AI models (Amershi et al. 2019,
Schleier-Smith J, 2015). AI model training requires many
iterations. Using Agile, the development team can focus on
developing a minimal viable model at each sprint, receive
the firm’s inputs and incorporate the necessary changes
towards the next sprint. The constant review and iteration
development process allow the firm to have continuous
inputs into the AI model performance and react to changes
in the business environment that will affect the efficacy of
the planned AI system. The AISG development team can
take advantage of the constantly advancing field of AI and
machine learning during the project period, to introduce
more efficient and powerful algorithms into the final AI
system.
The typical Agile process is shown in table 3. While
each project will have minor variations (for example, model
training and fine-tuning might be completed by the
discussion on deployment architecture can commence), all
projects follow an agile development period of seven months
over nine sprints.
TABLE 3
AISG INDUSTRY PROJECT MANAGEMENT TEMPLATE
FIGURE 3
ACTIVE AND COMPLETED AIS PROJECTS AS OF SEPTEMBER 2022
Parallel to the execution of AIS projects, AISG recruits
and trains aspiring AI engineers via its AI Apprenticeship
Programme (AIAP)®. Applicants must pass a technical
proficiency exam before being accepted into a 9-month fulltime apprenticeship that will see them working alongside
AISG’s staff engineers and the firms.
The apprentices are responsible for the technical
deliverables of the AIS project assigned, supervised by
AISG’s staff engineers. Firms can assess their apprentices
for hire at the end of the project to help deploy AI systems
and run ML-Ops.
AISG has completed over 49 AIS projects and another
42 ongoings at the time of conducting the study, as shown in
figure 3. Here the definition of “complete” meant the
project sponsor (firm) had accepted and signed off the AI
model and supporting documentation.
Four projects were selected to generalize the execution
of AIS projects and highlight the operational issues and
challenges that arose.
I. Dental Treatment Decision Support Systems
A leading private dental healthcare group in Asia
approached AISG to build an AI-enhanced dental treatment
decision support system for managing common dental
problems, such as Interradicular Bone Loss and Horizontal
Bone Loss. This support system must be able to reduce the
incidence of missed diagnoses from dental x-rays and
highlight problems that are either impossible or difficult for
human visual inspection. AISG successfully trained the
model with retraining capability and delivered a dockerized
application integrated with the group’s existing backend for
immediate deployment.
Commitment and constant communication between the
project sponsor and the AISG engineering team contributed
to the success. The project sponsor was very clear on their
business value and came prepared with quality datasets and a
supporting research paper (Kanagasingam et al 2016). This
enabled AISG to focus on the technical delivery within the
project timeline. The regular sprint updates and progress
increased cooperation and the project sponsor put in
additional investment and manpower to improve their data
governance as per AISG’s advice. This allowed the sponsor
to fully realize the benefit of an AI model retraining pipeline
that will continuously improve the system during actual use.
The sponsor initially planned for the finished project to
be deployed and maintained through a cloud provider.
However, after acquiring project experience and the
availability of qualified AI engineers, the sponsor decided to
spin off a new AI division and hired AISG’s engineering
apprentice to manage the deployment and conduct further
research and development. The new division will also
explore a SaaS model (Software as a Service) to offer to
other dental industry players.
II. Product Quality Risk Classification Project
A global technology leader in cloud solutions and cognitive
computing services with over $50 billion in annual revenue
and operating in over 170 countries, approached AISG to
evaluate suitable AI technologies to enhance their existing
business solutions.
Their engineers are constantly
overwhelmed by large volumes of real-time data and
analyzing them to derive meaningful business insights is a
big challenge.
The project sponsor was a senior executive of the
analytics solutions team, with operational experience in data
analytics and prediction modeling software. With the firm’s
familiarity with data analysis and engineering, the discussion
on a potential AI project went smoothly. The sponsor
decided on Quality Problem Detection of their storage
devices business unit as the project scope. The project
encountered difficulties during execution, primarily with the
datasets and the production environment. Exploratory data
analysis (sprint 1) revealed errors in the original code scripts
for raw data processing which need to be modified by the
apprentices.
The firm also had problems providing
machines to test the model in production. The project
sponsor is a proponent of Agile and both teams were able to
agree on revised sprint deliverables within the available
time. Fortunately, the assigned apprentices were still able to
improve the current prediction accuracy and increase the
quality problem detection significantly (reduction of average
inspection time from 30 mins to less than 5 minutes). AISG
deployment experience also helped the firm with its
deployment issues and since then, the firm reported a cost
savings of 3 times for the business unit.
AISG in post-project Agile reflection decided to
increase emphasis on the sponsor’s data readiness during
project suitability evaluation and conduct a baseline model
review before approving the project. They also introduce a
new standard practice to include an interim deployment
milestone at sprint 6 to mitigate the risk of deployment
problems in the last sprint.
III. Digital and Computational Pathology project
Singapore’s largest acute tertiary hospital wished to improve
patient care and advanced their pathology practices by
implementing a fully digital histopathology workflow.
Recent advances in clinical practice have opened the
possibility of using AI to analyze histopathological features
of biopsies captured in whole-slide images (WSIs) that assist
in pathological classification and diagnosis (Cheng et al
2021).
In general, healthcare AI projects have two primary
challenges in development and implementation – data
availability and AI model explainability (Aniek et al 2021).
The provision of quality and well-annotated data contributed
to smooth development sprints. AI model explainability and
interpretability are also essential in a regulated industry such
as healthcare. AISG incorporated a LIME tool to help the
project sponsor to visualize the edges of the suspected
tumors in the WSI that is used to generate the prediction
results.
LIME, which stands for local interpretable modelagnostic explanations, is a technique that approximates any
black box machine learning model with a local, interpretable
model to explain each individual prediction. For this project,
the medical user knows exactly the areas of the WSI used to
generate the prediction, greatly increasing the confidence
and efficacy in interpreting the AI results.
This was the first project to incorporate interim
deployment milestones earlier than the usual sprint 6. At the
onset of the project, there were insufficient data for model
training and introducing a prototype early into the sponsor
production environment helped more stakeholders to
understand and appreciate the AI functions. The project
sponsor was able to access and provide better data over time
and the increased cooperation becomes a key factor for
success.
AISG has since introduced interim deployment
into its Agile process for projects with a risk of insufficient
or low-quality data.
IV. Delivery Agents efficiency project
One of Singapore’s on-demand delivery platforms faced
operational issues with its decentralized distributed model.
It provided flexible delivery options for customers as a
market differentiator with its crowdsourced network of
mobile delivery agents organized around postal regions.
However, the model resulted in multiple delivery agents
accepting jobs with similar pick-up and drop-off points,
reducing operational capacities.
The project sponsor
approached AISG to develop an intelligent system that will
create jobs “bundles”, allows delivery agents to job pick
more efficiently and have a knock-on effect of shortening
the delivery lead time for the entire network.
There existed mature AI models for route planning and
optimization and AISG has, in fact, completed a previous
industry project for such a use case. While AISG can
authoritatively validate the business value of the project, it
knows those models are optimized for a centralized
distributed model with warehouse consolidation and not for
the firm’s decentralized model. From their recent project, its
platform and engineering teams determine that an academicbased R&D performer is not required to deliver the technical
performance of the AI model. AISG conducted at least two
discovery sessions to derive the solution architecture to
reduce the cognitive load of the delivery agents when they
are job picking, and to develop a clustering algorithm that
the firm successfully deploys into production and adopts into
core practice in three months (half the time of a typical
project).
Because of gained expertise from their recent project, its
platform and engineering teams did not require a researcher
to be involved as an innovative performer and developed a
new clustering algorithm in three months (half the time of a
typical project) which the firm successfully deployed into
production and adopt into core practice.
AI SINGAPORE PRACTICES
AISG takes an active role in right-sizing companies to
ensure optimal use of public funds in supporting AI projects.
Consequently, AISG uses organization readiness assessment
and project suitability evaluation as two main right-sizing
criteria to ensure a high probability of AIS project
completion and deployment before public funding is
approved to support the firm.
Organization readiness assessment. The goal is to
understand the readiness level of the firm to adopt AI and its
clarity on AI investment in the project. The assessment is
done with the project sponsor, usually a company executive
of sufficient seniority who also possess a complete
understanding of the firm’s overall structure in order to
provide a realistic view on the firm’s ability to realize the
benefits of the AIS project when it is completed and
deployed. The project sponsor also helps examine whether
the firm can identify the business value that needs to be
delivered from the project.
Project suitability evaluation. When the firm has
demonstrated clarity and sufficient readiness level, AISG
will co-develop with the firm the technical scope of the AIS
project, often with the assistance and inputs from the
company’s technical representatives. Together they develop
a plan that can deliver the agreed business value. The scope
identifies the database schema availability and suitability,
the viability of AI models, and the relevant expertise and
availability of the R&D performer from the public research
institutes. This is an important process in evaluating the
project’s suitability and can be iterated more than once
before the funding is approved.
Deployment end-goal. The project then moves into the
execution stage to deliver the agreed project scope. AISG
delivery is distinctly different from the traditional
intermediary model which stops at TRL 6, i.e. demonstrating
the technical value of the technology (in this case the AI
model). AIS projects are executed to the point in which the
business value is shown at TRL 7 to 8, i.e. an AI model
shown to work in a production environment and can be
moved into commercial deployment as the next step. To
facilitate rapid deployment at the end of the AIS project, any
resulting foreground IP created will be jointly owned by the
firm and the R&D performer as part of the funding terms
and conditions.
The practice has largely remained the same over the
AISG’s four years of operations. But as AI technology
knowledge commoditizes and standardized AI models have
matured for common industry problems, AISG has evolved
its practices with the following three key changes in order to
become more efficient and effective to drive successful AI
adoption and grow a vibrant AI ecosystem as part of its
public mandate.
Flexible involvement of R&D performers. R&D
performers used to be exclusively researchers from public
research institutes. As the AISG gained experience with
each completed industry project, it grew internal AI
engineering capabilities to assume the role of R&D
performer in the TRL 4 to 6. For projects which do not need
the full involvement of academic-based R&D performers,
AISG uses its own AI engineering team to develop the
solution and whenever needed, engages the researchers on a
consultant basis.
With this approach, they could shorten
the development cycle from the original 18 months to 9
months. Researchers also welcome this change as they can
contribute to the project in a time-efficient manner and focus
on their research works
Establishing platform and engineering teams.
Recognizing AI-specific capabilities critical to successful AI
adoption as described in the earlier section, AISG also grows
ML-Ops engineering capability that can bring the AIS
project to the maturity level of TRL 7 to 8 for the firm’s
adoption. It also forms partnerships with ML-Ops platform
providers and trains their internal teams so that developed AI
models and solutions from the projects can be easily used by
firms who wish to use platform providers’ services.
AI manpower diffusion into the ecosystem. AISG
developed a world-leading AI talent development program
that trains AI engineers to be released into Singapore’s AI
ecosystem. Branded the AI Apprenticeship Programme
(AIAP)®, AISG has trained over 200 qualified AI engineers
with operational experiences across TRL 4 to 8. This trained
manpower allows AISG to mainstream the institutional
knowledge it has accumulated from executing AI projects
and reduce the complexity and risk for firms to adopt AI.
KEY SUCCESS FACTORS AND LESSONS LEARNT
Three factors were identified from the case study as
important in successful AI adoption by firms, shown in
figure 4.
AIS projects, and increasing public value creation (Stan and
Vermeulen 2013).
Ensure business value delivery. AISG stresses the
deployment viability of all completed projects and firms’
commitment to move into production or commercialization.
In fact, one of the terms and conditions in accepting AISG
funding holds firms accountable for moving the deployed AI
model into production/commercialization.
To do that,
AISG ensure all projects development is done to a marketready level of technology maturity to maximize value
transfer (Chakrabarti & Rubenstein 1976) and also train
qualified AI engineers that can be hired by the firm to put
the AI solution into adoption,
This help to meet the
challenge which majority of industry AI projects do not
yield business outcomes (Fountaine et al 2019, Zolas et al.
2020, IBM 2021, Davenport & Zhang 2021).
Flexibility to accommodate changing innovation
process. By adopting an agile approach to helping firms to
adopt AI, AISG can pass down the benefits of evolving AI
technologies and maturity of AI tools to the firms,
shortening development and deployment cycles and helping
the firms recoup their AI investment faster. The approach
also helps to involve PRI appropriately.
Using the unpacked information from the case study, we
propose a new framework for AI intermediaries in the next
section.
NEW INTERMEDIARY FRAMEWORK FOR AI
A new framework is proposed for AI intermediaries and
existing technology intermediaries that are helping firms to
adopt AI by developing AI systems (AIS), as shown in
figure 5. Building on the traditional intermediary approach,
the new framework consists of three components and
introduces a new intermediary actor, the deployment
performer.
Firms need to be primed. Whilst this seems
counterintuitive for a public intermediary such as AISG to
be selective in accepting AIS projects to help and fund, we
see firms need to be at the right readiness level and the
project has to be selected with a high probability of success.
With this, AISG can maximize public value creation with
limited funding. Selective and priming ensure continual
successes that will continue to generate positive sentiments
for more adoption.
Technology value validation.
As an effective
innovation partner, the AI intermediary guides the firm to be
a successful adopter of AI and reduces the adoption risk
(Winch & Courtney, 2007). To do that, the AI intermediary
can assess the firm’s ability to innovate with AI and guide it
on clearly identifying the business value they expect from its
AI investment. The guide can be in the form of an
assessment as with AISG in the case study, or it has the
internal ability to authoritatively validate AIS business value
on behalf of the firm.
The careful selection of companies did not imply that
AISG took simple AI projects with known solutions. Every
AI project is unique and challenging. By taking on every
qualified project including challenging ones, AISG enhances
the general abilities and the skill levels of all actors
involved, promoting and improving success rates for future
The AI intermediary must have a clear understanding of the
relative advantage and knowledge associated with the
development and utilization of AI so that as AI technology
matures and AIS improves, it can provide a better
assessment to the firm and streamline its intermediary
practice to accelerate the technical validation of the AI
FIGURE 4
SUCCESS FACTORS OF AI ADOPTION PROGRAM OF AI SINGAPORE
FIGURE 5
NEW AI INTERMEDIARY FRAMEWORK
AI model so that firm can commercialize quickly to
outperform and outexecute competitors (Coyne &
Subramaniam 1996).
Business value creation. Firms need to see positive
returns from their AIS investment.
To do that, the AI
intermediary ensures the AI model is deployed into
production to realize the business value and provide the
deployment capabilities during the AIS project. Here we
introduce a new intermediary actor named Deployment
Performer.
The deployment performer develops the AI
model to the sufficient maturity level of TRL 8 to address
the current low adoption rate where most AI models trained
are never put into production (Staff, V. 2022).
The AI intermediary includes this actor into the
framework by either bringing in ML-Ops platform providers
and start-ups or helping firms build their internal ML-Ops
capabilities. The latter is preferred in the long run, as MLOps platform providers and start-ups provide a hosting
service, rather than an innovation partner. A firm innovating
with AI should invest internally to either manage ML-Ops
themselves or be technically proficient at maximizing their
partnership with ML-Ops platform providers and start-ups.
The AI intermediary helps the firms by growing the
talent ecosystem for hire. It can work with institutes of
higher learning on curriculum development and practices or
reference from AISG to build a similar apprenticeship
program. Either way sees the AI intermediary helping the
firm to grow their AI engineering competency and maximize
business value creation for every AIS project.
Technology leadership in AIS adoption. While firms
have extensive support and incentives to adopt AI
(OECD.AI), only a few can deploy AIS or use AI in core
practices (Fountaine et al. 2019, Zolas et al. 2020, IBM
2021). The AI intermediary influences more AI adoption
by reducing the complexity and risk and speeding the AI
technology diffusion (Howells 2006). To do that, the AI
intermediary would share its project management best
practices and engineering standards for AIS projects. Even
for a commercial intermediary such as innovation capitalists
(Nambisan 2012), establishing itself as a trust advisor and
platform is the commercially sound approach to generate
more clients.
CONCLUSION
AI adoption continues to be imperative for companies to
grow and stay competitive (Coyne & Subramaniam 1996)
and AI intermediaries play important roles in diffusing AI
technology into society at large, ensuring that its scientific
and economic progress is equally distributed and not being
monopolized by only a few firms (Mormina 2019).
The study of an existing AI intermediary revealed key
practices and capabilities needed for an intermediary to be
effective in AIS adoption and highlight operational
experiences to reference, as well as provide a better
understanding of how different intermediary actors
collaborate in the joint exploration and creation of AI
knowledge and value.
This paper proposes a new framework for AI
intermediaries that will effectively help firms adopt AI, and
suggest how existing intermediaries can evolve beyond the
traditional brokering of knowledge to be in a technology
leadership role to promote better results and outcomes for AI
adoption. Further studies of the framework can examine the
value technology leadership brings and what effective
structure and practices can be. Those results will help
position intermediaries to deliver value for future generalpurpose technologies after AI.
ACKNOWLEDGMENT
I would like to thank my paper supervisors, A/Prof M
Subramanian Annapoornima of the National University of
Singapore, and Director Laurence Liew of AI Singapore for
their guidance. I acknowledge the help from the AI
Singapore team in providing the information and statistics
for the case study. Any errors and omissions are my
responsibility.
REFERENCES
Agogué M., Yström A. and Masson P.L. 2013. Rethinking the role of
intermediaries as an architect of collective exploration and creation of
knowledge in Open Innovation. International Journal of Innovation
Management 17(2).
Agrawal A.K., Gans J.S. and Goldfarb, Avi. 2018. Economic Policy for
Artificial Intelligence. National Bureau of Economic Research. Working
Paper Series 24690.
Alexander A.T. and Martin D.P. 2013. Intermediaries for open innovation:
A competence-based comparison of knowledge transfer offices practices.
Technological Forecasting & Social Change, 80, 38–49.
Amershi S., Begel A., Bird C., DeLine R., Gall H., Kamar E., Nagappan N.,
Nushi B., and Zimmermann T.. 2019. Software engineering for machine
learning: a case study. International Conference on Software Engineering:
Software Engineering in Practice, IEEE Press, 291–300.
Aniek F. Markus, Jan A. Kors, Peter R. Rijnbeek 2021. The role of
explainability in creating trustworthy artificial intelligence for health care:
A comprehensive survey of the terminology, design choices, and evaluation
strategies. Journal of Biomedical Informatics 113.
Bruneel J., D’Este P. and Salter, A. 2010. Investigating the factors that
diminish the barriers to university-industry collaboration. Res. Policy 39,
858–868.
Chakrabarti A., Rubenstein AH. 1976. Inter-organization transfer of
technology: a study of adoption of NASA innovations. IEEE Trans. Eng.
Manag. 23(1), 20-34.
Cheng C.L., Md Nasir N.D., Ng G.J.Z. et al. 2021. Artificial intelligence
modelling in differentiating core biopsies of fibroadenoma from phyllodes
tumor. Lab Invest 102, 245–252.
Chesbrough H. 2003. Open Innovation: The New Imperative for Creating
and Profiting from Technology. Boston, MA: Harvard Business School
Press.
Clayton P., Feldman M. and Lowe N. 2018. Behind the scenes:
Intermediary organizations that facilitate science commercialization through
entrepreneurship. Academy of Management Perspectives 32(1), 104–124.
Coyne K.P. and Subramaniam S. 1996. Bringing discipline to strategy. The
McKinsey Quarterly. New York Issue 4.
Davenport T. and Zhang R. 2021. MIT Sloan Review. Retrieved 16 Apr
2021, from sloanreview.mit.edu/article/ achieving-return-on-ai-projects.
Etzkowitz H. and Leydesdorff L. 1995. The Triple Helix - UniversityIndustry-Government Relations: A Laboratory for Knowledge Based
Economic Development. EASST Review 14(1), 14-19.
Fitzgerald M., Kruschwitz N., Bonnet D. and Welch M. 2014. Embracing
digital technology: A new strategic imperative. MIT Sloan Management
Review 55(2). 1-12.
Fountaine T., McCarthy B. and Saleh T. 2019. Building the AI-powered
organization. Harv Bus Rev 97:62–73.
Hargadon A.B. 2002. Brokering knowledge: linking learning and
innovation research. Organ. Behav. 24, 41–85.
Horowitz M.C. and Kahn L. 2021. What influences attitudes about artificial
intelligence adoption: Evidence from U.S. local officials. PLoS One 16(10).
Howells J. 2006. Intermediation and the role of intermediaries in
innovation. Research Policy 35: 715–728.
IBM Global AI Adoption Index 2021. 2021. Retrieved 18 Apr 2021, from
newsroom.ibm.com/IBMs-Global-AI-Adoption-Index-2021.
Kanagasingam S., Lim C.X., Yong C.P., Mannocci F. and Patel S. 2016.
Diagnostic accuracy of periapical radiography and cone beam computed
tomography in detecting apical periodontitis using histopathological
findings as a reference standard. Int Endod J.
Lichtenthaler U. and Ernst H. 2008. Intermediary services in the markets for
technology: organizational antecedents and performance consequences.
Organ. Stud. 29, 1003–1035.
Lipsey R.G., Bekar C.T. and Carlaw K.I. 2005. General Purpose
Technologies and Economic Growth. Oxford University Press.
Lokuge S., Sedera D., Varun G.V. and Xu D. 2019. Organizational
readiness for digital innovation: Development and empirical calibration of a
construct. Information & Management 56(3). 445-461.
Martín-de Castro G. 2015. Knowledge management and innovation in
knowledge-based and high-tech industrial markets: the role of openness and
absorptive capacity. Ind. Marketing Manage. 47, 143–146.
Mormina M. 2019. Science, Technology and Innovation as Social Goods
for Development: Rethinking Research Capacity Building from Sen’s
Capabilities Approach. Sci Eng Ethics 25, 671–692.
Nambisan S., Bacon J. and Throckmorton J. 2012. The Role of the
Innovation Capitalist in Open Innovation, Research-Technology
Management, 55(3), 49-57.
Nepelski, D. and Sobolewski M. 2020. Estimating investments in General
Purpose Technologies: The case of AI Investments in Europe. Publications
Office of the European Union EUR 30072 EN.
Nielsen B. 2005. The role of knowledge embeddedness in the creation of
synergies in strategic alliances. J. Bus. Res. 58, 1194–1204.
OECD.AI. (n.d.). OECD AI's live repository of over 260 AI strategies &
policies. Retrieved June 18, 2022, from oecd.ai/en/dashboards.
Sacolick I. 2021. How the public clouds are innovating on AI. Retrieved 23
April 2022, from www.infoworld.com/article/3642468/how-the-publicclouds-are-innovating-on-ai.html
Schleier-Smith J. 2015. An Architecture for Agile Machine Learning in
Real-Time Applications. International Conference on Knowledge Discovery
and Data Mining. Association for Computing Machinery, NY, USA, 2059–
2068.
Sculley D., Holt G., Golovin D., Davydov E., Phillips T., Chaudhary V.,
Young M., Crespo J. F. and Dennison D. 2015. Hidden Technical Debt in
Machine Learning Systems. Advances in neural information processing
systems 28.
Shohert S. and Prevezer M. 1996. UK biotechnology: institutional linkages,
technology transfer and the role of intermediaries. R&D Management 26,
283–298.
Staff, V. 2022. Why do 87% of data science projects never make it into
production?
Retrieved
18
April
2022,
from
venturebeat.com/2019/07/19/why-do-87-of-data-science-projects-nevermake-it-into-production/
Stan M. and Freek Vermeulen F. 2013. Selection at the Gate: Difficult
Cases, Spillovers, and Organizational Learning. Organization Science
24(3):796-812.
Studer S., Bui T.B., Drescher C., Hanuschkin A., Winkler L., Peters S. and
Mueller K.R. 2020. Towards CRISP-ML (Q): A Machine Learning Process
Model with Quality Assurance Methodology, arXiv.
Tran Y., Hsuan J. and Mahnke V. 2011. How do innovation intermediaries
add value? Insight from new product development in fashion markets. R &
D Manage. 41, 80–91.
Turpin T., Garrett-Jones S. and Rankin N. 1996. Bricoleurs and boundary
riders: managing basic research and innovation knowledge networks. R&D
Management 26, 267–282.
Vlaar P.W.L., Van Den Bosch F.A.J. and Volberda H.W. 2007. Towards a
dialectic perspective on formalization in interorganizational relationships:
how alliance managers capitalize on the duality inherent in contracts, rules
and procedures. Organ. Stud. 28, 437–466.
Winch G. and Courtney R.C. 2007. The organization of innovation brokers:
An international review. Technology Analysis and Strategic Management.
November 2007.
Yin R.K. 2009. Case study research. Design and methods. 4. ed. Thousand
Oaks, pp.24.
Zhou Y., Yu Y. and Ding B. 2020. Towards MLOps: A Case Study of ML
Pipeline Platform. International Conference on Artificial Intelligence and
Computer Engineering, pp 494-500.
Zolas N., Kroff Z.,Brynjolfsson E., McElheran K., Beede D., Buffington C.,
Goldschlag N., Foster L. and Dinlersoz E. 2020. Advanced Technologies
Adoption and Use by U.S. Firms: Evidence from the Annual Business
Survey, United States Census Bureau. Working Paper CES-20-40.
AUTHOR INFORMATION
Sengmeng Koo, Senior Deputy Director of AI Innovations,
AI Singapore, and Deputy Director at the Office of Deputy
President (Research and Technology) of the National
University of Singapore – hosting institution for AI
Singapore program office funded by the National Research
Foundation of Singapore.