(Editor’s note: This is the second of a two-part series on AI centers of excellence. Part 1 covers the benefits of an AI COE and how to measure its performance.)
Many organizations are racing to adopt artificial intelligence in the hope of creating new business efficiencies, gaining competitive advantages, and boosting the bottom line. But a recent survey finds that most organizations face a series of challenges before they can reap benefits from those investments.
For instance, IDC’s July 2024 Future Enterprise Resiliency and Spending Survey found that 26% of respondents had already introduced several GenAI-enhanced applications or services into production, up from 17% of respondents in a similar IDC study in April. Common challenges slowing down GenAI deployments include securing private information, preventing hallucinations, controlling costs, as well as how best to monitor and manage GenAI applications in production (GenAI Operations: A Guide to People, Process, and Tool Requirements, IDC #US52781824, December 2024).
“Using AI and, more specifically, GenAI has become an all-encompassing strategic initiative for business — but it’s not yet clearly defined,” says Jason Hardy, CTO of AI at Hitachi Vantara.
That fact is inspiring some organizations to develop AI centers of excellence (COEs). The goals are to better understand AI capabilities, to align AI initiatives with broader organizational strategy and ethics, to build internal trust and external credibility, and to put governance and guardrails in place early in the process, explains Richard Buractaon, head of artificial intelligence at Andesite AI, an AI architecture firm based in McLean, Virginia.
“An AI center of excellence helps cut through the noise. Its role is to educate, dispel myths, and ground AI initiatives in reality,” Buractaon says.
Using the COE to Spread AI Knowledge in the Organization
Because widespread interest in AI is still fairly recent, there is a supply-and-demand gap for experienced AI professionals. An AI COE can help bridge this gap by gathering top employees from throughout the organization to work together in a new team, share expertise, and then bring newly gained knowledge and culture back to their original units.
For this reason, it is important that the “right” employees are assigned to an AI COE, which is expected to provide a community of practitioners who can share knowledge, expertise, and best practices in AI and related technologies, says Rick Torzynski, senior data and AI engineer and product architect at ECS, a leading provider of cloud, cybersecurity, AI, machine learning, and IT modernization services in Fairfax, Virginia.
“The COE should generate excitement and interest in AI and related knowledge domains, encouraging employees to learn and explore new technologies,” Torzynski explains. “The COE should also provide training and development opportunities for employees, enabling them to acquire the skills and expertise needed to work with AI and related technologies.”
Experiences and Skills Wanted with Team Members
When HItachi Vantara builds out an AI COE team, it taps a mix of disciplines and backgrounds, Hardy says. “On the tech side, think data scientists, AI engineers, and machine learning gurus — the people who can wrangle the data, build the models, and actually get these AI algorithms up and running.”
But it’s not just technical expertise that is important when it comes to staffing the COE, Hardy says.
“We also need business leaders and execs from the different departments that’ll be using AI — bringing the real-world know-how and making sure our AI projects actually solve business problems. And of course, we can’t forget the IT and cybersecurity crew who are crucial to making sure everything integrates smoothly and stays secure.”
Across the board, everyone on a successful COE team needs to be a good communicator, a team player, a solid problem solver, and someone who’s always up for learning new things, he explains. That’s what really drives innovation and gets AI adopted across the organization.
Job Roles Commonly Found in an AI COE
There are several specific job roles typically assigned to an AI COE, Torzynski says. They include:
- Data scientists, with a background in data science, machine learning, and statistics
- Software engineers, with a background in software engineering, computer science, and programming languages
- Business analysts, with a background in business analysis, operations research, and management science
- Subject matter experts, with a strong understanding of the AI knowledge domain and its applications
- Project managers, with a background in project management, agile methodologies, and scrum
Certain technology and business skills should also be included in the makeup of any AI COE, though not every member must possess them all, Torzynski explains.
Essential skills in the team include a strong understanding of the AI knowledge domain and its applications; a solid foundation in programming languages, data structures, and software development methodologies; the ability to analyze complex problems, identify patterns, and make data-driven decisions; the ability to communicate effectively with both technical and non-technical stakeholders; and the ability to work collaboratively with cross-functional teams and stakeholders.
“The COE’s team composition requires talent density in full-stack AI (machine learning, generative AI, deep learning and systems development life-cycle experts), domain fluency, and a flair for entrepreneurial mindset,” says Adnan Masood, chief AI architect at UST, a provider of digital technology and IT transformation services based in Aliso Viejo, California.
“We hire data strategists who appreciate how liquidity risk or M&A synergies intersect with quantitative modeling. We recruit engineers who can pivot to mission requirements at scale. We rely on AI-savvy project managers who spur iterative prototyping and keep strategic bet decisions on track.”
As to personal traits that will serve an AI COE well, Torzynski cites the following: a passion for learning and staying up-to-date with the latest AI trends and technologies; eagerness to work with cutting-edge technology and willingness to experiment and innovate; an ability to communicate effectively with both technical and non-technical stakeholders; adaptability to changing requirements and priorities; and the ability to think creatively and come up with innovative solutions to complex problems.
Qualities and Capabilities Wanted in AI OCE Team Leaders
Ideal leaders for AI COEs should have significant leadership capital and a vision-to-execution mindset, Masood explains. These individuals are often a chief AI officer or data-centric executive who practices radical candor and spurs creativity.
Leadership stamina is non-negotiable, since the COE’s arc extends from short-term projects to broad-stroke transformation blueprints, Masood says. They manage change while forging institutional legitimacy around data-driven decision-making.
Further, ideal leaders for an AI COE are visionary individuals with a strong understanding of both AI technologies and business strategy, Hardy says. They should typically have a proven track record of leading successful AI projects and building cross-functional teams. As such, they need to be influential and collaborative, capable of fostering a culture of innovation and knowledge sharing.
Also: “It goes without saying, but I’ll go ahead and say it anyway to be clear: Leaders should be adaptable and resilient, given the rapidly evolving nature of the AI field, and possess strong ethical considerations regarding AI implementation,” he explains.
By getting the team makeup and leadership right, one of the most significant benefits of an AI COE is that it can help create a company culture of creativity and innovation, Torzynski says.
“By bringing together experts from different departments and providing them with the resources and support they need, COEs can foster a culture of collaboration, experimentation, and risk-taking. This can lead to the development of new and innovative products, services, and processes that can help the company stay ahead of the competition.”
By following these lessons and adapting them to their own organization’s needs, companies can create a successful AI COE that drives innovation and growth, Torzynski says.
