Knowledge managers bring collectivity, nostalgia, and selectivity to AI

This article is part of an ongoing series looking at artificial intelligence (AI) in relation to knowledge management (KM), and KM in relation to AI.

By Katrina Pugh, PhD., Jonathan Ralton, Andrew Trickett, Marc Solomon, and Eve Porter-Zuckerman.

Artificial intelligence (AI), such as large language models (LLMs) and other neural networks, brings stunningly fast answers by accessing large amounts of the world’s knowledge. AI produces answers with seemingly human-like awareness, and has the potential to compress cycle time while elevating new learners and sage experts, alike¹. However, the use of AI in knowledge work poses risks: inaccuracy², manipulation³, compromised⁴ learning⁵, and social isolation⁶.

In an article⁷ authored by the SIKM Boston community members, Katrina Pugh, PhD, Jonathan Ralton, Andrew Trickett, Marc Solomon, and Eve Porter-Zuckerman, entitled “Knowledge Managers Bring Collectivity, Nostalgia, and Selectivity to AI,” we explore how knowledge managers (KM’ers) can intervene successfully with collective sense-making, example-selection, and a creative AI discernment. We call those three moves “collectivity,” “nostalgia,” and “selectivity.”

KM’ers uniquely differentiated to tame AI

Many disciplinary communities have looked at AI and asked, “How are we uniquely suited to respond to this?” For the knowledge management community, our differentiation has always been collective sense-making, content synthesis, and timely abstraction. This is because knowledge leaders stand with feet in two worlds: Information (the rows and columns of content and data) and collaboration (the circles of ideas shared within and between us).

A 15-year-old community of knowledge managers, the SIKM Boston professional peer group has about 30 active members. In 2024 SIKM Boston set out to work on articulating how we could proactively help our communities judiciously embrace AI by harnessing our skills in information curation and collaboration. In our article,  we present three case studies of knowledge managers wielding AI:

1. a global architecture, engineering and construction services firm,
2. a tech consultancy, and
3. a large multi-line insurer.

These cases illustrate three important ways that knowledge managers scrutinize the product of AI, inform the training of AI, and marry AI’s scale and speed with our human ingenuity. These cases show how knowledge management combats many of AI’s risks by using what we dubbed “collectivity,” “nostalgia,” and “selectivity”:

1. Collectivity: Delegation to AI can compromise our perseverance, accuracy, and professional relations⁸. Yet, knowledge-holder communities can vet AI’s output against their experience and best practices. In doing so, they build trust in AI and expand their collective reasoning capacity, “[H]uman conversation weaves experiences into a sort of lattice that contains unique perspectives and shared context.”

Our Case #1 of a global architecture, engineering, and construction services firm describes how, led by a SIKM Boston member, a community of practice provided gold-standard best practices to train the AI, and then convened regularly to vet the AI outputs. The community also is coming together to create a standard collection of AI prompts and prompt engineering methods.

2. Nostalgia: For large language models (LLMs), training data may reflect only a myopic view of a customer, technical domain, or problem space. However, when the LLM is trained on a continuous basis with vetted “exemplars” (artifacts selected for specific parameters), results can be more accurate, comprehensible, and expansive.

Our Case #2 describes how a SIKM Boston member at a technology integrator showed a blueprint for training the AI continuously on historical exemplars that have notable strengths, but that would have been overlooked by AI due to their age or other unrelated features. The team designed a process that would train AI on these exemplars, and use AI to grade current work products. This would shift knowledge-holders’ time from unscalable, reactive transactional solutioning, to proactive optimizing of the mechanized issue capture by defining a spectrum from Grade A to F. Their capacity can move to more valuable problem-solving, diagnosing, and advising.

3. Selectivity: The human-AI team is at its best when we invite the AI to find constructs we didn’t think of ourselves, but we humans must pay attention to how the result is relevant to our problem at hand. For example, when MIT’s Center for Constructive Communication used AI’s language extrapolation to pull out shared themes across community discussions, it surfaced “homelessness”⁹. Yet, humans had to translate that term “homelessness” into the more universal term, “housing insecurity.”

In Case #3, our SIKM Boston member at a multi-line insurance company used a novel AI approach to do benchmarking. He used AI to first pull back peer organizations’ unique ESG reporting approaches, calculating each one of their year-over-year commitment-fulfillment scores. But these reports were only meaningful to his organization when he used a shared taxonomy, or “master data,” to translate or normalize the peers’ categories. By doing this unique-to-normalized sequence, he could both judge the peers’ commitment-fulfillment and benchmark his own organization.

At the end of our study, we generated a rubric for divvying up AI’s and knowledge managers’ roles, illustrated in the table below:

Table: Balancing use of AI with the value of knowledge managers/KM teams

Skills of KM’ers are ever more important today

A reader might think that we’re disparaging AI, as it may dampen creativity, noting that AI “…tend(s) to short-circuit our self-expression and devalue or preempt human-human interaction.” On the contrary, we find that knowledge managers like those in the SIKM Boston community can successfully craft a partnership with AI. AI cannot substitute for our collectivity, nostalgia, and selectivity, but it can bring scale, reach, and speed. The onus is on us to continue to invest in AI and ourselves working together.

About the authors

Katrina (Kate) Pugh, PhD is a consultant, researcher, and educator on collaboration and sustainability. Since 2011 Kate has taught at Columbia University’s Information and Knowledge Strategy’s MS Program. As President of AlignConsulting, she helps build purposeful, productive conversation capacity among teams and networks, and has used Generative AI and data science to quantify the impact of conversation on sustainability outcomes. AlignConsulting clients range from large pharmaceuticals, to international development leaders, to nonprofits, to high tech innovators. She held executive KM roles with Fidelity, Intel, and JPMorganChase. In 2009 Kate co-founded the SIKM Boston community of practice that is mentioned in this article. Kate earned a PhD from UMaine (Ecology and Environmental Science), SM/MBA from MIT, and a BA in Economics from Williams College.

Throughout an IT journey of 22+ years, Jonathan Ralton crafts quality frameworks and mature, KM-based continuous improvement processes. A certified technical and change leader, he engages with stakeholders to overcome nuanced content and KM hurdles. Across industries from healthcare to manufacturing, and applications from IoT to intranets, he applies agile methodologies, information architecture principles, and a product strategy approach to shepherd executives and lead implementation teams toward value-based outcomes. Leveraging consultative methodologies, he’s successfully addressed complex business problems while exploiting emergent opportunities. Augmenting a well-developed technical acumen, Jonathan also possesses a flair for the creative and passion for good UX. A Boston native, he earned his BS in Information Technology from Northeastern University, graduating Summa Cum Laude.

Andrew Trickett is the ex-Arup Global Rail Knowledge Manager and is a highly experienced knowledge management practitioner with experience practically and academically over a 20-year period. Andrew is recognized globally across Arup and in the KM world as a subject matter expert, especially around capturing lessons learnt on projects as well as Communities of Practice (CoPs) as a driver of innovation within an organization. He has considerable experience in developing improved knowledge sharing within organizations based on people, process, and technology circles to improve connectivity and innovation. Andrew earned an MBA from Aston University Birmingham, United Kingdom.

Marc Solomon is a ESG Reporting Automation Manager at a large U.S. insurer. In 2019 he authored Searching Out Loud, a digital information literacy textbook for journalists and legal professionals. He has also taught in Boston University’s Professional Investigation program. Marc is a graduate of Hampshire College and George Washington University’s Master’s in Political Management program.

Eve Porter-Zuckerman is a consultant on knowledge management and strategic talent and technology initiatives for individuals and nonprofit and for-profit organizations. She was Chief Knowledge Officer for a premier executive recruiting firm serving mission-driven organizations, where for over 20 years she supported the firm’s growth and success, guiding KM programs and the development of the firm’s bespoke customer relationship management (CRM) system. As a founder, board member, and staff person, Eve has been involved in building and guiding nonprofits, helping them hone their missions and strategies, guide recruitment, and build sustainable structures. Eve currently serves on the board of her rural local library. She earned a BA from Yale University in history, and a Master’s degree from the Graduate Institute in Geneva, Switzerland.

Header image source: Created by Jonathan Ralton with ChatGPT.

References: