Troubling case highlights that the KM community lacks key capabilities needed for safe AI use

As can be seen in the long-running RealKM Magazine artificial intelligence (AI) series, research and practice in regard to the use of AI in knowledge management (KM), and KM in AI, continues to accelerate at a rapid rate.

However, an article by Adi Gaskell reports on research¹ supporting the need for the KM community to proceed with caution in regard to AI. The research looks at risks associated with using generative AI chatbots for content generation. The authors delve into what they refer to as “botshit,’ which they define as inaccurate or fabricated content produced by chatbots. They state that:

Our paper explains that when this jumble of truth and falsehood is used for work tasks, it can become botshit. For chatbots to be used reliably, it is important to recognize that their responses can best be thought of as provisional knowledge

To highlight the dangers of using botshit in decision-making, I then wrote the article “Two horror cases highlight the dangers of blind faith in what AI generates,” reporting on the Australian Robodebt AI and UK Post Office Horizon scandals which have damaged or destroyed the lives of many people.

The botshit paper authors alert to the need for checking, thinking about, and questioning AI outputs if such horrors are to be avoided. Reinforcing this, notable KM pioneer Tom Davenport advises² that critical thinking to be able to decide if outputs are correct is a key capability needed in using these new technologies.

Does the KM community have these capabilities? In this article, and as suggested by the image and disclaimer above, I document a further case which unfortunately shows that the answer to this question is at present a very strong “no.”

The “guest post” scam

Anyone who manages a website with a good ranking by search engines is very likely to find themselves bombarded with emails asking to publish “guest posts” on their site, or using similar euphemistic terms such as “content collaboration” as the awareness of the “guest post” scam grows.

When I first started receiving these “guest post” requests through the RealKM Magazine contact form and email address, I thought they were quite strange. The emails all had very similar wording, and made no reference at all to KM or related research, despite our purpose being research communication which is very clearly stated in our information for contributors. It also seemed odd that people would be randomly asking to publish such articles on our site when there are major platforms that are free to legitimate writers, for example LinkedIn Pulse, Medium, and Substack.

Some of the emails included links to example articles written by the sender. These articles were largely shallow and formulaic, with links to other similarly shallow articles rather than research references or other credible sources. Online searches using the names of the authors of the emails and articles also did not find any matching subject matter experts on LinkedIn or with their own professional websites, including not finding anyone active in any of the KM communities. This lack of demonstrated KM expertise or experience combined with the lack of references in the articles meant that anything meaningful that had been included in the articles was very likely plagiarized.

Many years of dealing with fake knowledge for sustainability meant I was already long used to treating all information sources as provisional, as recommended by the botshit paper authors (as quoted above). So, I researched the “guest post” situation.

I found a mix of advice. I found some articles on credible websites, such as the following one³ from 2014. These articles advised that guest posting had in the past been legitimate content collaboration, but by that time, had become a search engine optimization (SEO) scam:

The article references advice from Matt Cutts⁴, the former head of the web spam team at Google, who clearly warns⁵ that publishing these guest post scam articles would very likely result in search engine ranking penalties:

I also found other articles supporting guest posting, but the writing style looked suspiciously like the example guest post articles I had reviewed, and it would be unsurprising for guest post writers to be writing other guest posts aimed at legitimizing their activities.

Given both the low quality and SEO scamming nature of guest posts, my reaction was to leave them well alone, and to state on our contact page that we don’t welcome guest posts. However, despite this, the volume of guest post emails progressively increased to a deluge, and the senders of the emails have become much more pushy and even aggressive, often sending multiple emails. I became so fed up with having to sift through the constant stream of guest post emails that I removed the contact form from our contact page, and no longer use email as the contact method for article submissions.

The following linked image shows many of the guest post scam emails still received by RealKM Magazine in 2024, even after the removal of our contact form. It is only now after we closed our public email account that they have finally stopped.

An even darker turn

As well as an increasing deluge of guest post emails over the decade since the warnings above were issued, guest post content has also taken a much darker turn, and started to promote links to services that are inappropriate or even illegal, which means they can’t be advertised normally.

Unfortunately, another KM organization has become a victim of this type of “guest post” scam email. While many of the organization’s posts are legitimate, having been authored by KM experts, something that is promoted in a number of the guest posts on their website is so harmful that in Australia, where RealKM Cooperative is registered, it is a serious crime that is being accompanied by blackmail and threats of violence.

This is part of the reason for the disclaimer above and the shortness and redaction of the guest post excerpts below. Another reason for these measures is that it is not my desire to criticize the particular organization because their actions in publishing these guest posts are not intentional (in contrast to the clearly intentional acts of the people who wrote the posts and submitted them to the organization). Rather, the publishing actions of the KM organization are a symptom of a wider KM community problem.

The harmful guest posts published by the KM organization advertise academic cheating services. This is where students cheat by paying online writing services to write their essays, reports, and dissertations for them, subverting their learning. Such is the scale of this cheating that research found 1 in 10 Australian university students⁶ were using these services, with most not being found out.

This led the Australian Government to establish an Education Integrity Unit⁷ tasked with shutting down academic cheating services, and to introduce legislation⁸ making it an offence to provide or advertise academic cheating services.

As shown in the following excerpt from the legislation, the promotion of commercial academic cheating services is a criminal offence in Australia carrying a penalty of 2 years imprisonment or 500 penalty units, or both (at the time of writing this article, one penalty unit is AUD $313⁹, so 500 penalty units is a fine of AUD $156,500).

One of the guest posts published on the KM organization website includes the following text:

The article is about technical writing in KM, but the “TopEssayWriting” link does not go to a technical writing agency who can help develop a training program and create training materials. Rather, it goes to the website of a commercial academic cheating service – the type of service that is the target of the legislation referenced above – as shown in the following screenshot:

Another of the guest posts published on the KM organization website includes the following text:

As with the first example above, the “Essay Republic” link does not go to a business or technical writing agency. Rather, it goes to a website listing a number of different commercial academic cheating services. The biography of the author of this post also contains the link “GetGoodGrade” which from its name presumably also went to the website of yet another commercial academic cheating service, but is currently a dead link:

These article excerpts clearly show the intention of the writers to use their guest posts on the KM organization website for the purposes of promoting academic cheating services. They also clearly show that the guest post writers are the very same people who are writing the fake essays, dissertations, reports, and papers for the academic cheating services.

If you’re still in any doubt that this is what is happening, the six different links in the following three different biographies from one author on the KM organization website go to five different commercial academic cheating service websites:

Fake research put lives at risk

Just how bad is this? While the use and promotion of commercial academic cheating services is not yet a crime in other countries, it needs to be because it is endangering lives.

When they first appeared, commercial academic cheating services, also called “paper mills,” primarily targeted undergraduate university students. However, in the time since, they have increasingly also been writing for cheats at masters degree and PhD level, as the “dissertation writing service” and “buy dissertations” links in the above guest post excerpts show.

Now, frighteningly, commercial academic cheating services are also being engaged by dishonest researchers who are using them to write the papers they submit to academic journals. As the following commentary¹⁰ in the academic journal Nature warns, this problem is exploding:

This fake research is putting lives at serious risk. The following article¹¹ sounds a very loud alarm, warning that fake papers are “slowing down research that has helped millions of people with lifesaving medicine and therapies from cancer to COVID-19” and a researcher investigator stating that “The threat of paper mills to scientific publishing and integrity has no parallel over my 30-year scientific career”:

Organized crime

As referenced above, it has been more than a decade since Matt Cutts, the former head of the web spam team at Google, warned that “if you’re doing a lot of guest blogging then you’re hanging out with really bad company.”

The academic cheating services promoting themselves through their guest posts have now grown to where their operations are industrial in scale, just as has happened with other major scams¹². And, as with those other major scams, commercial academic cheating service providers are now much worse than just really bad company, having taken on the tactics of organized crime including blackmail, intimidation, and threats of violence.

A 2021 article¹³ reported that students who had used commercial academic cheating services were being targeted with blackmail and extortion, with the threat of their cheating being exposed if they don’t pay up. As the following 2024 article¹⁴ shows, this problem has continued to grow:

The tactics of commercial academic cheating service providers now extend to even making bomb threats¹⁵:

What the KM community needs to do

As I stated above, the KM organization’s publishing of guest posts promoting commercial academic cheating services has not been intentional (in contrast to the clearly intentional acts of the people who wrote the posts and submitted them to the organization). Rather, the publishing actions of the KM organization are a symptom of a wider KM community problem.

However, by not detecting disinformation when it should have, the KM organization has published posts promoting something that is seriously harmful. The human equivalent of botshit has clearly existed long before AI came along, and if the KM community lacks the critical thinking capabilities of checking, thinking about, and questioning needed to detect human disinformation, then it won’t be able to detect botshit either.

Four interrelated KM community actions can help to build these capabilities:

1. Closing the research-practice gap

In his significant 2021 review¹⁶ of scientometric research of the KM discipline, prominent KM researcher Alexander Serenko reports that the longstanding research-practice gap in KM is not only wide but growing. He alerts that:

At this stage, it is obvious that the direct knowledge dissemination channel – which assumes that practitioners directly access, read and benefit from academic publications – does not function.

Instead, KM practitioners are seeing opinion without academic or any other credible references – for example guest posts – as acceptable information sources.

The KM research-practice gap is observably smaller in the KM for sustainable development aspect of KM than the organizational KM aspect of KM. For example, 20 years ago, the Knowledge Management for Development (KM4Dev) global community of practice established its own associated academic journal, the Knowledge Management for Development Journal, and strong research-practice links are still very evident in KM4Dev’s current activities¹⁷.

In the organizational KM aspect of KM, the establishment of the KM Global Network (KMGN) research community is an excellent step in helping to close the research-practice gap. But the KM organization that has published the guest posts that are the subject of this article also comes from the organizational KM aspect of KM, showing that much more needs to be done.

In his review, Serenko advises that the solution to the widening research-practice gap in KM should include “the introduction of formal and informal positions of knowledge brokers who aggregate, summarize and deliver the academic knowledge … to busy practitioners in an easy-to-comprehend format.” This is the founding purpose of RealKM Magazine.

2. Taking dark side KM tactics and fake knowledge more seriously

The consequences of the wide and growing research-practice gap in KM include research related to dark side KM tactics and fake knowledge – topics that are directly relevant to the actions of the guest post scammers – not making its way into KM practice. This means that the KM community has so far not taken these issues seriously enough.

The directly relevant KM research includes Professor Emeritus Stephen Alter’s 2006 paper¹⁸ “Goals and Tactics on the Dark Side of Knowledge Management” and section 3.6 of John Edwards and Antti Lönnqvist’s landmark 2023 review¹⁹ of the past and future of KM, which addresses “fake knowledge.”

Nearly 20 years ago, Professor Alter warned that:

Although a great deal has been written about KM … the dark side of KM is largely ignored. Most discussions of KM take a utopian view that the goal is to capture essential knowledge and make it available wherever needed. Further, that knowledge will be collected and distributed accurately and with the best of intentions, leading to efficiency, better decisions, and protection of intellectual property. Not mentioned is the possibility that KM activities could be motivated by inappropriate goals or intentions to commit crimes.

Leadership in addressing this extremely dangerous knowledge gap in KM, including extending actions in regard to dark side KM to AI, is coming from the Knowledge for Development Partnership (K4DP). The newly published 4^th edition of the Agenda Knowledge for Development²⁰ includes the following sub-goals:

2.4 We build an alliance against the misuse or ignorance of evidence and abuse of knowledge by individuals and groups aiming to mislead with a harmful impact on the general public.

2.6 While acknowledging the power of AI in advancing sustainable development, extraordinary care must be taken of intentional misuse or tool-driven failures, like hallucination and other forms of misinformation.

3. Embracing evidence-based practice

An effective way that the KM community can help to close the wide and growing research-practice gap in KM is by embracing evidence-based practice.

The Center for Evidence-Based Management (CEBMa) advises²¹ that there are six steps in evidence-based practice:

Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available evidence from multiple sources by

1. Asking: translating a practical issue or problem into an answerable question
2. Acquiring: systematically searching for and retrieving the evidence
3. Appraising: critically judging the trustworthiness and relevance of the evidence
4. Aggregating: weighing and pulling together the evidence
5. Applying: incorporating the evidence into the decision-making process
6. Assessing: evaluating the outcome of the decision taken

to increase the likelihood of a favorable outcome.

For step 2, the CEBMa identifies four sources of evidence:

1. Scientific literature – empirical studies
2. Organisation – internal data
3. Stakeholders – values and concerns
4. Practitioners – professional expertise.

In evidence-based practice, guest post scam emails and posts will be very quickly rejected because, as discussed above, they don’t align with any of these sources.

4. Developing and implementing ethics frameworks

The KM community overall also lacks ethics frameworks. While the KM organization’s publishing of guest posts promoting commercial academic cheating services has not been intentional, the existence of a KM ethics framework emphasizing the seeking and communication of the truth may have prompted them to more thoroughly scrutinize the guest post submissions they have been receiving.

For example, the Project Management Institute (PMI) Code of Ethics and Professional Conduct²² includes items related to the communication of truthful information:

The Global Think-Tank of Organizational Tacit Knowledge Management (GO-TKM) has become the first KM body to progress the development of an ethics framework, and all major KM bodies and networks should do similar.

Note: Because of legal and other risks, comments on this article have been disabled, and social media comments will be actively moderated.

Header image source: pxfuel, Public Domain.

References: