Proposed knowledge management metrics for public sector organizations

A paper¹ presented at the 20th Annual Conference of the International Research Society for Public Management has proposed knowledge management (KM) metrics for public sector organizations (PSOs), based on a systematic literature review.

The paper authors advise that as KM initiatives are seen as investment decisions, their performance outcomes must be evaluated and measured. The measurement of KM is considered essential in ensuring that KM objectives are tuned to organizational strategy, and metrics should show the areas where there are improvement opportunities. Performance measurement systems should be implemented throughout organizations, and not just limited to measuring the knowledge, expertise, and performance of individual employees.

Systematic literature review

The systematic literature review was performed using established guidelines and procedures. Firstly, a research protocol was developed, with inclusion and exclusion criteria, research questions, and quality attributes. A search of academic databases was then conducted, and primary studies for the systematic literature review chosen using the research protocol. The initial database search located 3289 studies, and after review, 17 of these were found to meet the criteria for the systematic literature review.

Inclusion (I) and exclusion (E) criteria:

I1: Contains keywords: knowledge management, public administration, public sector, e-government, measurement, performance evaluation, metrics.
I2: Period of publication is from 2005 to 2015.
E1: The document is in a language other than English, Spanish, and Portuguese.
E2: The entire document is no longer available.

Research questions (RQs):

RQ1: What are the main approaches or focus of KM measurement?
RQ2: What are main metrics, criteria of measurement proposed by literature?
RQ3: What are the levels of measurement proposed or implemented?
RQ4: What seems to be the benefits related with KM measurement in PSO?

Quality attributes (QAs):

QA1: The study is carried out in PSO.
QA2: There is empirical validation or evidence.
QA3: The study method is quantitative or qualitative.

Proposed KM metrics based on the systematic literature review

KM in PSOs is different to that in the private sector, so the paper authors have “defined metrics that could demonstrate KM initiatives value, help to continuously improve KM process, increase costumer’s satisfaction or citizen’s perception, and … provide sustainable long term competitive advantage for PSO.” KM in PSOs is not measured from a market value perspectve, as it is in the private sector, but from the perspective of knowledge quality and quantity, KM processes, KM initiatives, KM outcomes, and potential or demonstrated benefits.

The proposed metrics are listed below, organized by levels of analysis.

Level: Knowledge

Metrics:

• Understandability, clarity, unambiguity, conciseness, retrievability, traceability, consistency. correctness, completeness, reusability, credibility, and use of knowledge.
• Use of KM systems.
• Number of frequent KMS users.
• Amount of knowledge stored in traditional/manual filing systems and computers.
• Quality of knowledge stored.
• Number of intellectual properties owned.

Level: KM initiatives (also, KM process)

Metrics:

• Increase in the quality of KM process outcomes.
• Increase in the capability of knowledge acquisition, creation, generation, application and utilization.
• Increase on contributions to organizational repositories.
• Number of times employees attend training/seminars/courses to acquire knowledge.
• Number of times employees acquire knowledge from the owner-manager.
• Number of times employees contact customers/suppliers to acquire knowledge.
• Amount of time spent browsing the Internet/World Wide Web to acquire knowledge.
• Number of times employees access the company’s knowledge repositories to acquire knowledge.
• Number of times employees work in teams to create new knowledge.
• Number of times employees participate in brainstorming sessions to create new knowledge.
• Amount of time spent codifying and storing knowledge in the company’s knowledge repositories.
• Amount of time spent updating the company’s knowledge repositories.
• Employees’ level of willingness to contribute to the company’s knowledge repositories.
• Number of times employees participate in informal discussion to share knowledge. Frequency of having meeting sessions.
• Frequency of employees using technological tools (e-mail, etc.) to transfer knowledge.
• Investment in basic ICT (computer, Internet, intranet, etc.).
• Investment in organizational infrastructure (meeting room, filing rack, etc.).
• Frequency of organizational infrastructure maintenance.
• Degree of alignment between KM strategy and business strategy.
• Clarity of the company’s KM strategy.
• Employees’ degree of awareness and support toward the company’s KM strategy.
• Amount of budget allocated for KM initiatives.
• Number of employees involved in KM initiatives.
• Amount of time allocated for employees to perform KM activities.
• Number of professional development activities organized for employees.

Level: KM outcomes or KM results

Metrics:

• New knowledge, ideas, and solutions created per employee.
• New products, inventions, and services generated.
• Math model, research output, scientific paper.
• Documents and articles accessed or downloaded per employee.
• Documents and articles uploaded or updated per employee.
• Active communities of practice, research groups, and special interest groups.
• Communications per employee per month.
• Problems solved and ideas implemented per employee.
• Knowledge assets generated per year.
• Acceleration of learning times.
• Increase in sharing experiences.
• Decrease of capability gap between staff skills and job requirements.
• Increase of search cycle efficiency or time taken to find knowledge necessary to perform a new task.
• Improvement of stakeholders perception of the value of the organisation.
• Increase of internal reward and recognition.
• Improvement of staff morale and productivity.
• Improvement of employee capabilities.
• Efficient use of an allocated budget.
• Quantity of products or services provided.
• Increase of customer satisfaction/citizens perception.
• Rewarding employees under his/her supervision.
• Increase of perceptions of internal communication.
• Revenue generated from intellectual properties.
• Number of new knowledge, ideas and solutions created.
• Amount of rewards given to employees who create new knowledge, ideas and solutions.
• Number of times employees apply useful proposals/ideas in practice.
• Number of times employees apply knowledge to solve problems.
• Number of new products/services launched.

What do you think?

The paper authors state that they have put forward the proposed KM metrics in the interests of advancing KM measurement in PSOs, and that they welcome academic debate in regard to the metrics. What do you think? Practical aspects should also be considered: are the proposed metrics straightforward to measure? Add your comments below.

Reference:

Also published on Medium.