Data variety and why it matters

By Richard Berry. Originally published on the Integration and Implementation Insights blog.

What are the differing characteristics of data? Why are they important for systems to function effectively? What is requisite variety of data?

There are nine characteristics of data variety which agitate systems. These are volume, velocity, variety, veracity, validity, vulnerability, viscosity, vectors and virtualisation. Together, the ‘9Vs’ constitute a data requisite variety framework and are described below.

1. Volume
   Description: The amounts of available data.
   Example: Volume can vary widely from the results of small-scale research to the tsunami of digital material accessible through the internet. The latter can overwhelm both people and organisations.
2. Velocity
   Description: How quickly data move across a network.
   Example: Mobile, fixed line and satellite networks can operate at vastly differing directional speeds. These determine the informational capabilities that people and organisations can develop, access and use.
3. Variety
   Description: The different types of data formats.
   Example: There are numerous protocols for packaging and moving data. For example, internet protocol version four (IPv4) has a 32-bit address whereas, version six (IPv6) has a 128-bit address and provides far more capacity.
4. Veracity
   Description: The truthfulness of data, ie., honest intent.
   Example: Data can be created and provided with the intention to deceive recipients. For example, specific spoofing apps can falsify telephone numbers and device locations. Social media ‘bots’ can create deliberately manipulative content.
5. Validity
   Description: The extent that data are accurate representations of reality and the number of errors. Validity necessitates tracing data to a source and being able to explain the processing journeys.
   Example: Facial recognition systems vary in accuracy due to data capture factors like position, angle, poor image quality, etc. Artificial Intelligence can produce predictive results. However, accounting for accurate processing can be technically problematic.
6. Vulnerability
   Description: The extent to which data creates vulnerabilities for organisations and people.
   Example: Private and public data can increase risks, for example, commercially available location data can reveal personal information about routines and lifestyles.
7. Viscosity
   Description: The solidity of data, it may run away, disappear or remain static for long periods.
   Example: Some data can be overwritten easily, such as telematics that track the opening and closing of car doors. On the other hand, formal records can be retained by authorities for many years.
8. Vectors
   Description: The routes and travel of data.
   Example: A phone call can be on the cellular network or through a Wi-Fi system.
9. Virtualisation
   Description: The global location of data, where it is stored and curated in accordance with local legislation.
   Example: Differing laws exist for privacy and access. Therefore, online services can provide options for ‘data residency’ to ensure users can have choices about storage locations.

Ashby’s Law and the Importance of Requisite Variety

The cybernetic principle of requisite variety is often referred to as Ashby’s Law (Ashby, 2015). We can think of it in these terms:

To remain stable a system needs enough variety to match the variety within the demands placed upon it.

The nature of these demands can be assessed using the 9Vs found in a data requisite variety framework. Such analysis can inform how a system might adapt and therefore stabilise in accordance with Ashby’s Law.

We can learn from events where requisite variety was not achieved:

• In 2019, the UK police inspectorate reported that over 25,000 mobile devices were awaiting forensic examination. There was no plan or capability to deal with the volume and complexity of the work required. There was insufficient requisite variety to meet demand, the system became unstable and government intervention followed.
• Concurrently, Denmark found that over 10,000 criminal cases were at risk because of fallacious geolocation data from mobile networks. Soon after the software error was discovered, over thirty prisoners were released due to concerns about unsafe convictions and another forty cases were postponed. In terms of the data requisite variety framework the validity, variety and vulnerabilities of these data could not be absorbed by the Danish justice system.
• In 2024, the UK formally recognised its largest miscarriage of justice when false data from the Post Office Horizon accounting system were used to prosecute over 900 innocent postal staff for theft and fraud offences. In terms of the data requisite variety framework these data lacked validity and were unreliably presented as evidence of true facts, ie., veracity. Software glitches were blamed. The data vulnerabilities have resulted in criminal investigation of those involved in compiling the original cases. The Post Office and criminal justice system remain in oscillation, as efforts are made to develop stability.

Requisite Variety and Future Demands

We can expect emergent technologies like autonomous cars and more sophisticated drones to propagate new data varieties. For example, data viscosity could reflect the potentially large volumes of sensor data which will be processed by such devices. Some of these data will disappear, some will be stored locally within the devices and other data will be stored in virtualised applications. The vectors of these data could be multiple from global positioning satellites to the use of localised mobile networks. New data variety often requires systems to adapt. A data requisite variety framework can inform how to promote stability in accordance with the principles of Ashby’s Law.

Do you have other examples to share of different data variety characteristics? Are you aware of other systems which have been disrupted by a lack of data variety? How might the data requisite variety framework apply to your research area? How can our understanding of data variety be improved?

Reference:

Ashby, W. R. (2015). An Introduction to Cybernetics (4th ed.). Martino Publishing: Mansfield Centre, Connecticut, United States of America.

Use of Generative Artificial Intelligence (AI) Statement: Generative artificial intelligence was not used in the development of this i2Insights contribution. (For i2Insights policy on generative artificial intelligence please see https://i2insights.org/contributing-to-i2insights/guidelines-for-authors/#artificial-intelligence.)

Biography:

Richard Berry PhD is a Fellow of the Cybernetics Society and a former police officer now based at the Centre for Information Management, Loughborough University, UK. His research interests are security cybernetics, strategy and capability leadership within complex adaptive systems.

Article source: Data variety and why it matters. Republished by permission.

Header image source: Markus Spiske on Unsplash.