Browser unable to execute script; please use the site map to navigate the site.

Digital forensics and the big data deluge — some concerns based on Ramsey Theory



(Citation)Citation information

M. S. Olivier. “Digital forensics and the big data deluge — some concerns based on Ramsey Theory”. In: Advances in Digital Forensics XVI. Ed. by G Peterson and S Shenoi. Advances in Information and Communication Technology. Springer, 2020, pp. 3–23


Constructions of science that slowly change over time are deemed to be the basis of the reliability with which scientific knowledge is regarded. A potential paradigm shift based on big data is looming — many researchers believe that massive volumes of data have enough substance to capture knowledge without the theories needed in earlier epochs. Patterns in big data are deemed to be sufficient to make predictions about the future, as well as about the past as a form of understanding. This chapter uses an argument developed by Calude and Longo [6] to critically examine the belief system of the proponents of data-driven knowledge, especially as it applies to digital forensic science.

From Ramsey theory it follows that, if data is large enough, knowledge is imbued in the domain represented by the data purely based on the size of the data. The chapter concludes that it is generally impossible to distinguish between true domain knowledge and knowledge inferred from spurious patterns that must exist purely as a function of data size. In addition, what is deemed a significant pattern may be refuted by a pattern that has yet to be found. Hence, evidence based on patterns found in big data is tenuous at best. Digital forensics should therefore proceed with caution if it wants to embrace big data and the paradigms that evolve from and around big data.

(Full text)Full text

A pre- or postprint of the publication is available at

(Authoritative version on publisher's site)Definitive version

The definitive version of the paper is available from the publisher.
DOI: 10.1007/978-3-030-56223-6_1

(BibTeX record)BibTeX reference

author={Martin S Olivier},
title={Digital forensics and the big data deluge --- some concerns based on {Ramsey} Theory},
editor={G Peterson and S Shenoi},
booktitle={Advances in Digital Forensics {XVI}},
series={Advances in Information and Communication Technology}, vol={569},
publisher={Springer} )