Digital forensics and the big data deluge — some concerns based on Ramsey Theory



Citation information

M. S. Olivier. “Digital forensics and the big data deluge — some concerns based on Ramsey Theory”. In: Sixteenth IFIP WG 11.9 International Conference on Digital Forensics. Presented; to be published as book chapter. New Delhi, India, Jan. 2020


Constructions of science that slowly change over time are deemed to be the basis of the reliability with which scientific knowledge is regarded. A potential paradigm shift based on big data is looming — many researchers believe that massive volumes of data have enough substance to capture knowledge without the theories needed in earlier epochs. Patterns in big data are deemed to be sufficient to make predictions about the future, as well as about the past as a form of understanding. This chapter uses an argument developed by Calude and Longo [6] to critically examine the belief system of the proponents of data-driven knowledge, especially as it applies to digital forensic science.

From Ramsey theory it follows that, if data is large enough, knowledge is imbued in the domain represented by the data purely based on the size of the data. The chapter concludes that it is generally impossible to distinguish between true domain knowledge and knowledge inferred from spurious patterns that must exist purely as a function of data size. In addition, what is deemed a significant pattern may be refuted by a pattern that has yet to be found. Hence, evidence based on patterns found in big data is tenuous at best. Digital forensics should therefore proceed with caution if it wants to embrace big data and the paradigms that evolve from and around big data.

Full text

A pre- or postprint of the publication is available at

BibTeX reference

author={Martin S Olivier},
title={Digital forensics and the big data deluge --- some concerns based on {Ramsey} Theory},
booktitle={Sixteenth IFIP WG 11.9 International Conference on Digital Forensics},
address={New Delhi, India},
note={Accepted for presentation} )

[Publications] [Home]
Page maintained by Martin Olivier
Record refreshed: June 30, 2020

Beta version of new bibliography database; please report errors (or copyright violations) that may have slipped in.