Evolution of big data at CERN - a short overview by Dr. Tim Smith
There is a mind-boggling amount of data floating around our society. Physicists at CERN have been pondering how to store and share their ever more massive data for decades – stimulating globalization of the internet along the way, whilst ‘solving’ their big data problem. Tim Smith plots CERN’s involvement with big data from fifty years ago to today.
Dr. Tim Smith works in the CERN IT Department leading a group that provides services for the 10,000 strong CERN user community covering the domains of Audio Visual, Conferencing, Document Management and ePublishing. This includes a team which develops, installs and maintains instances of Invenio, the CERN Open Source Digital Library system which powers the CERN Document Service.
Here’s a quotable extract from his talk:
If big data has been around for so long, why do we suddenly keep hearing about it now? Well as the old metaphor explains, “The whole is greater than the sum of its parts” – and it is no longer just Science that is exploiting this.
The fact that we can derive more knowledge by joining related information together and spotting correlations can inform and enrich numerous aspects of every day life…
- in realtime such as traffic of financial conditions,
- in short term evolution, such as medical or meteorological ,
- in predictive situations such as business, crime or disease trends.
Virtually every field is turning to gathering big data…
- with mobile sensor networks spanning the globe
- cameras on the ground and in the air
- archives storing information published on the web
- and loggers capturing the activities of internet citizens the world over
The challenge is on to invent new tools and techniques to mine these vast stores, to inform decision making, to improve medical diagnosis, and otherwise to answer needs and desires or tomorrow’s society in ways that are unimagined today.
