Header Big Data

Big data – The raw material of the 21st century

Every day, a worldwide 2.5 trillion bytes of data are generated and the “Digital Universe” study by EMC predicts data volumes of 44 zettabytes by 2020, meaning the Internet is currently growing at some 70 terabytes per second. This digital universe, being fed worldwide by companies as well as private individuals using computers, mobile devices and electronic sensors of all kinds, has a name: Big data. The term now not only refers to the immense volumes of data being generated daily, but also to its very own application.

The significance of big data to companies lies in achieving competitive advantages and efficiency gains, as well as creating innovations generated from, for example, tweets, purchase transactions, weather sensors, fitness armbands or even the GPS signals from smartphones. Big data thus facilitates new business potentials and has revolutionised processes, organisations and entire industries over recent years.

What does the future of big data look like – and what opportunities and challenges lay ahead?

80% of worldwide data generated is still unstructured and 95% of it cannot be automatically evaluated. This shows clearly that big data has nothing in common with a classically structured databank. Instead, it consists of a gigantic mass of data where the search for interesting data and data linkage can be compared to the famous search for a needle in a haystack. Despite the enormous challenges presented by the exploitation of big data, ever more companies are investing in the analysis and application of the data. Worldwide today, over one third of companies are pursuing active big data projects. Big data potential can thus be exploited in all business sectors, whether it be marketing, sales, risk controlling or in production. Once implemented, big data projects quickly pay off: According to IBM, two thirds of all organisations reported in 2015 that big data projects performed had achieved corporate objectives or even exceeded them.


Objectives companies pursue with big data

Almost half of all these companies are pursuing consumer-orientated objectives with their investment in big data. (Source: Self-representation based on IBM Institute for Business & Saïd Business School at Oxford University, 2012)


The IBM study “Analytics: The real-world use of big data” cites that companies are primarily pursuing consumer-orientated objectives in their programs. Organisations want to improve upon customer experience and to better understand and predict customer preferences and behaviour. Big data provides the opportunity for this purpose – data from transactions, multi-channel interactions and social media have increased the ability of organisations to paint a complete picture of customer preferences and requirements. With this deeper understanding of customer behaviour, companies uncover new opportunities for interacting with existing and potential new customers. This applies to both business-to-business as well as business-to-consumer enterprises, such as the telecommunications and healthcare sectors, for example, or in banking and finance companies – but most especially of all in the area of retail.

Big data will become the decisive competitive advantage for retailers

More than ever, customers are relying upon new technologies for purchasing. When retailers utilise the incidental data created here, sales can then be increased and costs reduced. In the modern era, in which multi-channel strategies and digitally networked consumers have become the norm, data has thus become a crucial factor. The success of retailers is becoming increasingly dependent upon whether they can manage, integrate and understand these huge volumes of data created. In light of advancing technologies and the shift in the nature of data, ever more retailers are focusing on the potential of big data. 62% of retailers are expecting a competitive advantage from big data and 72% had already began planning a big data strategy, or the implementation of big data projects, by 2013.

To create added value from the new technologies available, fresh requirements for retailers are also arising. Included here are access to enormous and diverse volumes of data, as well as the essential analytical capability using appropriate software tools and the required know-how to utilise and interpret the results. Retailers can use insights into customer behaviour and the competitive market gained from big data, among other things, to optimise products and product ranges, or to adapt prices and improve customer satisfaction.

From big data to smart data

Big data is only of use to companies when useful, high-quality and secure data can be generated from it.“Only those who comprehend the data can create added value. To evaluate such volumes of data, the data itself must be understood…”, said Wolfgang Heuring, Head of Group Research at Siemens AG. An initiative from Trusted Cloud research has termed this data smart data and suggested the following equation:

Smart data = big data + benefits + semantics + data quality + security + data protection = useful, high-quality and secure data

In eCommerce in particular, high expectations are held for smart data in the twin objectives of winning repeat custom and increasing sales. Customer purchasing behaviour should become as accurately anticipated as possible so that product suggestions become even more precise. The open source company pentaho also mentions these highly individualised purchase recommendations in its Big Data Trends 2016, as well as the constantly growing “Internet of Things”.


In the Internet of Things, everything communicates with everything. By 2020, some 50 billion devices are to be connected to the Internet.

The Internet of Things is a major contributor to big data, since as ever more objects become interlinked and communicate together, then ever more data will be the end result. A good example is the earlier mentioned fitness armbands that send their generated health data to an app. The owner might thus be reminded about purchasing new running shoes and where the nearest shop happens to be. This data received, however, could potentially also be sent to the health insurance company, which in turn could decrease or increase contributions dependent upon how active the jogger actually is. The risk that personal data can be used disadvantageously against individuals is thus always present. And herein lies the major weakpoint of big data: The more data generated, the more transparent our lives become and the more transparent we as individuals become.

Big data in online retail – Use the flood of data profitably

Because of the valuable insights that data analyses can provide, companies today can no longer avoid the use of big data. There will be increasingly more applications flooding the market that facilitate usage of that generated data profitably. Modern-day analytical technologies assist retailers in extracting insights from data at a hitherto unattained degree of precision and speed. Above all, the use of company-external data holds great potential for cost optimisation and profit increase, whilst rapidly reimbursing the investments needed for its introduction. In blackbee, we have created an SaaS solution that collects together millions of individual product data daily from the Internet – both worldwide and irrespective of sector. Our technology then structures and processes this data so that retailers and brand-name manufacturers can apply the supplied information directly for market analyses. It is in this way that valuable product information can be gained from big data.

Would you like to learn more about how you as an online retailer can effectively use product information in the age of big data? You can discover more on this topic in our whitepaper “Product Matching Excellence”.