Many companies, whether big or small have vast amount of data stored in traditional databases in-house or in public and private clouds, often without much knowledge around how much value this data has for:
- increasing operational efficiencies
- adding relevance to marketing towards more targeted audiences
- understanding customers better in order to develop products and services more aligned with their demands and needs and hence improving relationships
- gaining a competitive edge by better understanding trends and predicting market demands whilst preparing for their next strategic moves
CIOs already have a hard time storing and managing data in a way that is cost efficient, secure and accessible whenever required, as the scale of information to be stored has exponentially increased over the past years.
While public and private clouds have become an effective alternative to store “cold” data, companies need to ensure that both structured and unstructured data can be classified in terms of when and where it has left the network premises.
Against a backdrop of vast amounts of data, many companies are still faced with the challenge of being able to query the data as well as selecting big data hardware and software that will deliver insightful information.
Many CIOs are currently anxious about moving data to the cloud due to the costs associated, the challenges of choosing adequate partners, questions over security and reliable access to this data.
However, there are a number of vendors that offer approaches to helping companies take the first steps.
DELL amongst others are shifting towards tiered storage and flash technologies, closer to the processing units (servers), analysing what data should be stored where so it can be efficiently utilised and in a rapid manner in big data applications.
HP, for example, is also working closely with its customers, by supporting them in their projects with tools and technology, resources and software to test, evaluate and validate big data uses and scenarios.
There are also a number of companies, specialised in big data and dedicated to certain industries and applications, with tools and data scientists capable of taking on data, analysing it, querying it and according to requirements providing insights into that data.
However for many CIOs these companies are still to provide a blue print on how the process can be streamlined to their own applications in a costs effective way, besides other projects that may be more critical to them.
Big data management remains one of the technologies where one size does not fit all and is still very much a ‘personal’ case for the CIO and industry consultant who can guide their companies through a new area of aggregation, syndication and adoption of best practices.