BIG data" is increasingly becoming one of the most closely watched technology trends and many companies are already acquiring the required resources to process large amounts of data and content created internally and externally.

"Big data" is a phrase that refers to the software tools, processes and procedures that allow companies to create and manage very large data sets that conventional database systems cannot handle.

Companies use big-data technologies for insights into new and emerging types of data and content and related trends, to make their businesses more agile.

For example, a company may use big-data-related software to analyse big data in order to monitor customer perceptions and sentiments about its brand on social media websites.

The increasing volume and detail of information, the rise of multimedia, social media, and the rise in internet use and access is fuelling exponential growth in data and making the rapid growth of data more challenging to manage, according to analysts.

According to IBM, every day 2,5-quintillion bytes of data are created. This data comes from everywhere, such as sensors used to gather climate information, postings to social media sites, digital pictures and videos, purchase transaction records, cellphones and satellite navigation systems, to name a few, says IBM.

Companies are spending billions of rands on software related to data management and analytics. For example, the UK telecommunications group BT has a big-data product called "Assure Analytics" which it uses to protect against copper cable theft. The service is helping the company analyse crime statistics, fault reporting and geographical information - highlighting patterns and theft hotspots to inform its prevention and response policies. It also has the potential to be connected into the company's systems that gives BT and the police an early warning of cable thefts in progress.

Global research firm International Data Corporation (IDC) has predicted that big data will join the likes of mobile and the cloud this year as the next "must have" competency, as the volume of digital content continues to grow.

IDC predicts that the market for big-data technology and services will reach $16,9bn by 2015, which is an increase from 2010's figure of $3,2bn and represents a 40% year-on-year growth rate.

Martin Rennhackkamp, the chief operating officer of South African technology firm PBT Group, says big data, which includes both structured and unstructured data, internal and external to the organisation, will continue to grow in importance due to "intrinsic value that data holds for any company".

"Enterprises are dealing with data that is increasing exponentially in both size and complexity," says Gary Carroll, general business director at IBM.

Manoj Bhoola, country manager for Avanade SA, a technology firm, says big data has become everyone's business and it has gained a top spot on the agenda of business leaders for the value it has begun to create. "The challenges of big data remain, but the opportunities are even greater. Business leaders are really moving from defence to offence in their data management strategies. Forward-looking companies are empowering more people across the enterprise with the tools and skills needed to make better business decisions and ultimately, harness the power that big data promises," Mr Bhoola says.

A study by Avanade shows that big data has become "pervasive" as more types of employees have more access to more technology options to manage and analyse data.

As big data is all about volumes of information, security becomes one key factor that businesses using big data must consider, says Alexander Erofeev, chief marketing officer at software security group Kaspersky Lab.

"Certainly it is critically important for businesses to look at technologies such as real-time business intelligence software to manage this data.

"However what businesses also need to consider, which in our opinion is also lacking, is the security aspect and potential risk that big data can present to an organisation, especially if the data is not managed correctly," he says.

Cybercriminals are shifting from the consumer market to businesses as they are fast realising the value that business data holds, and are exploiting this for their own financial gain, Mr Erofeev says.

"Data is a business asset, but an asset that needs the right protection if it is to provide the benefits an organisation wants. Compromised data can destroy business servers and end up causing more damage than good," he says.

Wikipedia describes big data as "a loosely-defined term used to describe data sets so large and complex that they become awkward to work with using on-hand database management tools".

mochikot@bdfm.co.za