A recent article in the UK Computing journal suggests that the new frontier in industry is extracting insights from big data. While a decade or more ago data ingestion was the greatest challenge to companies collecting large swathes of data, that problem has now largely been overcome.

Of 300 IT professionals interviewed, only 13% saw raw data access as the biggest challenge in their work. The majority were split between transforming raw data into useful data and extracting actionable insights from the useful data.

Database platforms such as SQL and Hadoop and other tools have largely standardised the warehousing and accessing of data across big data consumers. However, insight extraction is the next frontier and there is still no runaway leader or dominant player in this space. It is difficult to build a one size fits all solution to the problems a traditional business analyst might work on.

But it is likely we will see more and more contenders coming to the fore in this potentially lucrative space. It will be an exciting arms race to see play out in the data science world!