Big Data in 2016: Hadoop

Posted on

Ken Hess of ZDNet reports on the results of a recent big data trends survey that focused on Hadoop use. The survey asked 250 data architects, IT managers, developers, BI and data analysts and data scientists on what they expect in 2016.

The overall theme of the survey results writes Mr. Hess is, “the move away from Hadoop experimentation into full production with big data analytics.”  The big “three” trends identified in his article were:

  • Apache Spark production deployments
  • Conversion from other platforms to Hadoop
  • Leveraging Hadoop for advanced use cases

70% of respondents were interested in Apache Spark. Mr. Hess explains, “The two primary factors in this interest in Spark is that it is easy to deploy and its speed.” The conversion from older platforms to Hadoop is also interesting. When asked what they expect to gain from switching to Hadoop the respondents cited the following:

  • 63% expect Hadoop to increase business and IT agility
  • 55% expect to improve operations and reduce costs
  • 51% expect it to allow them to give business users more access

Also, Mr. Hess reports, “More than half the respondents view Hadoop as a way to innovate by using social media data and data from IoT sources.”  One of Hadoop’s strengths is its ability to bring data together. Companies used to struggle with just combining internal data that was often in siloes. In today’s world, companies must also find ways to combine and process external data with their internal information and that is where Hadoop  can come in to help.

As January winds down, what does your team hope to accomplish on the big data front in 2016 and what tools are you most interested in learning more about?