We are so excited to be part of the popular practitioner’s field guide to analytics and emerging tech, returning to Normandale Partnership Center for its fifth straight year on May 30. We can’t wait to explore AI, Machine Learning, Deep Learning, NLP, Robotic Process Automation, Graph Technologies, and much more. Plus, we have an opportunity to speak about hybrid cloud and share some awesome data case studies pertaining to The Weather Company. Although the event is sold out, we will be posting updates on the sessions on our blog, so stay tuned for some great information!
By Doug Polen, Software Sales Specialist
Many people don’t know this, but Weather Underground was a part of The Weather Company, which was acquired by IBM a few years ago. They had been offering an API that many customers and weather junkies had been using for quite some time to gather weather data for a vast array of applications.
To accommodate Weather Underground’s rapidly growing customer base, The Weather Company made the decision to move the Weather Underground platform to IBM’s enterprise API infrastructure and set this popular API to be shut down this month.
There are several different flavors of the API that are being published. The scalability of IBM’s infrastructure will allow existing Weather Underground customers to continue to receive the consistent customer experience they are used to, as well as better serve the developers working on the next generation of weather data.
As a result of this change, I was called upon to help with the surplus of inbound inquiries this decision created. Little did I know when I agreed to help work with these folks, the wealth of information I would take in about weather data and its seemingly endless use cases.
So why is this something that’s worth writing about? The Weather Company continues to remain the world’s most accurate forecaster and IBM is committed to ensuring its customers receive precise and accurate weather data at rapid speed.
Weather is something that impacts everyone daily in their personal lives. It’s a lot like the 82,000 memes you’ve seen on social media around the January 2019 Polar Vortex here in Minnesota … this guy survived his first -30° day ever, and here I am writing a weather blog in the postmortem.
Is it going to be sunny today?
What’s the wind chill today?
Will I need an umbrella?
This information is invaluable from a business perspective.
Will a snow storm impact a shipment?
Does an electric utility require more power today because more people will be using their AC’s?
What is the historical sun/wind/rain pattern that could impact agricultural output?
You get the point, the bottom line for business is having the ability to access better weather information can really have an impact on how business decisions are made. Helping clients make their businesses better and more competitive through data, is what I do. These new IBM offerings are both cost effective and robust.
Truth be told, in my sales career, I’ve never enjoyed the customer conversations more than I am right now. These are really fun conversations to have. I’ve learned about vineyards and how weather affects wine production, how off-shore oil rigs rely on accurate weather information to make decisions on worker safety and asset protection, and the ways public safety uses weather to predict what’s coming so they can best plan and schedule the resources necessary to keep public utilities going during weather events.
Speaking of fun, I’m positioning the Weather Company for Enterprise use, but I’ve got a new app on my phone that I simply love. WTForecast is a great app that I recently discovered (not affiliated with IBM). If you want a little humor (and let’s face it, when it’s -30° you need to laugh, albeit carefully, so as not to crack your face), be sure to give this one a try.
The point is, take a look at how decisions are made in your organization. Could better weather data help your company to make better choices to enhance profitability, make a more enjoyable workplace, or maybe even save an employee’s life? Let’s get to work today and uncover what we can do for your future.
Doug Polen is a Software Sales Specialist at Evolving Solutions. He has been with Evolving Solutions since 2015, after spending 16 years at IBM as a Software Client Leader and Client Executive.
He specializes in IBM PassPort Advantage, Software as a Service, Analytics, Cloud, Cognitive, IoT, Security, Social & Weather solutions and holds numerous IBM software certifications.
Like what you read? Follow Doug on LinkedIn.
Mary Shacklett of Tech Republic recently remarked, “Storage is an often-overlooked area, but with the increase in big data, it’s worth paying attention to.” She further explains that in her experience, instead of looking for underutilized data center storage, many managers simply purchase more. Big data injects more volume and complexity into your data center storage solution and simply “buying more” will eventually not be cost effective. Ms. Shacklett instead recommends that data center managers look to practices that optimize storage to prepare for big data. Here are some tips to get started:
- Make sure you have a solid tiered data strategy. Not all data is equal or even needed at all times, so your data center storage solution should not provide all access and anytime access for all. Tier your data based on importance and need and assign the best storage solution to handle
- Know your cloud storage costs. Make sure you have a clear understanding of the cost to scale your cloud storage when demand is high, so you can manage costs
- Manage your storage assets well. Inventory all of your storage assets. Chances are you will find not just underutilized storage, but storage that is not even being used. An IT asset management system can be great way to stay proactive
- Review your data retention policies. Once you start to use big data it can grow fast and having a strong strategy in place for what needs to be kept and for how long will help you manage the volume better
Before you jump into big data, it is important to make sure your data center storage solution and policies are ready, costs are transparent and space management and processes optimized.
We continue to see more and more businesses leverage the power of big data analytics to make decisions, grow their customer base and expand into new products and services. Today, let’s look at some of the trends driving this technology. If you are not yet using big data, at the end of this post we also outline how to get started.
Thor Olavsrud of CIO reports the following trends taking shape in the marketplace:
- AI will continue to shape processing. Mr. Olavsrud points out that AI or Artificial Intelligence is the umbrella word for machine learning, machine intelligence and cognitive computing and is proving to be a good choice to provide better scale and processing efficiencies for high volume repetitive tasks
- Silos of data will be replaced by clouds of data
- Data governance will continue to be a top discussion point for companies as their data stores grow
- IT will focus more on developing business-driven big data applications to ensure that big data analytics are being applied to real situations in real-time from back office processing to front office operations
- Analytics decisions will continue to move from post-event and real-time to preemptive decisions based on analytics. More embedded analytics in applications and systems will help decision makers drive better business outcomes
- “Data agility” will provide a clear competitive advantage. Big data analytics tools will continue to evolve to provide users the ability to understand data in context to take a business action
- Hybrid data center architectures will continue to advance
How to get started with big data?
Big data analytics can bring more decision making power to your company to drive better outcomes, but your data center must be prepared to take advantage. Start with a review of your data center architecture. Evolving Solutions can help you take this initial step and partner with you to review your workloads, data needs and capabilities.
We are all familiar with the potential benefits of big data analytics to a company’s future success, but how do we start laying the groundwork to support it? Here are some starting points from Information Management.
Data. First, define all your data sources. Determine where all your data currently lives and how it flows through your operation. You should also prioritize and categorize your data, identifying the critical pieces.
Infrastructure. How will your system move and support all of this data? How does your system handle structured and unstructured data? Evaluate the infrastructure in place and what would need to be developed or integrated to ensure data flows well between collection, processing and ultimately to your analytics applications for decision making. Another key part of infrastructure is its processing speed. Your platform also needs to be able to support rapid processing of raw data to ready it for analysis.
Storage. The sheer volume and formats of big data and the rate at which it can change can be daunting. You need a solid storage strategy in place. Utilize solutions that help you manage capacity, performance and cost.
Security. Part of building a solid foundation means thinking through data security. Be sure to explore the processes for securely transferring data in, out and around your data center and big data analytics applications.
Checks and balances. Analytical insights are only as good as the data put into the analysis. Discuss what oversight and review is needed to ensure data quality and integrity.
Big data analytics gives companies the ability to make more informative, real-time decisions, but it all starts with a solid foundation to operate on.
Big data analytics is providing many new insights for companies around the world. Bernard Marr reports for Forbes that predictions from big data analytics are being made every day by some of the most well-known companies in the world – buying habits on amazon.com, Netflix movies and even dating sites. There are many other situations that big data is being applied to that you might not realize. Mr. Marr points out several:
- Weather. Always a topic of conversation in Minnesota, weather affects not just our weekend plans but also the operations of many businesses. More accurate weather reports, especially in rural areas, are now possible due to the multitude of sensors and increase in mobile devices. More data allows for better weather predictions down the road
- Health and disease. From diabetes to cancer, big data analytics is being utilized to identify risk factors and early signs. Big data analytics is being used to improve patient outcomes as well as find ways to combat rising healthcare costs
- High school dropouts. Mr. Marr reports that nearly one in five students in the U.S. do not complete high school on time. Many states and even districts have their own programs in place to increase graduation rates, but what if a larger approach could be taken using big data methods. The Center for Data Science and Public Policy at the University of Chicago is doing just that – applying data discipline to identify dropout risk factors to help more youth complete high school
- Cyber attacks. A timely topic in recent news, big data analytics tools can actually be utilized to help detect and predict security threats
Big data analytics has many applications. From consumer buying behavior, patient symptoms, public policy and even the weather, the insights it can provide are on track to make long lasting impacts.
A recent article on the Star Tribune by Joe Carlson brought the conversation about healthcare and big data analytics close to home. Several Minnesota-connected companies led discussions on working across the industry to utilize data from insurance, hospital, pharma and medical device sources to drive better patient outcomes and manage rising costs. Working with big data analytics tools has been key to making this possible.
The first example comes from Medtronic and UnitedHealth. Many might think medical device and insurance companies might not always have the same goals in mind but not true. The two companies partnered together to discover the value of the features of a Medtronic insulin pump vs. other pumps on the market. Mr. Carlson reports they discovered that there was a decrease in hospitalizations due to complications from diabetes when the Medtronic pump was used, thus decreasing overall insurance costs. This led to a change of thinking. Dr. Richard Migliori, chief medical officer of UnitedHealth explains in the article, “It caused us to stop looking at line-item cost figures and start looking at, what is the total value? Because we saw a total value, we then began to wonder; shouldn’t we be (paying) on the basis of total value?” In 2016, UnitedHealth announced that Metronic would be its preferred insulin pump provider.
In Mr. Carlson’s article, he also gains insight from Mayo CEO John Noseworthy. Mr. Noseworthy adds that the industry has changed in the last five years and the atmosphere is one of “partnership, trust and transparency” in order to work together to bring down rising costs and improve patient outcomes.
Finally, St Jude Medical is working on a solution to reduce chronic pain by using a medical device versus pain medications. According to Mr. Carlson’s article prescription opioid painkillers are not only addicting but they lead to 15,000 deaths per year. Using big data analytics, St. Jude is looking to make the case for using its device which has a larger upfront cost but may reduce overall costs long term.
The power of big data analytics and a great spirit of partnership has helped make these connections possible.
68% of manufacturers are currently investing in big data analytics in the next twelve months and 67% are moving forward with these investments even as they cut in other areas. Mr. Louis Columbus of Forbes breaks down for us a recent analytics/IoT Honeywell survey of 200 North American manufacturing executives.
According to the survey, 46% of respondents no longer see big data analytics as “optional.” Instead, the benefits of big data analytics are clear to many in the industry:
- Equipment performance. Unscheduled downtime was sited as a top threat to revenue. 51% of respondents agree that the combination of the Industrial Internet of Things (IIoT) and big data analytics will help to predict equipment downtime, maintenance needs and breaks/repairs
- Supply chain management. 46% of the executives surveyed agree that big data analytics will help with supply chain performance by allowing them to better plan for and use resources more efficiently
- Safety. 47% of respondents agree that access to better equipment and operational data through big data analytics could help detect possible safety issues
Mr. Columbus reports most executives are fairly confident on the analytics path they are on. In fact, 65% report being on the “right track” or “above the curve” when it comes to their use of big data analytics. In summary, the results of this survey show that big data analytics will be key to allowing those in the manufacturing industry to improve revenue and operations for their companies.
Entrepreneur reports that by 2020 every person online will create 1.7 megabytes of new data every second, and according to a study on Computerworld, by that same year $72 billion will be spent on big data hardware, software and professional services. Data is growing and its coming in many forms both structured and unstructured, and companies are looking for the tools to help them manage and utilize it.
Big data can provide a more complete picture of your customer, marketplace or product/service. But before you can take advantage of big data, you must first have the right systems and tools in place. Now is the time to consider your own infrastructure to make sure the right workloads are working on the right technologies. In fact, it may be time for your company to take a deep look into how your data warehouse architecture can be modernized, so you are ready to move ahead with big data analytics projects.
Modernizing your data warehouse can increase your ability to deploy new workloads and handle new data sources. You can also simplify your operations. Modern appliances can be integrated with your data warehouse, filling service gaps and making deployment and management of data easier. Finally, a whole new set of tools for big data analysis are available. These analytics tools can integrate into your system allowing you to crunch massive amounts of data to form actionable insights.
So how do you get started on the right path? A trusted partner can help you develop a plan for the changes needed to modernize your data warehouse architecture and can help identify the right big data analytics tools for your company. Evolving Solutions takes the time to fully assess your data and business objectives, develop customized software solutions based on your needs, and we provide ongoing support. We take the time to understand your goals and help you to accomplish them. Contact us today to start discussing how to modernize your data warehouse.
Big data use has been expanding across a variety of industries. Today let’s focus on the banking industry and how it is using big data.
Business Insider reports that some of the biggest banks in the world hope to use the information found on social media channels like Facebook, Twitter and Instagram as part of a customer’s credit history or in place of one when none is available. Of course, developing a system to handle this type of data does not only bring technology and system challenges but also regulation and privacy challenges. Still, many feel tapping into social media would allow the industry to get amore complete picture of the consumer’s financial status and credit.
Bernard Marr spotlights Citibank’s use of big data in a Forbes article. Citibank is taking a “data-led” approach to its decision making and using big data analytics across areas such as customer retention and acquisition, customer service and compliance and fraud. Citibank has over 200 million customer accounts and operates in hundreds of countries which all equates to a lot of data to handle and store. Adopting big data tools and processes has also helped Citibank to utilize data better and manage and store data more effectively. Mr. Marr writes, “At Citi, model testing allows for a holistic understanding of innovative use cases by deconstructing data at its most granular level as well as synthesizing structured and unstructured data sources.”
The banking industry is making inroads with big data to better serve its customers. It continues to look for solutions that strike a balance between regulations and privacy and solutions that support the vast amounts of transactional and customer data it already has with new data sources.