Big Data at Sears | SYS-CON INDIA

Big Data at Sears

Sears used IMS (IBM’s first generation database product) on mainframe plus Teradata

Nov. 8, 2012 10:00 AM

Sears plus its acquired entity Kmart belong to Sears Holdings whose goal is to get closer to its customers. That requires big time analytic capabilities. While revenue at Sears has declined from $50B in 2008 to $42B in 2011, rivals like Wal-Mart. Target and Amazon have grown steadily with better profit. Amazon’s retail business has gone from $19B in revenue in 2008 to $48B in 2011, passing Sears for the first time.

Sears used IMS (IBM’s first generation database product) on mainframe plus Teradata. Its ETL process using IBM DataStage software on a cluster of distributed servers took 20 hours to run. Since their adoption of Hadoop back in 2010, one of the steps (taking 10 hours out of the 20 hours) ran at 17 minutes. Their slogan is “ETL must die”, as they would like to load raw data directly to Hadoop. The old systems consisted of EMC Greenplum, Microsoft SQL Server, and Oracle Exadata (four boxes) for analytical workload. That is all being replaced by Hadoop, Datameer, MySQL, InfoBright, and Teradata.

Sears’ process for analyzing marketing campaigns for loyalty club members used to take six weeks on mainframe, Teradata, and SAS servers. The new process running on Hadoop can be completed weekly. For certain online and mobile commerce scenarios, Sears can now perform daily analyses. The Hadoop systems at 200 Terabytes cost about one-third of 200-TB relational platforms. Mainframe costs have been reduced by more than $500K per year while delivering 50-100 times better performance on batch jobs. The volume of data on Hadoop is currently at 2 Petabytes. As the CTO says, Hadoop is no longer a science project at Sears – critical reports run on the platform, including financial analyses; SEC reporting; logistics planning; and analysis of supply chains, products, and customer data. Sears uses Datameer, a spread-sheet style tool that supports data exploration and visualization directly on Hadoop. It claims to develop interactive reports in 3 days that used to take six to 12 weeks before.

Sears has actually spun off a new subsidiary called MetaScale to offer cloud services to other retailers with Hadoop platform. They are leveraging their three years of acquired expertise in Hadoop to make money in analytic services. There are many open questions on whether Hadoop will be that platform that brings big success to Sears in the future.

Read the original blog entry...

Published Nov. 8, 2012— Reads 3,589
Copyright © 2012 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.

About Jnan Dash
Jnan Dash is Senior Advisor at EZShield Inc., Advisor at ScaleDB and Board Member at Compassites Software Solutions. He has lived in Silicon Valley since 1979. Formerly he was the Chief Strategy Officer (Consulting) at Curl Inc., before which he spent ten years at Oracle Corporation and was the Group Vice President, Systems Architecture and Technology till 2002. He was responsible for setting Oracle's core database and application server product directions and interacted with customers worldwide in translating future needs to product plans. Before that he spent 16 years at IBM. He blogs at http://jnandash.ulitzer.com.

SOA World Latest Stories

The Real Value of Cloud Computing

By Elizabeth White

“Cloud has everything to do with what has happened with Big Data,” explained Jason Deck, Director of Strategic Alliances at Logicworks, in this exclusive Q&A with Cloud Expo Conference Chair Jeremy Geelan. “Big Data doesn’t exist in its easily accessible way without cloud. From reduced...

Cloud Expo New York Speaker Profile: Dave Linthicum – Cloud Technology Partners

By Jeremy Geelan

With Cloud Expo New York | 12th Cloud Expo [June 10-13, 2013] hurtling towards us, let's take a look at the distinguished individuals in our incredible Speaker Faculty for the technical and strategy sessions at the conference coming up June 10-13 at the Jacob Javits Center in New York ...

Cloud Expo New York: Migration to the Cloud for Data Intensive Jobs

By Pat Romanski

In his session at the 12th International Cloud Expo, Dave Eichorn, Global Data Center Practice Head at Zensar, will share a case study describing how a utility services company handled the migration of its Microsoft platform to the cloud. Challenged with the time-consuming task of open...

Focus on Business, Not Infrastructure

By Elizabeth White

Join us for an exclusive briefing with Gartner where they will talk about how IT Executives are modernizing data centers and transforming operations to focus resources on business value. During the session they will discuss how innovative IT solutions are needed to help IT deliver ag...

On-the-Go Big Data Requires a Different Approach: Four Guidelines

By Mark van Rijmenam

By the end of 2013, 1.4 billion smartphones will be in use by consumers from around the world. Around 2015, Gartner predicts that more tablets will be sold than PCs worldwide and in 2017 1.75 tablets will be sold for every PC. The growth of smartphones is not only in the Western world....

Splunk & Hortonworks Team Up

By Maureen O'Gara

Splunk, the software platform for real-time operational intelligence, and Hortonworks, the Hadoop Big Data distribution start-up, have allied so organizations can get operational intelligence using open source Apache Hadoop. Their pact means that data can be moved between Splunk Ente...