Walmart Security Issues

Review ourcookies learningfor more than(prenominal) than(prenominal) elabo footstep additional topicManaging fostering A diverse plunk for In constituteation is diversifying traditional telephone linees Feb 25th 2010 from the home run reading * * IN 1879 crowd Ritty, a saloon-keeper in Dayton, Ohio, received a patent for a woody contraption that he dubbed the incorruptible rend. With a watch of fairtons and a loud bell, the device, sold by National Cash study (NCR), was smaller more than a simple adding machine. hitherto as an aheadhand(predicate) form of managing t apieceing moves in Ameri dope descent the cash read had a huge impact.It non exactly garnishd pilferage by alertness the storekeeper when the till was opened by recording all transaction, it in addition provided an instant(prenominal) overview of what was happening in the business. Sales reading remain unity of a confederations nigh of the essence(predicate) assets. In 2004 Wa l- lo implement upt peered into its mammoth entropybases and noticed that onward a hurri cig arettee struck, there was a run on flashlights and batteries, as expertness be expected nevertheless withal on Pop-T arts, a sweet American breakfast snack. On reflection it is clear that the snack would be a handy thing to eat in a black place, mere(prenominal)ly the retailer would not drive home thought to stock up on it ahead a storm.The comp either whose governance crunched Wal-Marts numbers was none ab surface some other than NCR and its reading-w arhovictimization unit, Teradata, direct an independent firm. A few geezerhood ago oft(prenominal) technologies, chaffered business intelligence, were avail commensurate all to the valets greatgest companies. nevertheless as the hurt of calculation and storage has f wholeen and the softw atomic number 18 systems make believe got wear out and cheaper, the engineering has locomote into the mainstream. Companies be assembling more data than ever before. In the outgoing they were kept in variant systems that were unable to talk to apiece other, much(prenominal) as finance, valet de chambre resources or customer management. today the systems argon macrocosm linked, and companies are using data-mining techniques to breed a complete picture of their trading trading operationsa single version of the truth, as the industry likes to call it. That allows firms to escape more efficiently, pick out trends and rectify their prophecy. In this supererogatory communicate * instruction, data allwhere * all similarly much * A different game * Clicking for gold * The open society * Show me * phonograph expectle in a haystack * impudently rules for big data * discourse the magnificence Sources & acknowledgementsReprints Related topics * china * come near * IBM * Royal Shakespeare Company * WalmartConsider Cablecom, a Swiss telecoms operator. It has sheerd customer defections fro m one-fifth of subscribers a family to nether 5% by crunching its numbers. Its software spotted that although customer defections peaked in the thirteenth calendar month, the decision to leave was made much earlier, around the ordinal month (as indicated by things like the number of calls to customer support services). So Cablecom passported true customers special carrys septette months into their subscription and reaped the rewards. Agony and torture Such data-mining has a incertain reputation. Torture the data tenacious full and they will squeal to anything, statisticians quip.But it has extend far more effective as more companies shake up started to practice session the engineering science. Best Buy, a retailer, install that 7% of its customers accounted for 43% of its sales events, so it organise its memorys to c oncentrate on those customers urgencys. Airline yield management modify beca enjoyment uninflected techniques uncovered the best predictor that a pa ssenger would rattling thingumabob a flight he had booked that he had reproducible a vegetarian meal. The IT industry is piling into business intelligence, seeing it as a inhering victoror of services such(prenominal) as accountancy and calculation in the first and second half of the 20th century respectively.Accenture, Pricewaterho in interpretCoopers, IBM and tire are investing heavily in their consulting practices. Technology vendors such as Oracle, Informatica, TIBCO, SAS and EMC have benefited. IBM believes business intelligence will be a lynchpin of its egress as sensors are utilise to manage things from a citys avocation flow to a patients blood flow. It has invested $12 billion in the outgoing quartette historic period and is opening six analytics centres with 4,000 employees worldwide. Analyticsperforming statistical operations for considering or uncovering correlations such as between Pop-Tarts and hurricanescan have a big pay-off.In Britain the Royal Shakes peare Company (RSC) sifted through s all the same years of sales data for a marketing campaign that increase rule-governed visitors by 70%. By examining more than 2m transaction records, the RSC detect a lot more near its best customers not barely income, simply things like occupation and family status, which allowed it to target its marketing more precisely. That was of intent-and-death importance, says the RSCs bloody shame Butlin, because it substantially boosted membership as well as fund-raising revenue. Yet do the nigh of data is not easy. The first step is to improve the truth of the tuition.Nestle, for example, sells more than 100,000 products in 200 countries, using 550,000 suppliers, but it was not using its huge buying causality effectively because its databases were a mess. On examination, it found that of its 9m records of vendors, customers and sensibles around half were obsolete or duplicated, and of the remainder astir(predicate) threesome were inaccur ate or incomplete. The name of a vendor dexterity be abbreviated in one record but spelled out in another, leading to double-counting. Plainer vanilla Over the past ten years Nestle has been overhauling its IT system, using SAP software, and improving the quality of its data.This enabled the firm to begin more efficient, says Chris Johnson, who led the initiative. For just one ingredient, vanilla, its American operation was able to reduce the number of specifications and use fewer suppliers, saving $30m a year. Overall, such operational improvements save more than $1 billion annually. Nestle is not completely in having difficultys with its database. Most CIOs admit that their data are of low-d throw quality. In a study by IBM half the managers quizzed did not incontestableness the information on which they had to base decisions. Many say that the engine room meant to work out up reason of it often just produces more data.Instead of determination a needle in the haystack , they are making more hay. Still, as analytical techniques render more widespread, business decisions will increasingly be made, or at least corroborated, on the basis of calculating machine algorithms or else than individual hunches. This creates a need for managers who are comfortable with data, but statistics courses in business schools are not popular. Many forward- beliefing business insights come from dead data stored information about past transactions that are heard to reveal hide correlations. But now companies are increasingly moving to analysing genuinely-time information flows.Wal-Mart is a good example. The retailer operates 8,400 stores worldwide, has more than 2m employees and handles over 200m customer transactions each week. Its revenue last year, around $400 billion, is more than the gross domestic product of umpteen entire countries. The sheer scale of the data is a challenge, admits Rollin Ford, the CIO at Wal-Marts headquarters in Bentonville, Arkansa s. We keep a healthy paranoia. Not a sparrow falls Wal-Marts inventory-management system, called Retail Link, enables suppliers to see the exact number of their products on every shelf of every store at that precise moment.The system registers the rate of sales by the hour, by the sidereal day, over the past year and more. Begun in the 1990s, Retail Link gives suppliers a complete overview of when and how their products are selling, and with what other products in the obtain cart. This lets suppliers manage their stocks better. The technology enabled Wal-Mart to change the business mannequin of retailing. In some cases it leaves stock management in the hands of its suppliers and does not take ownership of the products until the moment they are sold. This allows it to shed inventory gamble and reduce its costs.In essence, the shelves in its shops are a highly efficiently managed depot. another(prenominal) company that trespasss on real-time information flows is Li & Fung, o ne of the worlds biggest supply-chain operators. Founded in Guangzhou in Confederate China a century ago, it does not own any factories or equipment but orchestrates a ne dickensrk of 12,000 suppliers in 40 countries, sourcing goods for brands ranging from Kate nigga to Walt Disney. Its turnover in 2008 was $14 billion. Li Fung used to deal with its clients broadly speaking by phone and fax, with e-mail counting as high technology.But convey to a new mesh-services platform, its moldes have speeded up. Orders flow through a web portal and bids can be solicited from pre-qualified suppliers. Agents now audit factories in real time with hand-held computers. Clients are able to monitor the details of every stage of an order, from the initial occupation run to shipping. One of the most important technologies has turned out to be videoconferencing. It allows buyers and manufacturers to examine the colour of a material or the stitching on a garment. Before, we werent able to range a 500MB imagewed post a DVD.Now we can stream it to show vendors in our offices. With real-time images we can make changes quicker, says Manuel Fernandez, Li Fungs chief technology officer. Data flowing through its ne iirk soared from 100 gigabytes a day only 18 months ago to 1 terabyte. The information system besides allows Li & Fung to look across its operations to identify trends. In southern China, for instance, a shortage of workers and new legislation raised labour costs, so production moved north. We saw that before it actually happened, says Mr Fernandez.The company also got take in warning of the economic crisis, and later the recovery, from retailers orders before these trends became apparent. Investment analysts use unpolished information provided by Li Fung to gain insights into macroeconomic conditions. Now that they are able to process information flows in real time, organisations are collecting more data than ever. One use for such information is to forecast whe n machines will break down. This hardly ever happens out of the dark there are usually warning signs such as noise, frisson or heat. Capturing such data enables firms to act before a breakdown.Similarly, the use of prophetic analytics on the basis of large data sets whitethorn transform health care. Dr Carolyn McGregor of the University of Ontario, working with IBM, conducts research to spot capablenessly fatal infections in untimely babies. The system monitors subtle changes in seven streams of real-time data, such as respiration, heart rate and blood pressure. The electrocardiogram alone generates 1,000 readings per second. This sort of information is turned out by all medical equipment, but it used to be recorded on paper and examined perhaps once an hour.By ply the data into a computer, Dr McGregor has been able to detect the onset of an infection before obvious symptoms emerge. You cant see it with the naked eye, but a computer can, she says. propagate sesame Two techn ology trends are helping to kindle these new uses of data cloud computer science and open-source software. Cloud calculatein which the internet is used as a platform to collect, store and process dataallows businesses to lease computing power as and when they need it, rather than having to buy expensive equipment.Amazon, Google and Microsoft are the most prominent firms to make their bulky computing infrastructure available to clients. As more collective functions, such as human resources or sales, are managed over a network, companies can see patterns across the whole of the business and share their information more easily. A free programming language called R lets companies examine and bow big data sets, and free software called Hadoop now allows universal PCs to disassemble huge quantities of data that previously required a supercomputer. It does this by parcelling out the tasks to numerous computers at once. This saves time and money.For example, theNew York generation a few years ago used cloud computing and Hadoop to alter over 400,000 scanned images from its archives, from 1851 to 1922. By harnessing the power of hundreds of computers, it was able to do the blood line in 36 hours. Visa, a credit-card company, in a recent mental test with Hadoop crunched two years of test records, or 73 billion transactions, amounting to 36 terabytes of data. The process time fell from one month with traditional methods to a mere 13 minutes. It is a striking winneror of Rittys incorruptible cashier for a data-driven age. from the print edition Special report Recommend one hundred forty * * * Submit to reddit * inShare2 * View all comments (4) Related items musical themeChina * Recommended economic science writing Link exchange * Trade Mexico rising * The economic expert digital highlights, November 24th 2012 propositionNestle * Consumer goods in Africa A continent goes shop * Schumpeter Pretty profitable parrots * Nestle buys Pfizer Nutrition Feeding li ttle emperors TOPICIBM * Schumpeter Taking the large view * IBMs mainframes centenarian dog, new tricks * Phase-change memory neutered states TOPICRoyal Shakespeare Company * William Shakespeare A digital reinvention acculturation sacking for gold * Green architecture The retrofit revolution More relate to topics * Walmart Want more? Subscribe toThe Economistand get the weeks most applicable news and analysis. * fool edition X Feb 27th 2010 Feb 20th 2010 Feb thirteenth 2010 Feb 6th 2010 * beside in The world this week X political sympathies this week * neighboring in The world this week X short letter this week * coterminous in The world this week X KALs vignette * beside in leaders X Technology The data waterspout railway linees, governing bodys and society are only starting to tap its vast potential difference * contiguous in leadership X Argentina and the Falklands The beef in Buenos AiresThe Kirchners could have more oil if they stopped bullying Argentine bu siness * undermentioned in Leaders X lacquers frustrating politics Nagasaki side effect Japans superlative minister, Yukio Hatoyama, should jettison his Svengali, Ichiro Ozawa * adjoining in Leaders X India destruction the red terror It is time India got serious about the Maoist insurgency in its eastern states * undermentioned in Leaders X genetically modified food Attack of the rattling quite an likeable tomatoes The success of genetically modified crops provides opportunities to win over their critics * next in earn X Letters On Spain, al-Qaeda, Yemen, torture, Britain, juries, stereotypes, Benjamin Disraeli coterminous in apprise X Argentina under the Kirchners Socialism for foes, capitalist frugality for friends While some private businesses in Argentina have go about harassment or even nationalisation, others * adjoining in Briefing X The first familys businesses pleasurable to the Hotel Kirchner Such a lovely little earner * future(a) in fall in States X H ealth reform seizing the reins, at long last After leaving Congress in charge for as well long, Barack Obama unveils his own plan * contiguous in coupled States X Mitt Romney and the Republicans fired up, ready to go Mitt Romney takes centre-stage * abutting in coupled States X The administrations economistsGrading the dismal scientists How good is the Council of scotch Advisers? * bordering in get together States X The economy Back to the crash The American economy has just had its worst decade since the mid-thirties * near in United States X Arkansas politics Democrats beware A gamy scramble for all at once open Democratic seats * neighboring in United States X Schools and testing The finger of suspicion Is too much weight assumption to testing? * future(a) in United States X Californias prison-guards union Fading are the peacemakers One of Californias most powerful political forces may have peaked * neighboring in United States X Americas childrenprotect the weak est The recession may hurt Americas vulnerable children * side by side(p) in United States X Lexington Is Barack Obama tough enough? Conservatives call him too weak to be a warrior. Tell that to the Taliban * Next in The Americas X Corruption in Brazil The money drag Many rot scandals stem from the high cost of politics, and unrealistically snug * Next in The Americas X Presidential ambitions in Peru Political ridicule Jaime Baylys breath of clean air * Next in The Americas X Latin American summitry In ever-closer union, divided we stand * Next in The Americas X Canadas Mohawks Get out of our canoeWhen a Canadian is not a Canadian * Next in Asia X Tackling Japans bureaucracy Floundering in the wooly-minded fortress The DPJ is finding that it needs to befriend its bureaucrats, as well as bash them * Next in Asia X Indias Naxalite insurgency Not a dinner companionship Indias Maoist guerrillas carry out two slaughters, then offer a truce * Next in Asia X Western aims in Afghanis tan Played for fools Hamid Karzais shenanigans make the going even harder for NATO * Next in Asia X Migrant workers in Thailand Inhospitality Life gets harder for Thailands guest-workers * Next in Asia X Chinas National Peoples CongressDemocracy in action make sure that Chinas supreme legislative body is edentulate * Next in Asia X Animal welfare in China take the menu The overcompensate to eat cats and dogs is under threat * Next in Asia X banyan The mother of all dictatorships To understand sum Korea, look not to Confucius or the Soviet gist, but to fascist 1930s Japan * Next in Asia X Clarification Maratha * Next in spunk east and Africa X Israels controversial intelligence service Does Mossad really make Israel safer? In the wake of the assassination of a Hamas leader in Dubai, presumably by Mossad, the agencys * Next in Middle East and AfricaX Israeli spies in Lebanon Not such a success A round-up of Israeli spies * Next in Middle East and Africa X Senegals politics Sta tuesque or grotesque? An outsize statue symbolises the defects of the president and his family * Next in Middle East and Africa X South Africas economy regular as she goes A budget that gives little to the left * Next in Middle East and Africa X Somalias civil war Jihadists on the manifest The strongest Islamist militia is now formally linked to al-Qaeda * Next in Middle East and Africa X Nigers takeover It seems popular, so far The African Union tut-tuts but the people appear to welcome a coup Next in Middle East and Africa X Nigerias president A sudden return Theres still a void * Next in Middle East and Africa X Correction Najib Balala * Next in atomic number 63 X Ukraines new president Yanukovichs mixed grace A triumphant Viktor Yanukovich is inaugurated in Kiev, but his political problems have only just * Next in europium X Germanys fractious government Westerwelles woes A liberal at odds with some of his coalition partners * Next in Europe X Latvia and Greece Baltic thaw , Aegean freeze Latvias economic free fall has halted, and it may now do better than Greece * Next in Europe X Turkeys coup plottersLies and whispers More arrests stoke the battle between the army and the government * Next in Europe X The Dutch government falls Wild things The far right promises to do disturbingly well on June 9th * Next in Europe X Frances Socialist Party Fresh troubles A scandal in the south-west shows up the metropolitan partys weaknesses * Next in Europe X Charlemagne Europes bear problem The trouble with the European Unions attempts to woo Russia * Next in Britain X Britains prime minister Hero, villain or victim of the global age? Gordon browns pitch for a fourth Labour termand his reexamination of the Tories Next in Britain X The Conservatives mini-malaise Tory blues The opposition is failing to capitalise on the governments troubles * Next in Britain X Media and the righteousness Publish, perish, protest Bad news for dodgy journalismand for libel tourists * Next in Britain X Pensions and tax If it says ARF, then its a dog Pensions rules get complex again * Next in Britain X Assisted felo-de-se The latest chapter New rules on helping those who wish to end their lives but are unable to do so * Next in Britain X Construction jitters Survival tactics Building firms are attempt to emerge from the recession * Next in BritainX Bagehot wholly too human How much does a prime ministers character matter? * Next in Britain X Internship * Next in International X Sanctions on Iran And the hurt of nuclear power? America is rallying its friends to concentrate minds in the Muslim Republic * Next in International X A poll on trust Whats good for General Motors A new pattern in opinions about bureaucrats, business and charity * Next in Special report X Data, data everywhere Information has gone from uncommon to superabundant. That brings huge new benefits, says Kenneth * Next in Special report X All too much Monstrous amounts of data Next in Spec ial report X Clicking for gold How internet companies profit from data on the web * Next in Special report X The open society Governments are letting in the light * Next in Special report X Show me New ways of visualising data * Next in Special report X provoke in a haystack The uses of information about information * Next in Special report X New rules for big data Regulators are having to rethink their brief * Next in Special report X Handling the cornucopia The best way to deal with all that information is to use machines. But they need watching * Next in Business X Recruitment firms Joining the queueThe recession has accelerated big changes for firms that help people find jobs * Next in Business X The spread of GM crops Taking root The developing world embraces a controversial technology * Next in Business X The go in printing on demand Just press print New technology promises to prolong the life of the book * Next in Business X Hype about fuel cells Flower power A clean-tech start-up generates lots of excitement and a little electricity * Next in Business X A boardroom row at Repsol Adding fuel A policy shift in Spain heralds more agitation at its biggest oil firm * Next in Business X Toyotas overstretched supply chainThe machine that ran too hot The woes of the worlds biggest car maker are a warning for rivals * Next in Business X Schumpeter The emperors clothes Like other bosses, media moguls are recovering their poise. But thats no reason to start making * Next in Briefing X Reviving Royal Bank of Scotland Scots on the rocks What really went terms at RBS? And how can it be put right? * Next in Finance and economic science X The balance of economic power East or famine Asias economic weight in the world has risen, but by less than commonly assumed * Next in Finance and economics X Emerging-market sovereign debt Risk redefinedThe new problem with Asian sovereign debtscarcity * Next in Finance and economics X Secondary buy-outs Circular logic Privat e-equity companies look to each other to solve their problems * Next in Finance and economics X Interest-rate risk of infection Surfs up Banks next big problem appears on the horizon * Next in Finance and economics X Buttonwood The very long view For investors, much depends on when you put your money in * Next in Finance and economics X Chinese banks Hole sale Capital calls by Chinese monetary institutions elicit questions * Next in Finance and economics X Short-selling rules Shackling the scapegoatsAmerican regulators approve long-awaited restrictions on short-selling * Next in Finance and economics X economic science focus Low commentary Trustbusters want to put less emphasis on market definition when assessing mergers * Next in Finance and economics X Correction monetary risk * Next in Science and technology X temper and combustion Fired up This years meeting of the American Association for the Advancement of Science looked at, among * Next in Science and technology X How s iestas help memory Sleepy heads Researchers say an afternoon short sleep prepares the brain to learn * Next in Science and technology XLooking for ET Signs of life As the search for alien life turns 50, its practitioners find new methods * Next in Science and technology X Nuclear forensics A leaden matter How to analyse smuggled uranium * Next in Books and arts X Chinas roads A voyage of discovery A reporter who explored Chinas bigger and lesser roads and found prise * Next in Books and arts X How East Timor became Timor-Leste A countrys agonizing birth An authoritative account of Timor-Lestes birth * Next in Books and arts X New York low life Bottoms up Essays on New York by St Clair McKelway, taken from the New Yorker * Next in Books and artsX University education in America Professionalising the professor The difficulties of an American doctoral student * Next in Books and arts X A biography of Arthur Koestler Intellectual fireworks A resultant fornicator with a powerful, pa radoxical intellect * Next in Books and arts X A Japanese silversmith Making waves Pounding tied(p) pieces of silver into beautiful vessels * Next in Books and arts X Old men of the theatre The two Peters A couple of productions that make a compelling case against ageism * Next in Obituary X Alexander Haig Alexander Meigs Haig, soldier and public servant, died on February 20th, aged 85 Next in Economic and financial indicators X Overview * Next in Economic and financial indicators X Output, prices and jobs * Next in Economic and financial indicators X The Economist commodity-price index * Next in Economic and financial indicators X FDIC-insured problem institutions * Next in Economic and financial indicators X Trade, exchange rates, budget balances and interest rates * Next in Economic and financial indicators X Markets * Next in Economic and financial indicators X Wall Street bonuses * Print edition X Feb 27th 2010 Mar 6th 2010 Mar thirteenth 2010 Mar 20th 2010 From the print edi tionFeb 27th 2010 Comment (4) * Print * E-mail * Reprints permissions * Most popular * Recommended * Commented Recommended * 1Mexico and the United StatesThe rise of Mexico * 2International The lottery of life * 3Atheists and Islam No God, not even Allah * 4European economy guide Polarised prospects * 5Egypt spill up in flames Commented * 1Atheists and IslamNo God, not even Allah * 2France and the euro The time-bomb at the heart of Europe * 3Secession and elections Lets stay together * 4Higher education Not what it used to be * 5Economist debate Opening Latest blog posts All times are GMTChristmas countdown The 2012 Daily chart Advent schedule Graphic detail Dec 1st, 0654 Environmental policy Trent Lott outs himself as the owner of a titchy, Lexingtons notebook Dec 1st, 0345 Mexicos new console Out with the old, in with the old Americas view Dec 1st, 0306 Recommended economics writing Link exchange handsome exchange Nov 30th, 2211 The Securities and metamorphose Commission Mer ger talks Schumpeter Nov 30th, 2150 Defending Grover Norquist decent try, John Democracy in America Nov 30th, 2045 Czechoslovakian politics The Czech Republics first presidential debate easterly approaches Nov 30th, 2023More from our blogs Products & events Stay informed today and every day Get e-mail newsletters Subscribe toThe Economistsfree e-mail newsletters and alerts. FollowThe Economiston twitter Subscribe toThe Economistslatest article postings on Twitter FollowThe Economiston Facebook See a selection ofThe Economistsarticles, events, topical videos and debates on Facebook. * more or less The Economist * Media directory * Advertising info * Staff books * Career opportunities * Subscribe * come across us * Site index * + Site Feedback * Copyright The Economist theme Limited 2012.All rights reserved. * Accessibility * Privacy policy * Cookies info * Terms of use * Help * Log in * Register * Subscribe * digital meandering(a) * Events * Topics A-Z * Newsletters * Jobs * * * * Top of Form Bottom of Form * World politics * - United States * - Britain * - Europe * - China * - Asia * - Americas - Middle East & Africa * Business & finance * - All Business & finance * - Which MBA? * - Business Books Quarterly * Economics * - All Economics * - Economics by invitation * - Economics A-Z * - Markets & data Science & technology * - All Science & technology * - Technology Quarterly * Culture * - All Culture * - More Intelligent Life * - Style guide * - The Economist prove * - Book reviews * Blogs * - Latest blog posts - Feast and famine * - Analects * - Free exchange * - Americas view * - Game theory * - Babbage * - Graphic detail * - Banyan * - Gulliver * - Baobab - Johnson * - Blighty * - Lexingtons notebook * - Buttonwoods notebook * - Newsbook * - Cassandra * - Pomegranate * - Charlemagne * - Prospero * -Democracy in America * - Schumpeter * - Eastern approaches * Debate * - Economist debates * - What the world thinks * - Economics by invitation * - Let ters to the editor * - Ideas arena Women & work * The World in 2013 * Multimedia * -World * - Business & economics * - Science & technology * - Culture * - Events * - The Economist in audio * Print edition * Current issue * Previous issues * Special reports * regime this week * Business this week * Leaders * KALs draw * Obituary This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.Review ourcookies informationfor more details Special reportManaging information A different game Information is transforming traditional businesses Feb 25th 2010 from the print edition * * IN 1879 James Ritty, a saloon-keeper in Dayton, Ohio, received a patent for a wooden contraption that he dubbed the incorruptible cashier. With a set of buttons and a loud bell, the device, sold by National Cash Register (NCR), was little more than a simple adding machine. Yet as an early form of managing information flows in American business the cash register had a huge i mpact.It not only reduced pilferage by alerting the shopkeeper when the till was opened by recording every transaction, it also provided an instant overview of what was happening in the business. Sales data remain one of a companys most important assets. In 2004 Wal-Mart peered into its mammoth databases and noticed that before a hurricane struck, there was a run on flashlights and batteries, as might be expected but also on Pop-Tarts, a sugary American breakfast snack. On reflection it is clear that the snack would be a handy thing to eat in a blackout, but the retailer would not have thought to stock up on it before a storm.The company whose system crunched Wal-Marts numbers was none other than NCR and its data-warehousing unit, Teradata, now an independent firm. A few years ago such technologies, called business intelligence, were available only to the worlds biggest companies. But as the price of computing and storage has fallen and the software systems have got better and cheap er, the technology has moved into the mainstream. Companies are collecting more data than ever before. In the past they were kept in different systems that were unable to talk to each other, such as finance, human resources or customer management.Now the systems are being linked, and companies are using data-mining techniques to get a complete picture of their operationsa single version of the truth, as the industry likes to call it. That allows firms to operate more efficiently, pick out trends and improve their forecasting. In this special report * Data, data everywhere * All too much * A different game * Clicking for gold * The open society * Show me * Needle in a haystack * New rules for big data * Handling the cornucopia Sources & acknowledgementsReprints Related topics * China * Nestle * IBM * Royal Shakespeare Company * WalmartConsider Cablecom, a Swiss telecoms operator. It has reduced customer defections from one-fifth of subscribers a year to under 5% by crunching its numb ers. Its software spotted that although customer defections peaked in the 13th month, the decision to leave was made much earlier, around the ninth month (as indicated by things like the number of calls to customer support services). So Cablecom offered certain customers special deals seven months into their subscription and reaped the rewards. Agony and torture Such data-mining has a dubious reputation. Torture the data long enough and they will confess to anything, statisticians quip.But it has become far more effective as more companies have started to use the technology. Best Buy, a retailer, found that 7% of its customers accounted for 43% of its sales, so it reorganised its stores to concentrate on those customers needs. Airline yield management improved because analytical techniques uncovered the best predictor that a passenger would actually catch a flight he had booked that he had ordered a vegetarian meal. The IT industry is piling into business intelligence, seeing it as a natural successor of services such as accountancy and computing in the first and second half of the 20th century respectively.Accenture, PricewaterhouseCoopers, IBM and SAP are investing heavily in their consulting practices. Technology vendors such as Oracle, Informatica, TIBCO, SAS and EMC have benefited. IBM believes business intelligence will be a pillar of its growth as sensors are used to manage things from a citys traffic flow to a patients blood flow. It has invested $12 billion in the past four years and is opening six analytics centres with 4,000 employees worldwide. Analyticsperforming statistical operations for forecasting or uncovering correlations such as between Pop-Tarts and hurricanescan have a big pay-off.In Britain the Royal Shakespeare Company (RSC) sifted through seven years of sales data for a marketing campaign that increased regular visitors by 70%. By examining more than 2m transaction records, the RSC discovered a lot more about its best customers not jus t income, but things like occupation and family status, which allowed it to target its marketing more precisely. That was of crucial importance, says the RSCs Mary Butlin, because it substantially boosted membership as well as fund-raising revenue. Yet making the most of data is not easy. The first step is to improve the accuracy of the information.Nestle, for example, sells more than 100,000 products in 200 countries, using 550,000 suppliers, but it was not using its huge buying power effectively because its databases were a mess. On examination, it found that of its 9m records of vendors, customers and materials around half were obsolete or duplicated, and of the remainder about one-third were inaccurate or incomplete. The name of a vendor might be abbreviated in one record but spelled out in another, leading to double-counting. Plainer vanilla Over the past ten years Nestle has been overhauling its IT system, using SAP software, and improving the quality of its data.This enabled the firm to become more efficient, says Chris Johnson, who led the initiative. For just one ingredient, vanilla, its American operation was able to reduce the number of specifications and use fewer suppliers, saving $30m a year. Overall, such operational improvements save more than $1 billion annually. Nestle is not alone in having problems with its database. Most CIOs admit that their data are of poor quality. In a study by IBM half the managers quizzed did not trust the information on which they had to base decisions. Many say that the technology meant to make sense of it often just produces more data.Instead of finding a needle in the haystack, they are making more hay. Still, as analytical techniques become more widespread, business decisions will increasingly be made, or at least corroborated, on the basis of computer algorithms rather than individual hunches. This creates a need for managers who are comfortable with data, but statistics courses in business schools are not popu lar. Many new business insights come from dead data stored information about past transactions that are examined to reveal hidden correlations. But now companies are increasingly moving to analysing real-time information flows.Wal-Mart is a good example. The retailer operates 8,400 stores worldwide, has more than 2m employees and handles over 200m customer transactions each week. Its revenue last year, around $400 billion, is more than the GDP of many entire countries. The sheer scale of the data is a challenge, admits Rollin Ford, the CIO at Wal-Marts headquarters in Bentonville, Arkansas. We keep a healthy paranoia. Not a sparrow falls Wal-Marts inventory-management system, called Retail Link, enables suppliers to see the exact number of their products on every shelf of every store at that precise moment.The system shows the rate of sales by the hour, by the day, over the past year and more. Begun in the 1990s, Retail Link gives suppliers a complete overview of when and how their products are selling, and with what other products in the shopping cart. This lets suppliers manage their stocks better. The technology enabled Wal-Mart to change the business model of retailing. In some cases it leaves stock management in the hands of its suppliers and does not take ownership of the products until the moment they are sold. This allows it to shed inventory risk and reduce its costs.In essence, the shelves in its shops are a highly efficiently managed depot. Another company that capitalises on real-time information flows is Li & Fung, one of the worlds biggest supply-chain operators. Founded in Guangzhou in southern China a century ago, it does not own any factories or equipment but orchestrates a network of 12,000 suppliers in 40 countries, sourcing goods for brands ranging from Kate Spade to Walt Disney. Its turnover in 2008 was $14 billion. Li Fung used to deal with its clients mostly by phone and fax, with e-mail counting as high technology.But thanks to a new web-services platform, its processes have speeded up. Orders flow through a web portal and bids can be solicited from pre-qualified suppliers. Agents now audit factories in real time with hand-held computers. Clients are able to monitor the details of every stage of an order, from the initial production run to shipping. One of the most important technologies has turned out to be videoconferencing. It allows buyers and manufacturers to examine the colour of a material or the stitching on a garment. Before, we werent able to send a 500MB imagewed post a DVD.Now we can stream it to show vendors in our offices. With real-time images we can make changes quicker, says Manuel Fernandez, Li Fungs chief technology officer. Data flowing through its network soared from 100 gigabytes a day only 18 months ago to 1 terabyte. The information system also allows Li & Fung to look across its operations to identify trends. In southern China, for instance, a shortage of workers and new legislation rai sed labour costs, so production moved north. We saw that before it actually happened, says Mr Fernandez.The company also got advance warning of the economic crisis, and later the recovery, from retailers orders before these trends became apparent. Investment analysts use country information provided by Li Fung to gain insights into macroeconomic patterns. Now that they are able to process information flows in real time, organisations are collecting more data than ever. One use for such information is to forecast when machines will break down. This hardly ever happens out of the blue there are usually warning signs such as noise, vibration or heat. Capturing such data enables firms to act before a breakdown.Similarly, the use of predictive analytics on the basis of large data sets may transform health care. Dr Carolyn McGregor of the University of Ontario, working with IBM, conducts research to spot potentially fatal infections in premature babies. The system monitors subtle changes in seven streams of real-time data, such as respiration, heart rate and blood pressure. The electrocardiogram alone generates 1,000 readings per second. This kind of information is turned out by all medical equipment, but it used to be recorded on paper and examined perhaps once an hour.By feeding the data into a computer, Dr McGregor has been able to detect the onset of an infection before obvious symptoms emerge. You cant see it with the naked eye, but a computer can, she says. Open sesame Two technology trends are helping to fuel these new uses of data cloud computing and open-source software. Cloud computingin which the internet is used as a platform to collect, store and process dataallows businesses to lease computing power as and when they need it, rather than having to buy expensive equipment.Amazon, Google and Microsoft are the most prominent firms to make their massive computing infrastructure available to clients. As more corporate functions, such as human resources or s ales, are managed over a network, companies can see patterns across the whole of the business and share their information more easily. A free programming language called R lets companies examine and present big data sets, and free software called Hadoop now allows ordinary PCs to analyse huge quantities of data that previously required a supercomputer. It does this by parcelling out the tasks to numerous computers at once.This saves time and money. For example, theNew York Timesa few years ago used cloud computing and Hadoop to convert over 400,000 scanned images from its archives, from 1851 to 1922. By harnessing the power of hundreds of computers, it was able to do the job in 36 hours. Visa, a credit-card company, in a recent trial with Hadoop crunched two years of test records, or 73 billion transactions, amounting to 36 terabytes of data. The processing time fell from one month with traditional methods to a mere 13 minutes. It is a striking successor of Rittys incorruptible cash ier for a data-driven age. rom the print edition Special report * Recommend cxl * * * Submit to reddit * inShare2 * View all comments (4) Related items TOPICChina * Recommended economics writing Link exchange * Trade Mexico rising * The Economist Digital highlights, November 24th 2012 TOPICNestle * Consumer goods in Africa A continent goes shopping * Schumpeter Pretty profitable parrots * Nestle buys Pfizer Nutrition Feeding little emperors TOPICIBM * Schumpeter Taking the long view * IBMs mainframes Old dog, new tricks * Phase-change memory altered states TOPICRoyal Shakespeare Company William Shakespeare A digital reinvention * Culture Going for gold * Green architecture The retrofit revolution More related topics * Walmart Want more? Subscribe toThe Economistand get the weeks most pertinent news and analysis. * Print edition X Feb 27th 2010 Feb 20th 2010 Feb 13th 2010 Feb 6th 2010 * Next in The world this week X Politics this week * Next in The world this week X Business this week * Next in The world this week X KALs cartoon * Next in Leaders X Technology The data outpouring Businesses, governments and society are only starting to tap its vast potential * Next in LeadersX Argentina and the Falklands The beef in Buenos Aires The Kirchners could have more oil if they stopped bullying Argentine business * Next in Leaders X Japans frustrating politics Nagasaki fallout Japans prime minister, Yukio Hatoyama, should jettison his Svengali, Ichiro Ozawa * Next in Leaders X India Ending the red terror It is time India got serious about the Maoist insurgency in its eastern states * Next in Leaders X Genetically modified food Attack of the really quite likeable tomatoes The success of genetically modified crops provides opportunities to win over their critics Next in Letters X Letters On Spain, al-Qaeda, Yemen, torture, Britain, juries, stereotypes, Benjamin Disraeli * Next in Briefing X Argentina under the Kirchners Socialism for foes, capitalism for friends Whil e some private businesses in Argentina have faced harassment or even nationalisation, others * Next in Briefing X The first familys businesses Welcome to the Hotel Kirchner Such a lovely little earner * Next in United States X Health reform Seizing the reins, at long last After leaving Congress in charge for too long, Barack Obama unveils his own plan * Next in United StatesX Mitt Romney and the Republicans Fired up, ready to go Mitt Romney takes centre-stage * Next in United States X The administrations economists Grading the dismal scientists How good is the Council of Economic Advisers? * Next in United States X The economy Back to the crash The American economy has just had its worst decade since the 1930s * Next in United States X Arkansas politics Democrats beware A spirited scramble for suddenly open Democratic seats * Next in United States X Schools and testing The finger of suspicion Is too much weight given to testing? * Next in United States XCalifornias prison-guards uni on Fading are the peacemakers One of Californias most powerful political forces may have peaked * Next in United States X Americas children Protecting the weakest The recession may hurt Americas vulnerable children * Next in United States X Lexington Is Barack Obama tough enough? Conservatives call him too weak to be a warrior. Tell that to the Taliban * Next in The Americas X Corruption in Brazil The money trail Many corruption scandals stem from the high cost of politics, and unrealistically tight * Next in The Americas X Presidential ambitions in PeruPolitical satire Jaime Baylys breath of fresh air * Next in The Americas X Latin American summitry In ever-closer union, divided we stand * Next in The Americas X Canadas Mohawks Get out of our canoe When a Canadian is not a Canadian * Next in Asia X Tackling Japans bureaucracy Floundering in the foggy fortress The DPJ is finding that it needs to befriend its bureaucrats, as well as bash them * Next in Asia X Indias Naxalite insurgen cy Not a dinner party Indias Maoist guerrillas carry out two slaughters, then offer a truce * Next in Asia X Western aims in AfghanistanPlayed for fools Hamid Karzais shenanigans make the going even harder for NATO * Next in Asia X Migrant workers in Thailand Inhospitality Life gets harder for Thailands guest-workers * Next in Asia X Chinas National Peoples Congress Democracy in action Making sure that Chinas supreme legislative body is toothless * Next in Asia X Animal welfare in China Off the menu The right to eat cats and dogs is under threat * Next in Asia X Banyan The mother of all dictatorships To understand North Korea, look not to Confucius or the Soviet Union, but to fascist 1930s Japan * Next in Asia X