Robots Need Us More Than We Need Them
24 February, 2022 / Articles
In our AI future, people—not the algorithms they deploy—will be the reason most companies succeed.
Imagine trying to find a particular image within the National Football League’s historical archive of hundreds of thousands of videos. A single season produces more than 16,320 minutes (some 680 hours) of game footage. If you include coverage of every pregame, halftime, and postgame show, every practice, and every media interview, you have a seemingly endless amount of footage. And that’s just for one season.
To make it easier for staffers to create highlight reels and other media from all this material, the NFL partnered with Amazon Web Services in December 2019 to use artificial intelligence to search and tag its video content. The first step of the process required the NFL’s content creation team to teach the AI what to find. The team created metadata tags for every player, team, jersey, stadium, and other visually recognizable content it wanted to identify within its video collection. It then combined those tags with Amazon’s existing image-recognition AI system, which Amazon had already trained on tens of millions of images. The AI was able to use both sets of data to flag relevant imagery within the video library, and the content creation team was able to approve each tag in just a few clicks. Whereas employees once had to manually search, find, and clip each video, store it in a repository, and then tag the video with metadata, Amazon’s AI automated most of the process.
In a previous HBR article (“Collaborative Intelligence: Humans and AI Are Joining Forces,” July–August 2018), we described how some leading organizations are defying the conventional expectation that technology will render people obsolete—they are instead using the power of human-machine collaboration to transform their businesses and improve their bottom lines. Now several companies are not merely out-innovating their competitors with this approach; they’re turning even more decisively toward human-centered AI technology and upending the very nature of innovation as it was practiced over the previous decade.
In the NFL’s case, for example, AI accelerated the image-recognition process, but the system would have failed without employees determining which data needed to be uploaded and then approved. And the NFL didn’t simply hand the job of making highlight reels over to AI; content creation experts performed that work, but they did it faster and more easily thanks to AI’s unique ability to quickly sort through massive volumes of information.
The new human-focused approach to AI is changing assumptions about the basic building blocks of innovation. Companies such as Etsy, L.L.Bean, McDonald’s, and Ocado are redefining how AI and automation can knit together a wide range of cutting-edge information technologies and systems that enable agile adaptability and seamless human-machine integration. (Disclosure: Several companies named in this article are Accenture clients.) These path-breaking firms have invested in digital technologies at unprecedented rates to respond to new operational challenges and rapidly shifting customer demands. They’ve dramatically increased investments in cloud services, AI, and the like, and they’re generating revenue at twice the speed of laggards, according to a 2019 Accenture survey of more than 8,300 companies. A second study, of more than 4,000 companies in 2021, shows that the 10% making the biggest commitment to digital technologies are rocketing even further ahead, growing revenue five times as fast as laggards.
We’ve turned what we’ve learned from this research into guidance that business leaders can use to compete in a world where most companies will owe their success to humans rather than machines. Our IDEAS framework calls for attention to five elements of the emerging technology landscape: intelligence, data, expertise, architecture, and strategy. It can help both technical and nontechnical executives to better understand those elements and conceive of ways they might be woven together into powerful engines of innovation.
In this article, we use the IDEAS framework to examine examples of businesses that have implemented human-driven AI processes and applications to solve problems in e-commerce, online grocery delivery, robotics, and more. You can do likewise, marshaling the skills and experience of your own people to manage technological innovation in everything from R&D and operations to talent management and business-model development.
Intelligence: Make AI More Human and Less Artificial
Human intelligence and artificial intelligence are complementary. No machine powered by AI can match the ease and efficiency with which even the youngest humans learn, comprehend, and contextualize. Accidentally drop an object and a one-year-old who sees you reaching for it will retrieve it for you. Throw it down on purpose and the child will ignore it. In other words, even very small children understand that people have intentions—an extraordinary cognitive ability that seems to come almost prewired in the human brain.
That’s not all. Beginning at a very young age, children develop an intuitive sense of physics: They expect objects to move along smooth paths, remain in existence, and fall when unsupported. Before they’ve acquired language, they distinguish animate agents from inanimate objects. As they learn language, they exhibit a remarkable ability to generalize from very few examples, picking up new words after hearing them only once or twice. And they learn to walk on their own, through trial and error.
Conversely, AI can do many things that people, despite being endowed with natural intelligence, find impossible or difficult to do well: recognize patterns in vast amounts of data; defeat the greatest champions at chess; run complex manufacturing processes; simultaneously answer many calls to customer service centers; analyze weather, soil conditions, and satellite imagery to help farmers maximize crop yields; scan millions of internet images in the fight against child exploitation; detect financial fraud; predict consumer preferences; personalize advertising; and much else. Most important, AI has enabled humans and machines to work together efficiently. And contrary to automation doomsayers, such collaboration is creating an array of new, high-value jobs.
At Obeta, a German electronics wholesaler whose warehouse is run by the Austrian warehouse logistics company Knapp, human workers are teaching a new generation of robot pickers how to handle differently sized and textured items. The robots employ an off-the-shelf industrial arm, a suction gripper, and a vision system. Crucially, they are also equipped with AI software from Covariant, a start-up based in California.
To train a robot, Knapp workers put unfamiliar objects in front of it and see if it can successfully adapt to them. When it fails, it can update its understanding of what it’s seeing and try different approaches. When it succeeds, it gets a reward signal, programmed by humans, to reinforce the learning. When a set of SKUs differs totally from other sets, the team reverts to supervised learning—collecting and labeling a lot of new training data, as happens with deep-learning systems.
Thanks to the Covariant Brain software, Knapp’s robot pickers are acquiring general-purpose abilities, including 3D perception, an understanding of how objects can be moved and manipulated, the capacity for real-time motion planning, and the capacity to master a task after only a few training examples (few-shot learning). These abilities enable them to perform their job—to pick items from bulk storage bins and add them to individual orders for shipping—without being told what to do. In many cases, the items have not been precategorized, which is unusual for industrial packaging systems; it means the robots are learning how to handle them in real time. This is a critical skill to have when dealing with electronics, especially when you consider the different care required to handle a light bulb and a stove.
To succeed in a commercial environment, robots must perform to a very high standard. Previously, Knapp’s robot pickers reliably handled only about 15% of objects; the Covariant-powered robots now reliably handle about 95% of objects. And they’re faster than humans, picking about 600 objects an hour versus 450 for humans. Nevertheless, they have not caused any staff layoffs off at the Obeta facility. Human workers, instead of losing their jobs, have been retrained to understand more about robotics and computers.
Data: Manage Info, Don’t Just Amass It
In 2018 McDonald’s was coming off one of its most challenging years in decades. Its competitors had used online delivery to leapfrog its lock on the fast-food market. The company’s leaders quickly devised an online delivery solution through a global partnership with Uber Eats that by 2019 was adding $4 billion to annual sales. But top executives knew that the company’s long-term future depended on making a rapid and complete transformation to become data-driven. That meant a strategy to reconfigure its restaurants into enormous data processors, complete with machine learning and mobile technology to support highly personalized customer orders and curbside delivery. Data crunching could also aid in calculating how external factors, from weather to big sporting events, would impact demand and restaurants’ ability to serve customers. And gathering and processing data was important for developing new products and initiatives that could be immediately successful. Within two years, the transformation effort had already achieved financial results: Few companies in the S&P 500 have outperformed McDonald’s. What the company’s leaders did was recognize that data was a source of valuable, untapped capital that needed to be used strategically.
To master the use of big and small data to generate value from AI, organizations must first lay a solid data foundation. Business data is often locked in legacy, on-site platforms that are siloed, making it difficult, if not impossible, for employees to get different types of data to work together. That makes it even harder for business users to find and process the right information to arrive at appropriate decisions. Creating a robust data foundation requires breaking information out of legacy silos so that it can be unified, optimally stored, easily accessed, and readily analyzed with new tools—all in the cloud.
Three capabilities are key: modern data engineering, AI-assisted data governance, and data democratization.
- Modern data engineering. In a strong, cloud-based foundation, data comes from multiple internal and external sources. It gets stitched together into curated and reusable data sets that can be employed for a variety of analytic purposes. A good foundation relies on frameworks for data ingestion and ETL (extract, transform, load) that support diverse data types. These frameworks also handle rules for standardizing information, classifying it, ensuring its quality, and capturing metadata. In addition, they enable a faster, templatized approach to using data, which allows engineers to quickly develop new analytic use cases and data products.
- AI-assisted data governance. Cloud-based AI tools offer the advanced capabilities and scale to automatically cleanse, classify, and secure data gathered in the cloud as it is ingested, which supports better data quality, veracity, and ethical handling.
- Data democratization. A modern data foundation gets more data into more hands. It makes data accessible and easy to use in a timely manner, while enabling multiple ways to analyze it, including through self-service, artificial intelligence, business intelligence, and data science. The latest cloud-based tools democratize data and empower more people across the enterprise to easily find and leverage information that’s relevant to their specific business needs.
Creating a robust data foundation requires breaking information out of legacy silos so that it can be unified, optimally stored, easily accessed, and readily analyzed—all in the cloud.
Together, these three capabilities help companies overcome some of the most common barriers to getting value from data: problems with its accessibility, trustworthiness, readiness for use, and timeliness. They enable companies to blend items from big and small data sets in real time, build agile reporting, and apply AI to create broadly accessible customer, market, and operational insights that deliver meaningful business outcomes.
With a solid data foundation—more data from more sources, managed with the help of AI and widely disseminated within your organization—you are no longer overwhelmed by data but able to maximize its potential. You can put it to increasingly powerful and fine-grained uses, but, just as with more-humanlike intelligence, that will require greater involvement by your people.
Expertise: Unleash Your Employees’ Talent
At Etsy, the online marketplace for vintage and handmade goods, the motto is “Keep commerce human.” And it took humans to teach the company’s search engine how to recognize what is the crux of many purchasing decisions—aesthetic style. When considering an item to buy, Etsy’s customers look not only at details such as its size, material, price, and ratings but also at its stylistic and aesthetic aspects.
For Etsy, classifying items by style is particularly challenging. Most of the products on its site are one-of-a-kind creations. Many borrow from multiple styles or exhibit no clearly identifiable style at all. And there are some 50 million items on offer at any given time. In the past, style-based recommendation systems produced unexplainable product suggestions for groups of shoppers. That’s because the AI assumed that two items must be similar in style if they are frequently purchased together by a common customer demographic. Another approach uses low-level attributes such as color and material to group items by style. Neither method has been able to understand how style affects purchasing decisions.
Who better to school AI in subjective notions of style than Etsy’s merchandising experts? Based on their experience, they developed 42 style labels that captured buyers’ taste across 15 categories from jewelry to toys to crafts. Some labels are familiar from the art world (art nouveau, art deco). Some evoke emotions (fun and humor, inspirational). The merchandisers produced a list of 130,000 items distributed across these 42 styles.
Etsy’s technologists then turned to buyers who tend to use style-related terms in their searches, typing in things like “art deco sideboard.” For each such query, Etsy assigned the chosen style name to every item the user clicked on, “favorited,” or bought during that search. From just one month of such queries, the company was able to collect a labeled data set of 3 million instances against which to test its style classifications. Etsy engineers then trained a neural network to use textual and visual cues to best distinguish between those classifications for each item. The result was style predictions for all 50 million active items on Etsy.com.
This became particularly useful when the Covid-19 pandemic struck and the supply chains of mass retailers broke down. Many buyers turned to Etsy for a much-needed product: masks. Among the hottest sellers within that category were masks tailored to the aesthetic sensibilities of customers, who could specify the design they were looking for—polka dots, floral patterns, animal faces, or what have you. Sales of masks went from virtually nothing at the beginning of April 2020 to some $740 million for the rest of the year. The company’s revenue more than doubled during that time, and its market value rose to $22 billion. The key was allowing buyers to find a mask “that expressed their sense of taste and style,” said Etsy CEO Josh Silverman.
Machine teaching will unleash the often-untapped expertise that exists throughout your organization, allowing a much broader swath of your people to use AI in new and sophisticated ways. Because it’s customizable for your business situation, it opens the way to real innovation and advantage—you no longer are simply playing technology catch-up. In supervised-learning scenarios, machine teaching is particularly useful when little or no labeled training data exists for the machine-learning algorithms—as it often doesn’t because an industry’s or a company’s needs are so specific.
To get the greatest value out of both systems and knowledge workers, organizations must reimagine the way nonspecialists as well as specialists interact with machines. You can begin by giving your domain experts a working knowledge of AI so that they can efficiently transfer their expertise to company processes and technology. Familiarity with the basics of artificial intelligence will also equip them to develop creative ways to apply it to the business.
Architecture: Build Adaptable, Living Systems
Legacy architectures are tightly bounded, maintaining barriers between lines of business, geographies, sales channels, and functions. They’re rigid, unable to adapt to new smart technologies or to accommodate new strategies, changing market conditions, and new operational opportunities. That’s why many companies’ innovation projects stall.
The rapid transformations that occur today and the sudden influx of new technologies have put IT architecture front and center. While laggards fail to seize the opportunity for IT innovation, leaders adopt a wide range of emerging information technologies and assemble them into what we call living systems because they are boundaryless, adaptable, and radically human.
By “boundaryless” we mean that they break down barriers—within the IT stack, between companies using cloud-based platforms to harness network effects, and between humans and machines—giving businesses infinite opportunities to improve the way they operate. By “adaptable,” we mean that the systems, powered by advances in data and intelligent technologies, rapidly adjust to business and technology change, minimizing friction, scaling innovation, and learning and improving. And when we describe the systems as “radically human,” we mean that they are modeled on human brains and behaviors and are able to listen, see, talk, and understand in more humanlike ways than previous generations of intelligent technology could.
Consider L.L.Bean, the 110-year-old retailer with a heritage that includes classic clothing, rugged outdoor gear, and a deep commitment to customer satisfaction. In recent years, as the company increasingly reached out to customers across multiple channels—print, brick-and-mortar stores, computer and mobile websites, email, and social media— it found itself hampered by a less valuable legacy: a cumbersome IT system, parts of which had been in use for two decades. Much of the system consisted of on-site mainframes and distributed servers. Different platforms, only loosely connected, supported each of the different customer channels, all of which were running on separate applications. Providing a seamless customer experience across all channels was next to impossible. And instead of focusing on delivering customer value, IT personnel had to spend time managing the infrastructure.
Meanwhile, 73% of U.S. consumers were using multiple channels for shopping, according to research reported on HBR.org (see “A Study of 46,000 Shoppers Shows That Omnichannel Retailing Works,” by Emma Sopadjieva, Utpal M. Dholakia, and Beth Benjamin). The research also indicated that multichannel shoppers spent more money than single-channel customers did—an average of 4% more on every trip to the store and 10% more online. Furthermore, multichannel shoppers were also more loyal and more likely to recommend a favored retailer to friends and family.
To compete successfully in the age of Amazon, L.L.Bean needed to offer customers a satisfying omnichannel experience that purely online retailers couldn’t match. So the retailer decoupled mission-critical applications from its legacy IT system and located them in Google’s cloud. The IT team can now integrate data from multiple systems, handle peak website loads more efficiently, and deliver new customer features faster. Because the cloud-based architecture is being continually optimized in the background, the company’s front-end developers spend less time managing it and more time using agile software to experiment with new features and launch them as soon as they’re ready. And with the flexible front-end architecture now residing in the cloud, decoupled from the legacy system, the company can easily, quickly, and cost-effectively scale up capacity in peak buying periods and scale down during lulls. This ability to rapidly respond to changing conditions is one of the most consequential advantages of living systems.
The way to this future will be determined by the choices your enterprise makes throughout your technology stack. You must transition to more human-centered approaches to AI and automation. You can start by accelerating investments in core technologies like cloud computing, data analytics, and mobility. You can reimagine your approach to application development to take advantage of cloud capabilities and microservices and the flexibility they unlock. And you can focus on creating reusable components that are maximally valuable rather than minimally viable. Organizations that successfully combine their business and technology strategies will be able to develop one-of-a-kind offerings with unprecedented agility.
Strategy: We’re All Tech Companies Now
For more than two decades Ocado, the world’s largest online grocery retailer with no physical stores, has been developing some of the world’s most advanced capabilities in AI, machine learning, robotics, cloud technologies, IoT (internet of things), simulation, and modeling—invaluable intellectual property that includes more than 150 patents, with hundreds more pending.
Ocado’s IP achievements are particularly remarkable because the grocery industry is one of the most demanding operating environments imaginable. It is the world’s largest retail category, and also one of the most complex: Unlike books or DVDs or many other goods, grocery products have widely varying shelf lives and storage temperature requirements. Take that complexity online, where customers who are spread over an entire country demand accurate and reliable order fulfillment at an attractive price, and the challenges increase exponentially.
Founded in 2000, Ocado grew from three people in a one-room office in London into a business with more than 18,500 employees serving hundreds of thousands of customers across the UK. Ocado’s customer fulfillment centers (CFCs) boast some of the most advanced grocery-picking technology in the world. A typical CFC is about the size of a soccer field. Inside, hundreds of robots, communicating with one another over a 4G network, wheel around a three-story aluminum grid known as the Hive.
Using swarm technology, which coordinates a group of autonomous robots to work as a system to accomplish tasks, the dishwasher-size robots bustle along at nearly nine miles per hour, lifting crates of grocery products with their mechanical claws. They either move the crates to another location (according to an algorithm based on frequency of product purchase) or drop them down a chute to a picking station. Two control centers staffed by employees are located at each CFC to monitor the robots and make sure their elaborate dance doesn’t degenerate into constant collisions. Human employees also do most of the work at the picking stations: They view a customer’s order on a screen, select the appropriate items from the product crates in front of them, and put them into shopping bags that robots have placed inside another crate. The product crates are then sent back to the grid to be refilled with items, while the crates with customers’ orders are routed to the shipping dock. A 50-item order can be fulfilled in as little as five minutes.
Organizations that successfully combine their business and technology strategies will be able to develop one-of-a-kind offerings with unprecedented agility.
Ocado could have rested on its laurels as a successful online grocer, but it made a strategic decision to extend its tech expertise further. In 2015 it created the Ocado Smart Platform, a combination of end-to-end e-commerce fulfillment, logistics, and swarm technology that other retailers around the world use to manage their own online grocery businesses. The platform allows them to profitably and scalably replicate Ocado’s model in their own regions.
Running in the cloud, the Ocado Smart Platform provides features such as real-time stock projection, last-minute order processing, and intelligent delivery-van routing. Retailers can offer customers mobile access to their sites via an app. And the cloud provides Ocado with an elastic, events-driven architecture that responds to spikes in customer demand in a cost-efficient way. It also enhances development agility. Ocado’s engineers can test out new initiatives without making upfront infrastructure commitments, and they can get ideas from concept to production in under an hour. The company can also integrate data from hundreds of microservices into a data lake that powers AI capabilities across the infrastructure.
