Machine learning and artificial intelligence have exploded onto the scene in recent years, offering the hope of greater business efficiency. At the same time, researchers have found virtually no empirical evidence supporting the promised strides in labor productivity and economic activity.
That is, until now.
A forthcoming paper from researchers in Olin Business School at Washington University in St. Louis draws a direct connection between language translation driven by artificial intelligence and an increase in international trade. The paper, which analyzes data from online e-commerce site eBay, is among the earliest tangible signs that AI and machine learning are living up to their promise.
“There is plenty of anecdotal evidence that AI has exceeded humans in many areas, but there was not much causal empirical evidence,” said Meng Liu, visiting assistant professor of marketing at Olin. She notes that there is evidence that AI correlates with economic growth. “There seems to be a discrepancy between what our intuition says about AI versus what is actually observed,” she said.
For example, aggregate productivity growth rates have been stagnating since the 2000s.
Liu and coauthor Xiang Hui, assistant professor of marketing at Olin, cited 2017 research from MIT and the University of Chicago highlighting the paradox between high expectations and modest productivity results for artificial intelligence. That paper, accounting for slower economic activity, cited stagnant or declining numbers for productivity and median income while the new technology burst on the scene. Erik Brynjolfsson of MIT was a co-author on that 2017 paper as well as this study with Liu and Hui.
“Pointing to aggregate growth statistics, AI pessimists say it’s not really helping our productivity,” Hui added. “The problem is that it normally takes time for organizations to ramp up complementary innovations, be it organizational or technological, to harvest AI’s benefits. This is where our paper comes in. Let’s look at the question in a friction-free platform where they use an AI-based translation system.”
Their paper, “Does Machine Translation Affect International Trade? Evidence from a Large Digital Platform,” was accepted in April for publication in the journal Management Science.
Using data from eBay, which managed more than $14 billion in trade across more than 200 countries in 2014, Hui and Liu demonstrated that a moderate improvement in the quality of language translation increased trade between countries on eBay 10.9%.
The paper contrasted trading results between buyers and sellers in the United States against those in countries that do not primarily speak English, including countries in Latin America, Europe and Asia. They looked at trade before and after eBay implemented a new AI-driven machine translation service in 2014. By some measures, that service improved translation quality by about 10%.
The researchers also compared their results to a measure of trade cost Hui had explored in earlier research. As distances decrease between trading partners on eBay, the cost of trade drops. “What we show is that the introduction of machine translation is equivalent to reducing distances between countries by 26.1%.”
The research team took two approaches to analyzing the trade data. First, they contrasted U.S. exports to countries where the advanced translation was available against those where it was not. The graph above shows results after the technology was introduced—and the visible increase in U.S. exports to countries where it was available.
Next, the research team mitigated the effect of other factors that could have increased trade (more marketing, for example) by examining how machine translation affected longer versus shorter product titles. The theory was that translating longer titles required greater cost and effort, but would yield greater payoff — meaning the benefit of introducing machine translation should be higher for these items. Meanwhile, if eBay increased marketing, it would affect longer titles similarly to shorter ones. However, the graph below shows the greater effect on longer product titles.
“These comparisons suggest that the trade-hindering effect of language barriers is of first-order importance,” the researchers wrote. “Improved machine translation has made the eBay world significantly more connected.”
The authors noted that since their research was completed, Google has rolled out an even more powerful language translation tool “that has significantly improved translation quality” and, based on their research, “the effect (of Google’s software) on cross-border trade could be large.”
The authors argue that the introduction of machine translation on eBay provides a clean experiment where we can measure impacts. But ultimately, AI’s effect will like be seen in almost all economic sectors. As new systems come online, the authors wrote, “they will provide new opportunities to assess the economic impact of AI via natural experiments such as the one examined in this paper.”
