The Age of AI: Who is power shifting to?

Worst of all, the system’s decision-making capacity is not necessarily superior to the human brain; instead, it has eroded Bianlifeng’s profitability. Deprived of the frontline experience of store managers, Bianlifeng has made “basic errors” in the supply of best-selling items such as bread and drinks, as the system fails to accurately predict consumer demand or effectively improve the inventory turnover rate. Bianlifeng franchisees have reported that the rigid algorithmic system has actually led to an oversupply of hot meals, resulting in significant food waste. Consequently, a system that appears highly intelligent to outsiders is mockingly referred to as a “brainless idiot” by the convenience store staff.

Zhuang Chenchao admits that algorithmic systems can make mistakes, but as a techno-progressivist, he believes the solution is not to strengthen the decision-making power of store managers or use the human brain to correct the system. Instead, he argues for the continued development of system algorithms to enhance the “machine brain”, allowing it to self-correct through learning.

In other words, he believes that the “dumb machine” is only dumb because the technology is not yet advanced enough.

III. Human Usefulness

Yet, humans still have their uses. Bianlifeng store managers with practical experience often possess many insights into running a shop. They frequently have their own ideas on how to adjust stock types and product placement based on changes in season, temperature, weather, and holidays.

However, Bianlifeng’s system maintains a tight grip on decision-making power, and there isn’t even a channel for the system to “condescend” to listen to a manager’s suggestions.

In truth, since the invention of the machine, systems have almost always included roles dedicated to “tidying up” after the machine. At the start of the Industrial Revolution, machines showed an overwhelming advantage in power and mechanical transmission; while highly efficient, they were far less “intelligent” than today’s machines. They often required workers to assist and flexibly handle various unexpected situations.

After the invention of the spinning frame, hand-spinning was no longer necessary, but textile workers had to be ready at any moment to tie off broken cotton threads. Victorian textile mills even employed child labourers to crawl beneath the machinery to clear away cotton waste.

Today’s AI may seem intelligent, but the “firefighting” work of filling gaps and fixing errors is still left to humans. Apollo Go still requires safety drivers to handle emergencies, relying on humans to save the day at critical moments.

IV. Can Algorithms Care for Workers?

Meituan also claims that the “Real-time Intelligent Dispatch System” they developed for food delivery is incredibly powerful.

To demonstrate its computational power with data: during peak order periods, it can execute 2.9 billion route plans per hour, calculating the optimal match between orders, riders, and routes within tens of milliseconds, while simultaneously considering dozens of factors including weather, road conditions, rider availability, and the speed at which merchants prepare food.

According to Meituan insiders, the “Real-time Intelligent Dispatch System” adjusts delivery times based not only on real-time variables such as the location of the rider, consumer, and merchant, as well as the weather, but also on historical data from previous deliveries.

When a rider, fearing a timeout, drives against the flow of traffic or runs a red light on a particular route, the time saved at the cost of that rider’s life is absorbed by the algorithm and becomes the new time standard. Through this cycle, the delivery time for every single route is eventually compressed to the absolute limit.

Therefore, the increase in efficiency is not simply because the system is “better at calculating”, but because it can monitor and control countless delivery riders, absorbing their desperate, all-out “calculations” into the system.

The price of this efficiency, however, is the safety and lives of the riders.

According to a 2023 survey, 53 riders passed through a certain junction in Beijing within half an hour during a delivery peak, 37 of whom were riding the wrong way, cutting across the road, or running red lights. Shanghai also released a set of data: in the first half of 2017, an average of one delivery rider was injured or killed every 2.5 days. In 2023, the total number of traffic violations by Meituan delivery riders in Shanghai reached as high as 6,500 in a single week.

I experienced such a moment myself: one rainy night, fearing a timeout, I crashed heavily into a guardrail in the middle of the road. My electric bike was wrecked, but the order wouldn’t wait, so I had to finish the delivery on a shared bicycle.

Afterwards, I imagined that, at the very least, the platform could allow riders to upload information about traffic accidents to the system immediately, granting them a grace period for the delivery. I wouldn’t have been in such a wretched state, and injured riders would at least have some breathing room to deal with the aftermath.

The platform has invested massive resources into optimising the system to reduce delivery times for corporate profit; would it really be so difficult to leave a “safety valve” for riders in an accident? Is the failure to design a perfect system due to a lack of technical capability, or is it the result of the designers’ own conceptions of power?

Public discussion regarding rider accidents has persisted for years, yet there has been no clear improvement in the system. Whether delivery platforms are actually willing to take action is an answer that speaks for itself.

We cannot truly know if “technical reasons” are merely a pretext for refusing to change; after all, the power to develop algorithmic rules lies with the companies, and workers are forever excluded from the decision-making process.