In the AI Era, Who Is Power Shifting To?

Worse still, the system’s decision-making ability has not necessarily surpassed the human mind; it has instead weakened Bianlifeng’s profitability. Stripped of the store manager’s frontline experience, Bianlifeng began making ‘elementary errors’ in supplying best-selling items such as bread and beverages. The system cannot accurately predict consumer demand or effectively improve inventory turnover rates. Bianlifeng franchisees report that the rigid algorithmic system has led to an over-supply of hot meals, resulting in enormous food waste. A system that appears strikingly intelligent from the outside has earned a nickname among shop assistants: “idiot”.

Zhuang Chenchao acknowledges that the algorithmic system makes mistakes too, but as a techno-progressivist, he believes the solution is not to strengthen store managers’ decision-making authority — to let the human mind correct the system’s errors — but to keep developing the algorithms, to strengthen the machine’s brain so that it can learn to self-correct.

In other words, he believes the ‘Silly Machines’ are silly simply because the technology is not yet up to scratch.

III. Humans Have Their Own Role to Play

Yet humans still have their uses. Bianlifeng store managers with years of practical experience often have plenty of ideas about how to run their shops. They have their own views on adjusting product ranges and shelf layouts according to changes in season, temperature, weather, and holidays.

But Bianlifeng’s system keeps a firm grip on every decision. There is not even a channel through which it might deign to listen to a store manager’s suggestions.

In fact, ever since machines were first invented, almost every system has included work for humans to do as a fallback. At the dawn of the Industrial Revolution, machines demonstrated an overwhelming advantage in power and mechanical transmission. They were highly efficient, but far from “intelligent” by today’s standards. Workers were needed to assist them, flexibly handling all manner of unexpected situations.

After the spinning mule was invented, no one had to spin thread by hand, yet female textile workers still had to be ready to rejoin broken cotton threads at a moment’s notice. Textile mills in the Victorian era also employed child labourers, who crawled beneath the machines to clear away scraps of cotton.

Today’s AI may appear intelligent, but the fire-fighting work of patching gaps and filling in the blanks still falls to humans. Apollo Go still needs safety operators to deal with emergencies, and when critical moments arrive, it is still humans who must step in to save the day.

IV. Can Algorithms Look After Workers Too?

Meituan also boasts that the “Real-time Intelligent Dispatch System” it developed for its food-delivery service is extraordinarily powerful.

Consider the figures on its computational power: at peak order volumes, it can execute 2.9 billion route-planning calculations per hour, computing the optimal match between orders, riders, and routes within tens of milliseconds while simultaneously factoring in as many as dozens of variables — weather, road conditions, the number of riders, restaurant preparation speed, and more.

According to Meituan insiders, the “Real-time Intelligent Dispatch System” not only draws on real-time variables such as the locations of riders, consumers, and restaurants, as well as weather conditions, but also uses historical data from previous deliveries to adjust delivery-time targets.

When a delivery rider goes against traffic or runs a red light on a particular route for fear of exceeding the time limit, the time saved at such life-threatening risk is absorbed by the algorithm and becomes the new time standard. And so it cycles, until the delivery time for every single route has been compressed to the absolute limit.

The rise in efficiency, then, is not simply because the system can “compute” better; it is because the system can monitor and control countless delivery riders, absorbing their desperate, all-out ‘calculations’ into its own.

Yet the price of that efficiency is the riders’ safety.

According to a 2023 survey, during peak delivery hours, 53 riders passed through a single junction in Beijing within half an hour, of whom as many as 37 were travelling against traffic, cutting diagonally across roads, or running red lights. Shanghai has also published a set of figures: in the first half of 2017, one delivery rider was injured or killed on average every 2.5 days. In 2023, in one particular week, Meituan delivery riders in Shanghai committed as many as 6,500 traffic violations.

I have experienced such a moment myself: one rainy night, worried about exceeding the time limit, I slammed hard into a median barrier in the middle of the road. My electric bicycle was wrecked, but the order would not wait — all I could do was finish the delivery on a shared bicycle.

Afterwards, I used to imagine that — even giving the platform the most generous benefit of the doubt — it could at least allow delivery riders to upload information about a traffic accident to the system immediately, granting the injured rider some extra time to complete the delivery. I, at least, would not have been left in such a sorry state, and a rider who was hurt would have had breathing room to deal with the aftermath.

The platform has already poured substantial resources into optimising the system, cutting delivery times, and turning a profit for the company. Is it really so hard to leave a “back door” open for riders who meet with an accident? Is the inability to design a perfect system truly due to a lack of technical capability, or is it the designers’ own conception of power at work?

Public debate about rider accidents has been going on for several years now, yet there has still been no visible improvement in the system. Whether the food-delivery platforms are genuinely willing to act, the answer speaks for itself.

We can never truly know whether “technical reasons” is nothing more than a pretext for refusing to change — after all, the initiative to set algorithmic rules rests firmly with the companies, while workers have never been permitted a say in decision-making.