During the 2020 harvest season in Russia, a mix of more than 350 New Holland, John Deere, Claas and Russian-built combines were outfitted with Cognitive Agro Pilot autonomous driving systems. The equipped machines combined — that is, cut, threshed, and cleaned a grain crop in one operation — about 400,000 acres, harvesting more than 720,000 tons of crop in the process. The system does not use GLONASS or GPS, instead relying on a video camera and artificial intelligence (AI).
The solution works autonomously without connection to GPS or satellites and does not require high-precision positioning such as RTK corrections. The system analyzes images from one video camera mounted on the machinery and employs a deep learning convolutional neural network fine-tuned for agronomic purposes. It automatically detects unharvested areas and plots a route accordingly.
The fleet harvested about 320,000 acres of wheat, soybeans, barley, oats, sorghum and buckwheat, reaping 590,000 tons of grains and oilseeds. Row crops made up the rest of the autonomous Russian harvest.
Cognitive Agro Pilot is developed jointly by two Moscow companies, Sberbank and Cognitive Pilot. The product is an autonomous AI-based) driving system for farming equipment such as combines, tractors and sprayers.
According to the company, this produced savings of $6.5 million dollars, from fewer machine hours at harvest, equipment depreciation, fewer human errors causing fewer accidents and inefficiencies. There was a 15 percent reduction in direct crop losses and five percent reduction in fuel consumption when compared to regular operations. Owners of the 350 combines were some of Russia’s largest agricultural holdings.
“A combine harvester equipped with Cognitive Agro Pilot starts seeing and understanding its surroundings and making decisions about further actions,” said Olga Uskova of Cognitive Pilot. “In essence, the computer understands the types and positions of objects in front of the machine. It builds movement trajectories and sends commands to perform maneuvers.”
“This is a system that ‘sees’ and ‘understands’ what’s lying ahead. This is an advantage over systems that utilize a raft of tools for their control models. They need GPS navigation with RTK positioning. The edge capture when controlling a combine with Agro Pilot is stable at 10 centimetres, which lets you avoid unnecessary passes and fuel losses.”
The Agro Pilot video camera is positioned on the combine cab, high on the left facing the header. A second camera, while not expensive in itself, would drive up the computational load, and the general processing unit (GPU) employed by the system accounts for a major share of the hardware costs. The onboard computing unit
featuring pre-installed software contains a compact, high-performance Nvidia Jetson TX2 CPU for data processing and analysis, an STM32 microcontroller unit, and a set of interfaces through which the system interacts with the user and machinery operating controls.
A FLIR Blackfly video camera provides a continuously updated view of the surroundings while the combine is in motion and has enough settings for automatic exposure control if lighting conditions on the field change.
Uskova said keeping the combine running right at the crop edge is the only way to increase its production. Swerving one way or the other off the edge leaves crop un-harvested or it means combining on ground already harvested. Either way, it lowers productivity.
She pointed out that combine operators in Russia often choose speed over efficiency because they are paid per ton of the crop in the tank. They are not paid to ensure the highest possible crop yield. “Since Agro Pilot now controls the driving mode, our robot safeguards drivers from mistakes.”
