Nothing Runs Like a GPU-Powered, Fully Autonomous Deere

Image courtesy John Deere

We’re still a few years from having self-driving cars take to the roadways en masse. But later this year, farmers will be able to start running fully autonomous John Deere tractors in their fields, enabling them to monitor the tractor and its work from a distance. According to folks at the Illinois manufacturer, this is the future of farming.

Automation is nothing new for tractor manufacturers and farmers. John Deere, for example, has outfitted its tractors with GPS-enabled self-driving automation for many years. But until now, farmers have had to ride along in the cab of the tractor to ensure that everything is working correctly.

Today at CES 2022, John Deere unveiled its first fully autonomous tractor. Available for purchase later this year, the new tractor is designed to work 24 hours a day, rain or shine, and plow fields, plant seeds, or spray herbicide or fertilizer within an inch of accuracy, giving farmers the ability to step out of the tractor cab and tend to other matters, whether work or play.

According to Deere, the new tractor is a field exercise in applied big data and data science. For the past three years, the company has collected 50 million images of fields in various states, which it used to train the neural network models at the heart of its autonomous tractor.

On the vehicle itself (a modified 8R frame) are six stereo cameras, which gives the tractor a 360-degree view of the environment. The cameras capture different visual aspects of the autonomous tractor’s surroundings, including RGB-D spectrum, where red corresponds to trees, green corresponds to ground, blue is the sky, and D stands depth, while anomalous objects are marked in yellow, according to Willy Pell, Deere’s vice president of autonomy and new ventures.

“Pixel by pixel, this machine is almost perfectly interpreting its surroundings,” Pell said during a hybrid press conference today. “The neural network classifies each pixel in about 100ms on an Nvidia GPU processor. Depending on whether an obstacle, like a tree branch or an animal, is detected, the machine either continues or it stops.”

Imagery captured from the navigational cameras atop John Deere’s autonomous tractor (Image courtesy John Deere)

Achieving high degrees of safety and productivity were the core challenge of this project, Pell said. For example, if a giant billboard falls in a field, the tractor will not be able to correctly identify what it is because there’s simply not enough training data of fallen billboards to train the model to accurately detect that particular situation. Instead, the tractor is designed to simply stop whenever it detects something that it doesn’t know.

“While the sparsity of relevant data is a disadvantage in agriculture autonomy, the ability for the tractor to stop is a huge advantage,” Pell said. “When we encounter an anomalous object, we stop. We don’t have to worry about being rear-ended by another driver. And the results is a 40,000 pound machine that is going to be running safely this year.”

During the press conference, Deere showcased the experiences of an early beta tester in the autonomous tractor program. Doug Nance, a fourth generation Minnesotan with about 2,000 acres of corn and soybean, never thought he’d ever see a tractor that could drive itself.

“For me it was really exciting the first time I got to take the autonomous tractor in the field,” Nance said. “Swipe my phone, watch the tractor start with no one in the cab, and start doing tillage. Come to the end of the field, turn around, and do tillage just as well as I can do myself with no one in the cab.”

The farmer is able to monitor the tractor’s progress via the John Deere Operations Center Mobile app running on a phone, tablet, or PC. “I can monitor the tractor, see how much of the field has gotten tilled. I can check the fuel level,” Nance said. “The app gives me all this information so I can monitor it very closely.”

Nance said the autonomous tractor will be a “life-changer” for him. “The thing that excited me the most about autonomy is not being locked in the tractor cab all day,” he said. “It will just allow me to run my business better because I can just pay closer attention to other tasks. Now I’ll be doing jobs that we always wanted to get done but never had time to because you were in the cab all the time.”

John Deere’s autonomous tractor can run unattended 24/7 (Image courtesy John Deere)

The data collected by the autonomous tractor will be advantageous in other ways, said Julian Sanches, Deere’s director of emerging technology.

“When tractor is running through field….it’s also collecting data about how well the job is being executed,” Sanchez said during the press conference. “So all of that performance and field data are continuously being sent to the John Deere Operation Center, so the farmer can keep tabs on how good the tractor is doing and how well the job is being done.”

Farmers will often sit down during the off-season to figure out what they did next year to come up with a strategy for improving their crop the following year. Now, with better data about how well the seeds are being planted as well as the rate of application of fertilizers and weed-killers, the data will combine to give farmers a better view on the state of their farm.

“These vehicles are essentially creating a digital footprint of the farm and documenting the decisions they make to grow better crops every year,” Sanchez said. “All of this is incredibly exciting. It’s not just about autonomy alone. The power of autonomy combined with automation that farmers have been taking advantage of for years makes this so transformative.”

With another 2.5 billion people expected to populate the earth by the year 2050, farmers will have to increase their per-acre production by 50% to ensure everyone has enough to eat, John Deere CTO Jahmy Hindman said. Technology will play a big role in achieving that extra output and efficiency, he said.

“If you visit a farm, you’ll see as much technology in the field as you do in Silicon Valley,” Hindman said during the press conference. “From robotics and sensors to AI and big data, a farm is a high-tech operation.”

Related Items:

A Virtual Ride Along with John Deere at CES

Digital Agriculture Program Yields Promising Prospects

Farmers Plant for Hyper-Local Forecasts with IBM’s ‘Deep Thunder’