Autonomous Tractors, Excavators and More: Examining the Path Toward a Driverless Tomorrow

By Jeff Jurgens, AEM Director of Product Stewardship

Autonomous equipment offers significant benefits with respect to efficiency, cost savings and safety. These machines are poised to have a profound impact in the off-road environment, upsetting and defying current models in relation to the size of the equipment and the role of the dealers and OEMs, as well as impacting the individual operator.

Although the equipment sectors that AEM represents have historically been hesitant to adopt innovative technologies, they are set for a significant amount of change, as autonomous vehicles (AV) dramatically alter the nature of the work. Future off-road autonomous equipment will handle the cycle of operation and cyclical tasks, while teleguided machines are maneuvered by operators, sometimes in simulators thousands of miles away.

History of Autonomous Equipment

The first autonomous testing began in Japan in 1990. However, the process actually began with the release of the Field Management Software (FMS) for mines in the late 1980s, and it became the foundation for Komatsu's autonomous platform. The first commercial deployment was in Chile in 2007. Just a year later, in 2008, another system was launched at a Western Australia iron ore mining operation. From a remote operations center in Perth, Western Australia, workers operate autonomous mining vehicles at mines more than 1,200 kilometers away. Finally, in 2013, Komatsu introduced the world's first dozer with fully automatic blade control.

The lessons learned from those applications eventually led to the first North American deployment in 2016. By 2018, Komatsu had moved over 2 billion tons, with 130 fully autonomous mining trucks running across seven operational sites (six of which are fully autonomous sites). As of June 2020, those numbers have jumped to 3 billion tons moved by 251 trucks, and the fleet still currently maintains a perfect safety record with no incidents.

As another example of the exponential rate this industry is developing, Caterpillar has doubled the amount hauled in the 16 months since reaching 1 billion tons hauled in November of 2018. To date, Caterpillar has 276 autonomous trucks in operation have driven over 42 million miles without a lost-time injury.

The Current Status of Autonomous Equipment

Autonomous vehicles (AVs) currently hold a market share of less than 1% of total construction equipment sales worldwide. However, given global equipment sales are projected to grow 2.5% each year until 2022, coupled with the exponential growth in adoption of AVs, the market share is expected to double in size in the next two years.

Automated shuttle buses are in use in many countries. As mentioned above, autonomous heavy haulers are already in use in mining in several countries, with automated agricultural and forestry equipment currently being developed as well. While a blend of GPS and other location tracking sensors, image sensors and telematics assist John Deere vehicles to navigate fields today, the company still can't truly replicate everything a human would see and feel sitting in the tractor cab. The determining factors here are farmer perception and added value. This will remain the case until more tasks, and not just movement, become automated.

Meanwhile, the forestry sector has seen limited progress with automation, largely due to working conditions being over considerably larger areas than those experienced by the other sectors. Work sites are often in remote areas, on steep slopes, and in very rugged and variable terrain. In these remote areas, there are generally no cell towers and no Wi-Fi, and communication is currently expensive with limited bandwidth. The next generation of very large fleets of low earth orbit (LEO) satellites, however, is expected to address this problem.

Designed in partnership with Swedish engineering company Sandvik, the Syama underground gold mine became the world's first fully-autonomous mine operation. It operates with fully automated trucks, loaders and drills. The fully autonomous operation means that the mine can operate 24 hours a day, with all operations overseen from a remote operation center.

Some companies are applying self-driving construction machinery to do repetitive tasks, such as pouring concrete, bricklaying, welding and even demolition. Excavation and other prep work is also being completed by autonomous or semi-autonomous bulldozers, which can ready a jobsite with the help of a human programmer to exact specifications. One company, which works in heavy civil construction and grading, used Built Robotics-equipped dozers to finish housing pads in a residential development.

'Built's autonomous dozer is the real deal, and we've had it finish-grade hundreds of pads so far,' said Eric McCosker, project manager at Independent Construction. 'Given the shortage of skilled hands in our area, that's added production we've needed.'

Nuts And Bolts

The challenges for autonomous off-road equipment vary significantly from passenger vehicles. In a farm field or construction site, lane markers or curbs are not present to assist with vision, nor is there a LiDAR system to reference to sustain the vehicle on a required track. Moreover, automobiles do not have projections like booms and buckets, nor can most rotate on their undercarriage like a shovel. Systems for an off-road machine need to know exactly where they are, their bearing and where their attachments are located.

Precision excavation is just what the name implies. First, the machine needs to know where it is and its orientation. In North America, we are most familiar with global positioning system (GPS) technology. As you have seen firsthand, if you have used a smartphone navigation app, regular GPS is only accurate to about 6-10 meters, and it cannot provide heading information unless the vehicle is in motion. These limitations are addressed using either Differential GPS (DGPS) or Real Time Kinematic (RTK) GPS. DGPS and RTK both require a local stationary GPS receiver, for which its location is precisely known. The stationary receiver compares its known location to its location as calculated by the GPS receiver.

From this information, it can generate corrections, which it transmits to the other mobile GPS receivers located on the vehicle, resulting in a much more accurate solution around 10 to 30 centimenters. The heading limitation of GPS is overcome by using two GPS receivers on the machine, with antennas separated in space by some reasonable distance (1 to 2 meters). The absolute position of receiver 1 to receiver 2 antennae is used to calculate the absolute heading. To supplement GPS and RTK, an Inertial Measurement Unit (IMU) is installed in the vehicle, usually centrally located to the vehicle. An inertial measurement unit measures 3 dimensions of linear acceleration (3D Accelerometer) and 3 dimensions of rotational rate (3D Gyro), and sometimes also a 3D magnetic field. In highly controlled work zones, the primary decision-making software and algorithms will reside in a centralized control facility and will probably include real-time human monitoring and oversight of site operation. The autonomous trucks operate with such repeatable precision that the engineers are now incorporating 'vectoring,' where an algorithm shifts the haulage route laterally a meter or so to keep the trucks from wearing ruts in the haul road.

In comparison to the applications in mining, agricultural robots are often small or mid-sized and are designed to autonomously navigate and automatically take some precise plant-specific action. Machine vision technology is often a core competency of these robots, enabling the robots to see, identify, localize, and to take some intelligent site-specific action on individual plants. The machine vision increasingly uses deep learning algorithms often trained on expert-annotated image datasets, allowing the technology to far exceed the performance of conventional algorithms, as well as to match or even exceed that of expert agronomists.

For instance, Continental is developing concepts for robots which can be fitted modularly for agricultural purposes, operates autonomously and can be used with the appropriate attachments to perform numerous tasks, such as monitoring the growth process of plants, precise weeding, or systematically sowing seeds and delivering nutrients. Camera-based, multispectral, or thermographic sensors are particularly important for monitoring plant growth or soil conditions.

Many robots have already grown in capability, offering faster speeds, higher frame-per-seconds, more ruggedized designs, higher on-board energy for longer operation time and a heavier load, and so on. Simple robotic implements utilizing basic row-following vision technology, are already mature and not uncommon in organic farms. Advances in vision technology are transforming tractor-pulled implements, upgrading them into intelligent computerized tools able to take plant-specific precise action.

This approach does not focus on autonomy, although the tractor itself can readily be made autonomous to render the entire system automatic if necessary. This system is designed to become competitive in large farms, which demand high productivity, which in turn is linked to technology parameters such as fps (frame per second), false positives, sprayer controller speed and so on. In the future, the system costs will likely fall, particularly if lighter versions of the algorithms on the inference side become available to render GPU processors unessential without a major performance sacrifice.

Although a rich assortment of technologies is available to AV designers-radar, GPS, GNSS, LiDAR, ultrasonic, infrared, and video, a confounding problem that inhibits off-road equipment from an easy path to autonomy is dust, dirt and other weather conditions. Not only does weather change how the machine should act in its environment, but it also reduces the accuracy of the sensors. One solution might come in redundancy-more sensors that could be used as backup.

Benefits - Addressing Workforce Needs

Moving toward autonomous solutions may help ease the operator shortage.

'Finding experienced talent in application areas is often a challenge,' said Fred Rio, construction digital and technology product manager, Caterpillar. 'Autonomous applications can provide employment opportunities to a new workforce for those with physical disabilities and wounded veterans, creating a value-added job for someone who wants to work but may face challenges in physically operating a machine. Technology and automation make the role of equipment operator more widely accessible.

'All this technology, bit data and analytics will actually increase the number of jobs in more analytical work, it is a change from trade jobs to more analytical ones.'

Senior Lecturer with Curtin University's School of the Mines Dr. Carla Boehl said students are being proactive in learning about then new technology.

'The students themselves are interested, they want to do their thesis in this field and learn more about automation,' she said. 'At a post graduate level, we are starting to do more work regarding maintenance, automation, we do a lot of big data understanding, what is big data, what algorithms can be used to support systems.'

Benefits - Productivity

'Using the machine autonomously improves productivity,' said Erol Ahmed, director of communications for Built Robotics. 'The machine uses less fuel and moves more efficiently, which prolongs the life of the machine, reduces maintenance and prevents unnecessary wear and tear. On average, a robotic piece of equipment from Built runs at the same speed and responsiveness as one being used manually by a person.'

By automating some tasks, skilled workers can work on more complex tasks or move to areas where they are needed most. Equipment automation also allows jobsites to run beyond normal operating hours (e.g., at night) and perform tasks in parallel so they can be done more quickly.

Autonomous machines can alter the economics of machine design, facilitating the increase of smaller machines. Indeed, this elimination of the driver overhead per vehicle is the basis of the swarm concept. There is clearly a large productivity gap today between current large and high-power vehicles and those composed of fleets of small robots. This productivity gap, however, can narrow, as the latter has substantial room for improvement.

'When we develop robots, people always say they should increase yields or lower costs, but that's not always the case,' said Eldert van Henten, a professor in Wageningen University's Farm Technology Group. 'Robots that offer the farmer time to devote attention to business and management also have economic value.'

Benefits - Reduced Costs

Construction and mining technology investor Foundamental reports an average increase in mine productivity of 30% with AV's via:

• Longer production hours
• Reduced load and unit cost of 15%
• Improved tire life of 40%, since optimized controls reduce sudden acceleration and abrupt steering.

While autonomous hauling system implementation would cost on average $13 million at a site with 10 trucks and 15 ancillary vehicles, the total benefits are calculated to be $19 million. This indicates that investment into autonomous technology ultimately pays off.

The first major agriculture target market is in weeding. Precision action reduces consumption of agrochemicals by 90% compared to untargeted application. It also improves yield (e.g., by 5-10%) because collateral damage of the crops by untargeted chemical application can be minimized. This technology can further enable farmers to tackle herbicide-resistant weeds, which are a growing problem. Finally, the robots leave behind no unusable compacted soil.

Benefits - Safety

As noted above, in the 13 years of autonomous equipment operation, between Komatsu and Caterpillar, they have moved over 5 billion tons of material, and Caterpillar alone has logged over 42 million miles with zero hauler-related incidents or lost time injuries. In contrast, a recent Department of Mines study analyzed the deaths of 52 miners over the past 12 years, finding worker fatigue and inexperience with mining risks to be the biggest cause of accidents. In a report that CAVCOE co-authored, they predicted that with full deployment of autonomous and connected vehicles, 80 percent of traffic collisions, fatalities and injuries could be eliminated. Autonomous machines can perform remote or difficult work, which helps keep people out of harm's way.

To date, most large autonomous machines are operated in restricted facilities where proximity to the automated equipment is controlled and extremely limited. Sensor technology and processing capabilities need to advance further to be able to identify many of the potential obstacles that must be avoided. Safety of autonomous systems is of utmost importance.

Obstructions

The market is certainly there. The productivity gains are meaningful. Cost savings are realized. So, one might wonder, is there a catch? Traditionally, construction has had a complicated relationship with technology, but the shortage of skilled labor coupled with the surging demand for building projects has forced the industry to re-evaluate that relationship to make up the shortfall.

Possible Futures

In the next five years we can expect the trend toward automation and electrification to accelerate from low levels to noticeable levels. Within 10 to 15 years, based on exponential trends, it is estimated that more than 80 percent of heavy equipment machinery will be powered by electric powertrains and have systems centric to autonomous operation. A few heavy equipment end-users have already expressed a desire for fully autonomous work sites and electric equipment, as this helps them to lower overall costs and to meet efficiency and environmental targets. The indications are that it is a no-brainer for heavy equipment operations to transition to fully autonomous and electric operations at the earliest commercially viable opportunity.

For ag, the key to commercial success lies in the development of robust robotic and associated artificial intelligence (AI) platforms which can be utilized across the harvesting season of different crops. This approach is already reflected in the latest prototypes or products on the market.

Autonomous haulage systems (AHS) will have a pronounced impact on future mining operations. Driverless trucks run continuously and more efficiently than their manned counterparts. AHS will improve everything associated with the mining operation, from safety to operations to maintenance. They will also change the approach to mine planning and scheduling. The 8-hour production shift, for example, was based on miner fatigue levels. Production shifts in the future might be based on the refueling needs of the excavator. Or, in fact, they might disappear entirely.

In the next five years, given the current state of AV technology in general, it is difficult to envisage any significant changes in this time frame. A construction project, for example, frequently involves operation close to existing structures, workers on the ground or an environment where there may be material delivery and site visits from 'civilians' who do not have the proper training to safely navigate a site shared with autonomous equipment. The current and near future of AV technology is not really designed for these use cases.

Looking 5-10 years ahead, AV technology will start to be introduced for certain tasks, such as automated snowplows on long lanes, in agriculture, and in the other sectors for moving a large number of items (i.e. timber, pallets, bales, etc.) from point A to point B, especially those that travel a significant distance.

By about 2030, it is expected that AI will have reached a point where its capability exceeds that of a human for many tasks, including operating off-road equipment. The technology will allow most, if not all, of the applications to be automated. This will likely be achieved by a significant redesign of machines (cabless), rather than by a humanoid robot operating the current generation of vehicles.

Since AVs are very suited to electrical propulsion, the other major trend that AEM members should be aware of is the move away from traditional fossil fuel propulsion to battery electric vehicles. Although, fuel cells and hydrogen power are a possible alternative.

The Bottom Line

The world is progressing in the direction of automation in its many different configurations. The innovations in sensors, artificial intelligence and connectivity, coupled with the productivity increases and other benefits, will transform the equipment industry. The outcome will not be the abrupt influx of automated heavy equipment in all sectors of the market. As an alternative, there will be incremental advances, such as driver assistance systems, and certain applications within each market will be automated first simply because they are easier to automate.

The total deployed number of units is minor compared to the conceivable market. However, the technical viability is long proven. Notably, there is abundant room to increase productivity and to expand applicability by making constant gains within established technologies. As such, it is more a question of timing and not of technical viability.

The second generation of products have already matured compared to the first, in terms of design, size, AI, speed, and so on. The iterations will continue. Now, the task at hand is to be open to new technologies and -- perhaps more importantly -- see it as an added value, not a competition.

Subscribe to our AEM newsletters for more perspectives from AEM staff.

Public link

Please use the above public link if you want to share this noodl on another website.