Nodar announced at CES that it is moving to accelerate its market push for its advanced stereo-vision software for autonomous vehicles (AVs) with its newly released Hammerhead devkit (development kit). The combination of software, hardware, and tools will help autonomous systems developers leverage the benefits of the company’s patented Hammerhead untethered stereo-vision software platform that uses images from multiple cameras to produce high-fidelity 3D point clouds in real-time.
The Hammerhead is engineered for forward collision warning, path planning, automatic emergency braking, traffic jam assist, VRU detection, and parking valet. It is capable of detecting objects as small as 10 cm at 150 m away as well as objects at a distance of up to 1250 m.
The devkit also enables new partners in the areas of last-mile delivery vehicles, trains, boats, air taxis, drones, and commercial and agricultural vehicles to rapidly assess and integrate its 3D vision technology that already supports automotive markets with precise and long-range camera-based sensing. Nodar believes that autonomous construction cranes, farming equipment, and air taxis can benefit from Hammerhead to perceive their environment in real-time with range, accuracy, and precision. The technology’s proprietary online calibration system delivers resilience in the extreme conditions of off-road, commercial, and in-air autonomous system applications.
“Every autonomous system must have the ability to ‘see’ and sense its environment in order to avoid obstacles and do path planning,” said Nodar CEO Leaf Jiang. “In the automotive industry, 3D vision is the cornerstone of developing safer autonomous vehicles. Recently, we’ve been hearing increasingly about the automation of robotic systems beyond the automotive market, and that’s very exciting for us. The devkit we are releasing makes Nodar technology accessible across a wide variety of markets and applications of autonomy, offering the reliability of active sensing systems at the price-point of cameras.”
Loving and leaving lidar
Founded in 2018 and backed by New Enterprise Associates and Rhapsody Venture Partners, Nodar has been developing the Hammerhead vision platform to deliver reliable, ultra-precise, and real-time 3D sensing at an ultra-long-range, providing better-than-LiDAR 3D data at the price point of camera technology. That comparison to lidar is a bit ironic, as company cofounder Jiang went to MIT, studied electrical engineering and computer science, and worked at the MIT Lincoln Labs for 12 years on LIDAR systems. The technology’s economics were a challenge then as now.
“I was like, this is a $10 million lidar and we’ve gotta make it for a hundred dollars,” he said. “Hmm, let’s do the math. I think it’s gonna be tough.”
While LIDAR was his “first love” and “it’s great for a lot of things,” it turns out that he felt the time was right for doing long baseline stereo cameras.
“You’ve got a big car, take advantage of the width of it, and you can do ultra-precise measurements,” he said.
Joining Jiang to cofound Nodar was COO Brad Rosen, who sold a company we had been running and was working with a VC looking for something to do in the AV sector.
“I am enamored with it personally—the notion of autonomous vehicles,” said Rosen. “I was particularly interested in aero, like air taxis, UAVs, and drones. But I felt, fundamentally, that the air taxi is coming, but it’s a ways off.”
So the two started looking at passenger vehicle autonomy with a focus on stereo vision.
New autonomous market opportunities
According to Rosen, Nodar’s emphasis for several years has been on automotive, but the launch of the devkit is giving the company’s offerings traction from alpha customers in trucking, agriculture, construction, air taxis, and last-mile delivery. He says that each one of these spaces has the need for the company’s kind of robust, low-cost, reliable depth sensing.
Macroeconomic forces are also driving change. The $270 billion U.S. commercial trucking market is suffering from an 80,000 driver shortage, due in part to the pandemic and substantially to the growth of e-commerce in general. Yet the cost of the driver constitutes about 60% of the overall cost to run a vehicle. While human drivers cannot legally drive more than 8 hours a day, autonomous trucks can be run nearly 24 h a day, improving delivery times and capacity. Further, trucking accidents account for nearly 5000 fatalities a year in the U.S.—a number that is expected to decline dramatically as autonomous functionality comes online.
Along with trucking, the construction industry will see increased automation, particularly with cranes, which require highly trained operators who must manage a swinging hook and move dangerous materials for hours at a time. In the boating market, autonomous systems are being developed with sensors that must accommodate constant motion and a shifting horizon. Commuter trains need ultra-long-range sensing to detect potential hazards far ahead, giving the train time to stop. Last-mile delivery robots, another new market, require precise yet low-cost and low-compute 3D vision to navigate roads and sidewalks safely as they bring packages and food to consumers’ doors.
As with autonomous vehicles and trucks, Nodar’s platform benefits all of these new markets with its support for independently mounted cameras and wide baseline configurations, offering camera-based resolution, a small compute footprint and the sensing reliability of lidar. Compared with other sensor systems, the Hammerhead solution offers up to 10 times the range while free of their inherent limitations, such as the dependency on AI and neural networks used in monocular camera solutions, and the scanning, low resolution, and high cost of lidar.
Innovating with stereo vision
Many passenger-car OEMs are still using single (or monocular) camera systems, but a few more forward-looking automakers are on the same stereo-vision path as Nodar.
The highest-profile two-camera system currently on the market is Subaru’s Eyesight, initially supplied by Hitachi in 2015 but now from Veoneer, shipping millions of units since its debut.
Nio’s watchtower sensor design layout on the ET7 includes prominently displayed forward-facing stereo camera bumps in addition to lidar to enable better visibility and improved safety in complex road conditions.
Another important stereo-camera application is from Mercedes-Benz for its Level 3 EQS and S-Class. The Drive Pilot system uses Veoneer’s 4th generation stereo vision camera system in addition to the supplier’s advanced 77-GHz radars using super-pulse modulation radar combined with perception software.
However, Nodar is looking to take stereo vision tech to the next level.
“We’ve set out to build something more accurate and more reliable, like a lidar, but higher performing, higher definition, faster, and much cheaper,” said Rosen.
The benefits of stereo vision are crucial for safety-critical ADAS and AV systems, which require a true measurement of the distance to everything around vehicles.
“That’s why people love lidar, time of flight, it’s like a ruler,” said Jiang.
However, there are several variables that can be measured beyond the time of flight.
“For us, it’s the angles, triangulation,” said Jiang. “You measure two angles from two cameras. If you measure those angles, you’ll very precisely know where something is very reliably.”
While stereo vision has been around for a while, what’s new here from Nodar, according to Jiang, is the ability to do long baseline stereo vision on a car that shakes like crazy.
“The thing that you’re fundamentally trying to measure, the angle, is something that’s so fragile that a hundredth of a degree will ruin your measurement,” he said. “Mechanically, you can’t really solve that. That’s why the older systems are typically short baseline on a very rigid structure, and they mechanically engineer the heck out of it so that it doesn’t move. But then you can’t see very far.”
The range is proportional to the distance between the cameras. That’s why Nodar is pushing the wide-as-possible hammerhead shark configuration for depth perception, and its special sauce is software.
“We’re a software company that has allowed these independently mounted cameras to move around and essentially calibrate them in real-time,” said Jiang.
He says that Nodar has innovated in three key areas—working with natural scenes; because cars vibrate like crazy, working with every frame; and being “exquisitely accurate” because the company’s tech focuses out to more than 1000 m. The company’s first three patents cover these aspects of stereo-vision technology.
Getting closer to production
Nodar’s technology is not on production vehicles yet, being an early-stage company that just raised its Series A. But company execs expect to be designed into a pre-production system soon.
“We want to be on every car,” said Jiang.
“Hopefully we’ve got more than one design-in secured in a year or two,” added Rosen.
To reach its goals, the company has built reference designs for customers, showing a demo at CES. The reference design ships with two cameras and Nodar’s software preloaded onto an Nvidia Orin automotive SOC (system-on-chip) processor. It’s engineered to be low-power, easy to set up, and compatible with many sensors and processors.
“We think it’s important to have an approach that’s flexible,” said Jiang. “It’s all about how do we get this on a hundred million vehicles for everybody? Because that’s how you really achieve true safety is that everyone has access to it.”
Part of that flexible process is being able to have multiple vendors for cameras and processors. It can be used with RGB, gray-scale, thermal, and shortwave IR cameras, with mixed optic types, and with a couple of processors popular for automotive applications.
Educating the market
After four years of POCs (proofs of concept) and pilots, the biggest challenge for Nodar is educating the market on its brand of stereo vision.
“We’re at the stage where we’re shipping development kits, we’re going through the certifications,” said Jiang. “Now the biggest challenge is probably to get over this learning curve of what’s possible with cameras. I think people just don’t realize that two cameras can generate point clouds.”
When company execs show customers the camera output, they ask what lidar was used and how long did it take to generate the point cloud.
They also have to overcome preconceived notions of camera range and resolution. Traditional stereo vision systems have had very short ranges on the order of 50 m and very low resolutions of around 1 MP, according to Rosen.
“We’re in a vastly different world with this next generation, and we have to educate the market,” said Rosen.
He says that Nodar’s innovations are shifting OEMs’ thinking on how they get to SAE Level 3. As new-generation processors like dedicated automotive GPUs—with more silicon real estate for AI and stereo vision for doing faster calculations—spool up, he says his company’s solution will get better and better.
“You can’t fall asleep in this business,” said Jiang. “Especially with computer vision and deep learning. You have to make your product better by a factor of two every year, like half the power or twice the pixels. Cameras are getting higher and higher resolutions, and the software’s going to need to keep up.”
