Multi-pedestrian detection

[J Alynn]

In light of the tragedy in New Orleans I think a study into using the front camera connection on the vehicles for detecting pedestrians/animals (deer), small objects (dropped boxes from a pick up)- over a certain number of objects 5+ when over a speed of like 10-15mph the vehicle just cuts power.

Rock crawling and similar activities that have spotters generally aren’t exceeding these speeds. I’m sure some won’t agree with this but imagine in time if the technology exists in all vehicles. Then it should also be set that if an owner disables that feature the VIN automatically gets reported back to SM for record keeping. I’m sure criminally minded people would find work arounds but in time if all vehicles had that tech it could help prevent a tragedy like this.
With OTA and theft protection, etc… they can find any of us at any time anyway as well as shut down a vehicle remotely. So imagine theft warning is triggered and shortly after the safety feature is disabled-they (SM) could immediately shut down the vehicle and track down its location in real time ???

 

[SpaceEVDriver]

I'm a remote sensing scientist. It's *incredibly* difficult to reliably detect humans. They don't show up in radar. Lidar is reliable, but it's very expensive. Sonar is reliable, but it's very, very short range. Cameras are great, but there's so much noise in the 3D products that it's actually quite difficult to pull the person out of the background quickly enough with the processing power we can put into a vehicle. This will be even more difficult in a place where a spotter might be active--all the rounded rocks, boulders, trees, etc., are very difficult to distinguish from humans. The rate of both false positives and false negatives is very high. And the risks associated with responding to the false positives can be just as catastrophic as not responding to the true positives.

This is a huge area of active research.

In my opinion, lidar and cameras combined with a much more powerful computing system than is typically in vehicles is the likely best answer, especially if we can get multiple wavelengths in both the Lidar and the cameras (visible and something in the mid-IR range). But that's going to add a lot of expense to the vehicle and we already have people wanting *no* driver aids because of the expected expense (and other concerns) of the Scouts.

Boxes, etc., are also nearly impossible to detect. Again, for the same reasons.

 

[J Alynn]

Makes sense. I knew LIDAR was the likely direction. I just figured with cell phones now being able to grab faces quickly when taking pics that maybe the technology had evolved enough. Thanks for your insight!

 

[SpaceEVDriver]

Makes sense. I knew LIDAR was the likely direction. I just figured with cell phones now being able to grab faces quickly when taking pics that maybe the technology had evolved enough. Thanks for your insight!

Yeah, finding faces when they're expected is one thing.
But finding people where they're not expected is entirely different.

The software is progressing. But the question is: Does Scout include Lidar at extra expense with a hope that the software will progress enough that it's a usable option at or very shortly after launch? If they don't plan today to include Lidar on the vehicles, then updates as the software progresses are pretty much impossible. But if they include Lidar and the detection capabilities don't come quickly enough, we may be paying for unused hardware.

 

[eddiet1212]

In light of the tragedy in New Orleans I think a study into using the front camera connection on the vehicles for detecting pedestrians/animals (deer), small objects (dropped boxes from a pick up)- over a certain number of objects 5+ when over a speed of like 10-15mph the vehicle just cuts power.

Rock crawling and similar activities that have spotters generally aren’t exceeding these speeds. I’m sure some won’t agree with this but imagine in time if the technology exists in all vehicles. Then it should also be set that if an owner disables that feature the VIN automatically gets reported back to SM for record keeping. I’m sure criminally minded people would find work arounds but in time if all vehicles had that tech it could help prevent a tragedy like this.
With OTA and theft protection, etc… they can find any of us at any time anyway as well as shut down a vehicle remotely. So imagine theft warning is triggered and shortly after the safety feature is disabled-they (SM) could immediately shut down the vehicle and track down its location in real time ???

A good thought and maybe technology will be utilized to assist in creating a safer environment for pedestrians.

Unfortunately, persons intent on comitting criminal acts will always be the first to defeat technology in furtherance of their activity.

 

[J Alynn]

A good thought and maybe technology will be utilized to assist in creating a safer environment for pedestrians.

Unfortunately, persons intent on comitting criminal acts will always be the first to defeat technology in furtherance of their activity.

Like the old saying
“Door locks just keep honest people out”

 

[Latrant]

In light of the tragedy in New Orleans I think a study into using the front camera connection on the vehicles for detecting pedestrians/animals (deer), small objects (dropped boxes from a pick up)- over a certain number of objects 5+ when over a speed of like 10-15mph the vehicle just cuts power.

Rock crawling and similar activities that have spotters generally aren’t exceeding these speeds. I’m sure some won’t agree with this but imagine in time if the technology exists in all vehicles. Then it should also be set that if an owner disables that feature the VIN automatically gets reported back to SM for record keeping. I’m sure criminally minded people would find work arounds but in time if all vehicles had that tech it could help prevent a tragedy like this.
With OTA and theft protection, etc… they can find any of us at any time anyway as well as shut down a vehicle remotely. So imagine theft warning is triggered and shortly after the safety feature is disabled-they (SM) could immediately shut down the vehicle and track down its location in real time ???

Stolen may not matter since the two vehicles used (one in New Orleans and one in Las Vegas) appear to have been rented on Turo.

Also, I thought the current version of FSD identifies pedestrians, though unreliably. At least that’s what I’ve seen on YouTube.

What does Waimo or Cruise use to ID pedestrians (before dragging them for a mile as occurred in one incident after striking a pedestrian)?

With rain or snow I’ve always thought relying only on cameras would not work well. Then again, our Audi Q5 PHEV looses/shuts down all cruise control if it’s radar is blocked by snow or cameras can’t see because of heavy rain more than snow.

 

[SpaceEVDriver]

Also, I thought the current version of FSD identifies pedestrians, though unreliably. At least that’s what I’ve seen on YouTube.

That's the issue. It's possible but unreliable. Brand-T uses cameras only--they even removed sonar from their newer vehicles. And when the visual space gets busy, reliability drops. Cameras-only also fail for other situations. For example, a cameras-only approach means that something with low rear taillights (a motorcycle, for example) seen in the dark appears farther from the car than it truly is because of the assumptions built into the stereo vision processing algorithm--and it's nearly impossible to make this work for the majority of the time while also working for these other times. Adding more cameras can help mitigate this, but exponentially increases the processing time required. That's also true for smaller objects (a child vs an adult for example), if other visual cues aren't accounted for or available.

What does Waimo or Cruise use to ID pedestrians (before dragging them for a mile as occurred in one incident after striking a pedestrian)?

Waymo uses Lidar (the giant, spinning hardware on the roof) as well as cameras and sonar. But that all can fail in busy environments. The pedestrian in question was dragged only a short distance, but once it hit her, it could no longer sense her. The only reason it was stopped was the intervention of bystanders.

With rain or snow I’ve always thought relying only on cameras would not work well. Then again, our Audi Q5 PHEV looses/shuts down all cruise control if it’s radar is blocked by snow or cameras can’t see because of heavy rain more than snow.

Cameras are unreliable in adverse weather conditions, yes.


Don't get me wrong: Many of these are surmountable problems and we should continue working on them. We just need to be realistic about their current capabilities and we need manufacturers to be realistic about what's possible. I don't walk across the street in front of a self-driving vehicle, and I usually avoid crossing in front of a brand-T unless I make eye contact with the driver.

 

[Latrant]

That's the issue. It's possible but unreliable. Brand-T uses cameras only--they even removed sonar from their newer vehicles. And when the visual space gets busy, reliability drops. Cameras-only also fail for other situations. For example, a cameras-only approach means that something with low rear taillights (a motorcycle, for example) seen in the dark appears farther from the car than it truly is because of the assumptions built into the stereo vision processing algorithm--and it's nearly impossible to make this work for the majority of the time while also working for these other times. Adding more cameras can help mitigate this, but exponentially increases the processing time required. That's also true for smaller objects (a child vs an adult for example), if other visual cues aren't accounted for or available.



Waymo uses Lidar (the giant, spinning hardware on the roof) as well as cameras and sonar. But that all can fail in busy environments. The pedestrian in question was dragged only a short distance, but once it hit her, it could no longer sense her. The only reason it was stopped was the intervention of bystanders.


Cameras are unreliable in adverse weather conditions, yes.


Don't get me wrong: Many of these are surmountable problems and we should continue working on them. We just need to be realistic about their current capabilities and we need manufacturers to be realistic about what's possible. I don't walk across the street in front of a self-driving vehicle, and I usually avoid crossing in front of a brand-T unless I make eye contact with the driver.

I would imagine Scout, if LIDAR or RADAR, or SONAR are included will shut these systems off when in ‘Off-Road’ mode, But keep cameras on for fun, recording fun, and visibility for obstacles?

Otherwise they mostly work well. I do have my parking SONAR on my 2018 A5 cab go off unpredictably when in drive and stopped in traffic. It never ‘brakes’ but still weird when it happens. Car is getting old and probably will be replaced by cabana Traveler !

 

[SpaceEVDriver]

I would imagine Scout, if LIDAR or RADAR, or SONAR are included will shut these systems off when in ‘Off-Road’ mode, But keep cameras on for fun, recording fun, and visibility for obstacles?

I hope it's an option to turn them off. My guess is that it'll depend on what they define as "Off-Road" and whether it's different from "Crawl" or something similar. I'd like the sensors and warnings to remain on if I'm on a forest service or fire road, and maybe even some lighter trails. But I wouldn't want them on if I'm crawling up the mountain.