Tesla Autopilot Nearly Recreates Fatal Crash

[Ocellaris]

What I don't get is how it made the decision that the correct action was to drive into a solid object.

I assume (perhaps incorrectly?) it was the driver who slammed on the brakes.

The edge face of barriers is quite small and the systems have trouble tracking them at a distance. They are far more suited to tracking large objects (cars) at the same approximate speed as the radar.

 

[velusip]

[Bug -- Open][Fatal Error][Reproducible] Car is strangely drawn to barrels and lane barriers.

Just another day as a software engineer.

 

[-Anomie-]

All ethical nonsense* aside, this sounds like a great theme for a game: GTA 6 - Autonomassacre!



*If you let your car drive you into a wall while you text, work, masturbate, whatever, don't blame the car. It's just proving how smart it is by removing your idiot genes from the gene pool.

 

[kju1]

You know what system of transportation has great autopilot? MASS TRANSIT.

We need more light rails and other similar options. Or sure, continue to figure out how to pack an ever growing population into one car each and have roads/infrastructure to support it.

*chuckle* You keep telling yourself that.

http://www.washingtonpost.com/wp-dyn/content/article/2009/06/22/AR2009062202508.html
https://en.wikipedia.org/wiki/December_2013_Spuyten_Duyvil_derailment
https://www.nbcbayarea.com/traffic/transit/SF_Muni_Train_Crash_Injures_Dozens.html

etc etc etc.

Also mass transit only works for two situations really:

1) Long haul point to point
2) In densely populated urban areas.

So remind me what people who dont live in the city are supposed to do? Walk the 20 miles to the nearest rail/airport/bus stop?

 

[Nanogrip]

Well, you wouldn't let a horse pull a cart by itself on a cobblestone road without some guidance over the horse, so why would you let an autopilot AI drive by itself at highway speeds?

 

[sfsuphysics]

This is the same crash where the guy who died previously complained that it happened 7-10 times already yet decided to go for an 11th try? Sounds like Tesla is helping weed out the gene pool of people who think we live in the future already.

 

[Khahhblaab]

The software needs to detect objects in the road, period.
It does not matter if the object is Crashed, un-crashed, pedestrian, pedestrian walking a bicycle, pedestrian walking a bicycle at night, pedestrian walking a bicycle at night while wearing all black, or in Tesla's case a fire truck or a 53' long white rectangle 4 feet off the ground.

So true. It will be horrible if the programmers at Tesla built expectations into the software forgetting that things change. The previous accident that got a lot of notice (53' long white rectangle) , was because the car didnt expect to see a semi's trailer perpendicular across the road that it was going down. So it went through - under the trailer - neatly slicing off the top of the cars cabin. You can imagine what happened to the driver. If these things keep happening, Tesla will have to admit that they are not ready for full release of its software.

It seems that its not understanding all of the things that its supposed to recognize as obstacles.

Its "Not" detecting things that its supposed to. Its not ready for full release.

 

[Khahhblaab]

It would still get ignored. Plenty of statistics show that warning buzzers that go off get ignored all the time...especially if they go off frequently. Ask any pilot who ever uninitentionally landed gear up if they heard the warning buzzer go off and damn near universally they will say that it didnt. Tests nearly universally show it was functioning perfectly .

Ask yourself how many SA's ignored the "warnings" in the logs or alert mechanisms.

Most people just dont pay attention well enough...let alone to some "bitchin betty" system.

...along with your examples, the recent news about frozen embryos accidentally coming out of frozen storage because they ran out of liquid nitrogen may be because someone turned off the alarm. The autofill stopped working correctly so the alarm went off to tell the person on call that it needed manual attention. After some time of this need to actually do some labor someone turned off or disabled the alarm so the nitrogen ran out.

The weird thing is that it happened to two different centers within days of each other. I immediately thought that they were hacked but it looks like a coincidence.

 

[amddragonpc]

There should be a slot in the dashboard where you insert a quarter and a handle you pull to start the "autopilot". Seriously, engaging the "autopilot" is gambling.

 

[nutzo]

You know what system of transportation has great autopilot? MASS TRANSIT.

We need more light rails and other similar options. Or sure, continue to figure out how to pack an ever growing population into one car each and have roads/infrastructure to support it.

So you prefer a larger number of people get killed at the same time?

https://www.cnn.com/2016/09/29/us/us-commuter-train-wreck-history-trnd/index.html

 

[theBrownLlama]

who the hell considered such warning system to be adequate?

even my cheapo car blasts a irritating and loud alarm non-stop if the passenger does not fasten the seatbelt....louder than the radio , louder than people talking, so loud you cannot do anything but tell your passenger to fasten up to get rid of it.

 

[theBrownLlama]

The 'hold steering wheel' message and accompanying flashing border occur within the first 10 seconds of hands free driving. After 15 seconds you get a loud warning sound and at around 30 seconds the warning sound plays until, if still kept hands free, the vehicle begins to decelerate into a stop at about 40 seconds.

So basically they give you about 45 seconds of wanking off behind the wheel before it deactivates and are unable to reengage it until a couple hours later.

in 10 seconds,, you're dead or someone's dead. 10s is a very long durationwhen you're driving at 85mph

 

[M76]

It's a system that follows lines, it followed the line.

It's not a self driving system it's a glorified lane assist.

The way Tesla is marketing their "lane assist" system it hurts self driving cars, because the people equate the two. When the difference in complexity is like comparing fireworks to the falcon9.

 

[focbde]

My car has a lane-keeping function as well as adaptive cruise which is a great assist on long trips. It will nudge back in the middle of a lane for example if you are not indicating, and of course maintain a distance to the car in front. If you don't have your hands on the wheel for more than 5 seconds, it will warn you, then shortly afterwards, switches off completely. If it detects no input (such as in a medical emergency event), it zig-zags in lane with the hazards on to warn other motorists and comes to a gentle stop at the side of the road.

These are all designed, as mentioned above, as ASSISTS. Anyone who crashes their Tesla because they let it completely drive itself without them having a hand on the wheel and paying attention is, unfortunately, a victim of their own bad decisions and failure to actually understand what it is there for. Tesla also are culpable, as others have mentioned, because of the way they marketed it, especially in the beginning - it's all very well later adding in requirements to have your hand on the wheel every now and then, and put up disclaimers, but a lot of the damage was done with the initial public perception of Autopilot, and how they deliberately marketed it.

 

[bbf]

in 10 seconds,, you're dead or someone's dead. 10s is a very long durationwhen you're driving at 85mph

Yep the time before a super disgusting, cannot ignore alarm triggers should be speed dependent and if there's a vehicle in front of you within radar/sonar distance going at approximately the same speed that the autopilot can follow. (Of course if the car in front of you is also a Tesla on autopilot with a negligent driver, both of you will be smashing into that barrier in quick succession :-O)

Since the object detection doesn't seem to work with CONCRETE barriers in front of the car at freeway speeds, the alarm should trigger within a second or two of non "hands on steering wheel" contact because at 75mph, in 5 seconds the vehicle travels 550 feet!

edit: oh, at 85mph, in 10 seconds a vehicle will travel approx 1250 ft. or 417 yards or 4 american football fields.

 

[ole-m]

LOL You're in a Tesla thread, buddy.. Mass transit sucks and if you had the option to summon a car, get in, and be somewhere in 10 minutes rather than an hour.. You would be all over that, just like everyone. This is the future, along with tunnels and lifts. Think of these machines as "personal mass transits" if you must.

You've clearly not seen a country with a system around it and not built around cars.
There is cities and even countries where a car is an annoyance to have to be in.

Me myself live in a country where cars is a must though.

 

[gamerk2]

So from what I can see here, the issue here is the lane markings as the lanes split in two directions. Which goes back to my long-standing argument that self-driving cars will have major issues when it comes to non-standard lane markings.

Either we need federal standards for how EVERY road in the country (and ideally, uniform markings across the planet going forward) are marked up, or self-driving cars will never be seen as reliable enough to succeed in the market.

 

[kju1]

You've clearly not seen a country with a system around it and not built around cars.
There is cities and even countries where a car is an annoyance to have to be in.

Me myself live in a country where cars is a must though.

And those arent America. They are likely the size of a small state. Please stop trying to make us into somewhere else. We like it here.

 

[VirtualMirage]

The 'hold steering wheel' message and accompanying flashing border occur within the first 10 seconds of hands free driving. After 15 seconds you get a loud warning sound and at around 30 seconds the warning sound plays until, if still kept hands free, the vehicle begins to decelerate into a stop at about 40 seconds.

So basically they give you about 45 seconds of wanking off behind the wheel before it deactivates and are unable to reengage it until a couple hours later.

Maybe Tesla has updated their software to shorten the time between hands off and alert. The video I watched (granted, it's about two years old now) the driver was able to go about 4.5 minutes before the first visual alerts started to display:

 

[Creig]

So from what I can see here, the issue here is the lane markings as the lanes split in two directions. Which goes back to my long-standing argument that self-driving cars will have major issues when it comes to non-standard lane markings.

Either we need federal standards for how EVERY road in the country (and ideally, uniform markings across the planet going forward) are marked up, or self-driving cars will never be seen as reliable enough to succeed in the market.

No, that isn't the solution. There are many areas of the country that receive significant amounts of snowfall per year which obscure lane markings for months at a time. And even if you can see them, pavement marking conditions can vary wildly depending on traffic wear, their age, bead reflectivity, the type of paint being used, etc. What is needed are multiple methods for determining lane position. The more, the better. This video shows an autonomous car in Finland operating without ANY lane markings visible whatsoever.

 

[thejokker]

You've clearly not seen a country with a system around it and not built around cars.
There is cities and even countries where a car is an annoyance to have to be in.

Me myself live in a country where cars is a must though.

Countries where the elites have ruled the masses for thousands of years. There is an entirely different mind-set in America. We are not subjects...

 

[Verge]

It's a system that follows lines, it followed the line.

It's not a self driving system it's a glorified lane assist.

The way Tesla is marketing their "lane assist" system it hurts self driving cars, because the people equate the two. When the difference in complexity is like comparing fireworks to the falcon9.

Musk said it would be fully autonomous by about now, the present.

 

[iliac]

So true. It will be horrible if the programmers at Tesla built expectations into the software forgetting that things change. The previous accident that got a lot of notice (53' long white rectangle) , was because the car didnt expect to see a semi's trailer perpendicular across the road that it was going down. So it went through - under the trailer - neatly slicing off the top of the cars cabin. You can imagine what happened to the driver. If these things keep happening, Tesla will have to admit that they are not ready for full release of its software.

It seems that its not understanding all of the things that its supposed to recognize as obstacles.

Its "Not" detecting things that its supposed to. Its not ready for full release.

And that is why it is still classified as Beta and they have to accept this disclaimer that BlueFireIce posted.

 

[kju1]

Musk said it would be fully autonomous by about now, the present.

And we were promised us flying cars 20 years ago...whats your point?

 

[M76]

Musk said it would be fully autonomous by about now, the present.

And Chris Roberts said Star Citizen will be released in 2014 with 90+ star systems. So what?

 

[M76]

So from what I can see here, the issue here is the lane markings as the lanes split in two directions. Which goes back to my long-standing argument that self-driving cars will have major issues when it comes to non-standard lane markings.

Either we need federal standards for how EVERY road in the country (and ideally, uniform markings across the planet going forward) are marked up, or self-driving cars will never be seen as reliable enough to succeed in the market.

Tesla's "autopilot" is a mere lane assist feature. and deriving any sort of opinion about self driving systems in general based on it is a huge mistake. It's like trying to figure out a total station based on a tape measure.

 

[kreno]

This is why I am not investing my money into self driving cars. Mixing autonomous vehicles in with human-controlled vehicles on roadways that cater to humans is a really bad idea. Humans are conditioned through experience to deal with the millions of unexpected situations that can happen when driving their car. Autonomous cars are nowhere near that sophisticated and even with high-end neural net learning can't deal with all the possibilities as well as a veteran human driver can.

Tesla's with Autopilot mode ARE NOT autonomous vehicles. In many tests the newer versions of autonomous vehicles drive better than people do and react to situations quicker than a human being can. Autopilot mode is designed to be used between cities for long commutes. It is very bad in complex road systems and even Tesla tells you this before using it. Whomever was in the vehicle obviously didn't read the instructions. This is on the driver not the car.

 

[spugm1r3]

A Tesla owner in Indiana decided to recreate the fatal crash from last month, where the cars autopilot mode drove straight into a K-Rail according to an article from electrek. The video below shows a Model S with the latest autopilot hardware mistakenly treating a right-side land as a left-side lane and pointing the vehicle squarely at a barrier.

How is it the radar system of neither of these cars could see a solid, immobile concrete barrier? Fortunately the Indiana driver in the video managed to stop just in time.

There seem to be a clear priority issue in their autopilot software if the presence of white lines isn't overruled by the presence of a solid object reducing its distance at a rate equal to the rate the car is travelling. Impending impact should always override a smooth drive.

 

[Jim Kim]

No, that isn't the solution. There are many areas of the country that receive significant amounts of snowfall per year which obscure lane markings for months at a time. And even if you can see them, pavement marking conditions can vary wildly depending on traffic wear, their age, bead reflectivity, the type of paint being used, etc. What is needed are multiple methods for determining lane position. The more, the better. This video shows an autonomous car in Finland operating without ANY lane markings visible whatsoever.

Had not seen that, thanks. Won't be long before autonomous vehicles will be able to traverse that same route in blackout blizzard conditions.

 

[greenman]

You've clearly not seen a country with a system around it and not built around cars.
There is cities and even countries where a car is an annoyance to have to be in.

Me myself live in a country where cars is a must though.

A third world country? Yeah buddy, I was born in one and thank God I moved to america. Because those things are awful!! PACKED buses, cheek-to-cheek. Trust me, it's *not* a solution. Just a poor man's game with poor outcomes. Elon is solving world problems yet somehow people find ways to complain about the problems becoming obsolete.

 

[Merc1138]

You've clearly not seen a country with a system around it and not built around cars.
There is cities and even countries where a car is an annoyance to have to be in.

Me myself live in a country where cars is a must though.

So the rest of the world is supposed to what, tear down everything built in the last 100 years from neighborhoods to highways to replace them with a "mass transit" optimized system that still doesn't account for people who don't live in densely packed city? You know there are people who happily live an hour drive away from even just a grocery store, right? Nevermind how gross mass transit is, just because in some places people can live without a car and get everything done within 5 blocks of their home, doesn't mean the rest of the world works that way.

 

[TwistedAegis]

So you prefer a larger number of people get killed at the same time?

https://www.cnn.com/2016/09/29/us/us-commuter-train-wreck-history-trnd/index.html

Per passenger mile, mass transit is much safer. And ultimately looking at this chart, it's much safer to be a passenger in anything other than a passenger car or light truck - otherwise it's a pedestrian or another vehicle dying, not the passenger(s) in the measured vehicle.

 

[thesmokingman]

You've clearly not seen a country with a system around it and not built around cars.
There is cities and even countries where a car is an annoyance to have to be in.

Me myself live in a country where cars is a must though.

And yet NYC and Paris are filled with cars.

 

[fwiler1]

No we fucking don't. Jesus, you make it sound like this Noble quest in a video game.

keep this shit off the road until it stops killing people.

Really? It's not killing people. People are killing themselves for not paying attention, reading obvious huge disclaimers, instructions, and using common sense. Compared to other cars, it is far safer than most and has avoided more accidents because of the system in place.

3350 people die a day in car accidents. That's about 40,000 a year. 4.6 million! crash related injuries, with half of those causing a debilitating injury. And you think autopilot is a problem? No, people are the problem, and autopilot when used properly will help save lives.

 

[Merc1138]

Really? It's not killing people. People are killing themselves for not paying attention, reading obvious huge disclaimers, instructions, and using common sense. Compared to other cars, it is far safer than most and has avoided more accidents because of the system in place.

3350 people die a day in car accidents. That's about 40,000 a year. 4.6 million! crash related injuries, with half of those causing a debilitating injury. And you think autopilot is a problem? No, people are the problem, and autopilot when used properly will help save lives.

Except the problem is that for a growing number of drivers of these vehicles, it's more like autostupid since they repeatedly use it wrong and the end result is a 5000+ pound missile flying down the road at 80mph that can randomly slam into a barrier or another vehicle because the operators can't keep their hands on the wheel and pay attention.

 

[kromc5]

Can't wait to see self driving in heavy snow, freezing rain, heavy fog, or add night time to the previous for more entertainment.

 

[Zarathustra[H]]

taint-taser in the seat.

Those are two words I never want to hear together again

 

[Zarathustra[H]]

You know what system of transportation has great autopilot? MASS TRANSIT.

We need more light rails and other similar options. Or sure, continue to figure out how to pack an ever growing population into one car each and have roads/infrastructure to support it.

Mass transit will only ever be successful once it gets you from random place A to random place B as fast or faster than a car would, without any consideration for how close places A and B are to a stop or station.

If it takes even a minute longer via public transit, I'm going to say "fuck public transit". Time is my most valuable resource by an order of magnitude. I have so little of it that every second is precious. Currently it takes me about 3 times longer to get to work via mass transit than it does via car. That isn't going to fly in any parallel universe

And no, it is not acceptable to make mass transit comparatively faster by becoming an enemy of the car, and reducing speed limits, using congestion tolls, narrowing streets and removing lanes, etc. I call for any politician pursuing this type of legislation to be taint-tasered and thrown out of office.

Mass transit needs to be improved such that it stands on its own and is faster than driving. Once this is successful, it will sell itself. One word of caution. It won't be cheap.

 

[PaulP]

we have to endure and go through these situations, accidents, deaths in order to continue the path to autonomy.
there will be sacrifices, for a reliable trustworthy end product. but these situations will and have to happen.

You would think differently if it was someone close to you that was killed. Tell you what, why don't you volunteer to beta test these things; just stay away from my loved ones. It's easy to talk all "big picture" and act noble, but it's a different story if it's you ass on the line, or someone you love.

 

[nutzo]

Per passenger mile, mass transit is much safer. And ultimately looking at this chart, it's much safer to be a passenger in anything other than a passenger car or light truck - otherwise it's a pedestrian or another vehicle dying, not the passenger(s) in the measured vehicle.

View attachment 64173

But a single person here & there usually doesn't make the news, while a full train or bus does.

Looking at the chart, I wonder how many of the "Others" killed under "commuter rail" and "transit bus" are also listed in the "users" killed in passengers cars.