Tesla Autopilot vlogs

Just watched this vlog showing Autopilot’s improvement on handling steep curves:

Pretty cool.


Brings a smile to my face every time.

I wonder how much more processing is required to be able to get that extra 4mph done safely. What the margins are now.


Curves below about 25mph recommended rate become a special kind of problem for the current (2018.48 or so) autopilot because lane curvature becomes so extreme that the full width of the lane cannot be observed in the main camera once you get to that level of curvature and are mid-turn.

Look at 3:39 on this video that verygreen made of autopilot camera views on a twisty mountain road: https://www.youtube.com/watch?v=SzRGVagAkJ4&t=345s

This is the first sub 25mph turn on the video where green stays in the middle of the lane because he’s driving manually at that point and you can see he’s at about 21mph. There isn’t enough lane in the view of the main camera to plan the path. In the fisheye you can see plenty of lane but from the lane boundary overlay you can see that the lane boundary predictions done by the fisheye are way off.

This video was taken using 2018.42 if I remember correctly.

The NN capability for the fisheye is a lot less than what has been used for main and narrow in recent versions. Fisheye was mainly being used for wiper activation for a long time but now it has objects, drivable space, and lane attributes being output but because the network is smaller and shallower it’s accuracy hasn’t been very good in edge cases (like very tight turns). I think Tesla has probably been adding capacity to the fisheye to manage tight turns (I haven’t seen a recent diagram for the fisheye so I’m not sure). If they do a good job with that it’ll probably be sufficient to do highway interchanges down to 25mph reliably and at decent speed, but there will remain turns that can’t be done well.

To manage really tight turns of unusual geometry (not simple right angles at street corners) it’s probably necessary to fuse the camera network for the pillars with the fisheye because the fisheye alone can’t see well enough to generate good lane boundaries in the real world. Just fusing the output of the camera networks is probably not up to the task - they might need to be fused down close to the input layers.

A unified camera network (like AKNET_V9) should be able to manage this nicely, but we haven’t see that one running in the wild yet so it’s still unclear.

I’ve been seeing the same thing that the OP’s video shows - good progress on interchanges going from 2018.38 to 42 to 48. It’s really rare to see a highway interchange that 48 can’t handle but there are a lot of them where it’s going 10mph under what a human would normally do. But I had 48 out on twisty mountain road this weekend and it can still fail due to high curvature on even well marked roads when lanes are narrow, the road surface undulates, or visibility is constrained by roadbed adjacent obstacles. I think camera fusion can handle these super-hard cases but think they might not get solved as long as single camera networks are being used.

1 Like