Tesla FSD v10.11 replaces "bag of points" with a transformer

@jimmy_d we beckon you for your expert analysis.

Elon chimes in:

1 Like

@jimmy_d I immediately went to your twitter account to see if you had responded to it :smile:

If I had to guess we’ve seen these “vector lanes” before in Green videos.

green on Twitter: “And then only the most strong of us would just go and play. https://t.co/CeE8fIKSu8” / Twitter

Presumably in past versions of AP they had the Birds Eye View network generating a “bag of points” (Seen on the left in the display) which was then fed into some black box either neural net or code that spit out the lane semantic segmentation.

If they’re solving both in tandem then the vector space on the right with lane semantics would just come straight from the camera space features instead of losing all of that fidelity by first passing through the BEV net.

Update from @jimmy_d:

Keep an eye on:

1 Like

What Elon said on this topic in December:

jimmy_d’s explanation is here!

Video:

Audio: