Hot on the back of Tesla’s 10.10 We, Robot event, the company has released a new Tesla Optimus video today. The video runs for just 1 minute, 24 seconds, but packs in a lot of new progress with their humanoid robot.
Last week Tesla had more than 20 robots walking among the crowd at the We, Robot event, serving drinks, dancing and at the time we thought we seen the latest from Optimus. When I first heard a voice from Teslabot, I had assumed we were seeing Tesla show off an early build of Grok integration.
Once we seen the second and third videos with different, very human voices, it was pretty clear this was teleoperators providing the voice interactions. While this disappointed many, it was clear Tesla were showing their hardware at the event, rather than their progress on a conversational AI voice model.
Tesla did bring their new hand to the event which showed 22 degrees of freedom, offering capabilities closer to what humans are capable of.
Today’s video shows a lot more about Tesla’s progress on the software side of their Optimus robot.
Smooth Walker
The video starts by showing TeslaBot walking with confidence. This section is at 1x speed and the rate appears to have increased a little, although still probably down on the initial spec of 5MPH. The feet now appear to be covered by shoes, the kneecaps look to be refined, and overall the robot looks to be more refined, perhaps a v2.5 prototype.
The next shot is sped up to 2x speed, but this is not the important part, the map in the bottom right corner, and the detail that Teslabot is now able to automatically avoid people and obstacles using neural nets running on it’s computer (the same HW4 computer form Tesla’s cars).
Tesla’s Senior Software Engineer @ Tesla Autopilot, Jack Han reposted ‘We trained another neural net‘.
As we know, Tesla has been training Neural Nets for the car for years and recently merged that to an end-to-end model that attempts to replicate the how humans perceive the world around it and plan a safe route through an environment.
From Han’s post, it appears that Teslabot now runs a completely different model than the cars which makes sense given they often tackle challenges on the Y axis, as we find out further in the video.
Being able to detect and avoid people is an important attribute of being able to inhabit the same physical space as humans. Being able to avoid obstacles will help ensure Teslabot can complete a task without falling.
The map in the lower-right corner shows the path planning from Optimus, allowing it to turn and walk between people and pallets of hardware next to the production line.
Seeing the world
In the next section of the video, we see a video from the head-mounted camera in Optimus. This view shows green dots that identify edges of objects, which is also likely to be used for depth perception.
What’s impressive to me is the field of view, this reminds me a lot of the rear-view camera in the car, a large fish-eye lens that provides great vertical and horizontal visibility. Its important that Teslabot can see the floor, as this is the surface it will be walking to in a few steps. The roof may also provide important context like exit signs above doors to know the difference between internal and external doors.
Multiple bots create Photosynth point cloud
Tesla shows a 3D point cloud similar to that of Microsoft’s Photosynth back in the day, constructed entirely from images. The special part about this map is that it’s created from multiple Teslabots.
It is important that Tesla doesn’t rely on this as not all deployments will include multiple Teslabots, but where you do have multiple, connected bots have the ability to actually see around corners, a feature not available to humans.
This 3D map can be used for navigation and we’d love to see the interface for Bot owners to issue tasks to the robot. I think it’s a safe bet that you’ll define the source and destination and the robot will find it’s path through the environment.
Teslabot Charging
For the first time, we just learned how Tesla plans of having the robot recharge and the good new is, they will be able to automatically recharge. One of the very first question since the unveiling of the humanoid robot is how will it charge.
Many speculated that Tesla would use wireless charging in the feet and like a robot vacuum, simply return to base and stand on top of it, similar to your rechargeable toothbrush.
From today’s video we now see that Tesla uses a variant of the Wall Charger for their cars, with a longer and thinner design which offers a charging tab at the top. The robot simple walks over to, backs up to it and the charger plugs into the back of the shoulders (very close to jacking into the back of the head like the matrix).
The text suggests this is done using the rear-camera alone, another impressive feat.
Something we see for the first time is the red light around the edges of the Optimus head, typically a friendlier white, it looks like this will act as a battery status indicator and turn red when the bot needs charging.
As we see the robot emerge from it’s charging session, the LED strip has returned to white.
Don’t skip arm day
We see Teslabot pickup a 5×6 case of 4680 batteries. With each cell tipping the scales at around 355 grams/cell * 30 cells = 10,650 grams, or 10.65 kilograms. Once we add in the case itself, we land around 11kg, a weight Optimus appears to move with ease.
Showing off
In this portion of the video, we see Teslabot picking a battery from a machine that is above shoulder height, which provides for a great opportunity to show off the bot’s range of motion.
In this image, we see the robot’s shoulder joint rotating up and notice the underarm is all covered nicely, appearing to show that Tesla has addressed possible pinch points.
Having this range of motion will certainly allow Teslabot to perform tasks not possible by a robot with more restricted movement.
Keeping it cool
Tesla makes a point of highlighting that as the bot autonomously carries significant payloads, it can do this without overheating. This suggests that previous versions were overheating as their actuators were pushed to the limit.
Obviously it’s important for the longevity and performance of the bot to maintain a comfortable thermal envelope. Also worth considering is that the bot is operating in an air conditioned factory, if it really is going to walk to the shop and return with your groceries on a 40 degree day, then it better be able to handle the heat.
Optimus can now walk up stairs
If your plan was to simply run up a flight of stairs to get away from robots if it all goes badly, then you’re out of luck. The video shows Tesla now has the software identifying stairs and being able to walk up the steps without using any safety cables or even placing it’s hands on the railing.
The front edge of the steps do feature hazard tape and it’s not clear if this is required or simply part of training. Clearly the real world does not feature this on every staircase, and we can’t expect it to be retrofitted either, so Teslabot needs to use vision alone to achieve this.
In the lower-right corner of the video, we see the depth map in 3D created by the vision system, suggesting the bot has clearly seen (and mapped) the stairs on the approach, as it’s head does not look down as it walks to the upper level.
Can it also walk down stairs? That’s an interesting question, clearly it’ll need to, but the fact we don’t see that occurring in the video suggests that maybe that’s still an active area of development.
Can Optimus take the lift? That’s also probably a no at this point, but given that’s a far more efficient method to move throughout a building, it’ll certainly have to in the future.
Interacting with humans
In the last sequence, we see a number of Tesla employees lining up to select items, similar to what would occur in a retail setting. The humans wave, Optimus waves back, the humans point at the product they want and Optimus retrieves the object and passes it to them (I’d expect the little issue of payment to come later).
We seen Optimus doing this behaviour at the We, Robot event, so nothing particularly new in that, however..
There is a shot that shows Optimus reacting to unsafe behaviours. As much as we’d like to think humans will always behave well are robots, that may not always be true.
At one stage the human holds out a fist and moves it towards the robot, I believe simulating an attempt to punch the robot. Teslabot’s response was to hold a finger up, as if to say, that’s not allowed. It’s easy to imagine this could be paired with voice, or an alarm to alert others if things got heated. This may be most applicable for deployments where Teslabot is performing security roles, either way, it’s a neat demonstration of their software.
Let us know in the comments what you think of the new Optimus video from Tesla.