PREVIEW

For the first time, a robot trained by watching videos of seasoned surgeons has successfully executed the same surgical procedures with the skill of human doctors.

The breakthrough in imitation learning could reshape robotic surgery by removing the need to program each individual move a robot must perform in a medical procedure. Instead, the robot learns from observing, bringing the field closer to full autonomy, where robots could one day perform complex surgeries independently.

The findings, led by researchers at Johns Hopkins University, are being highlighted this week at the prestigious Conference on Robot Learning in Munich. "It's really magical to have this model and all we do is feed it camera input and it can predict the robotic movements needed for surgery," said senior author Axel Krieger, assistant professor in JHU's Department of Mechanical Engineering. "We believe this marks a significant step forward toward a new frontier in medical robotics."

Working with a team from Stanford University, the researchers used imitation learning to train the da Vinci Surgical System robot to master three essential surgical tasks: needle manipulation, tissue lifting, and suturing. Each time, the robot trained on their model performed with a skill level matching that of human surgeons.

The model uses a machine learning architecture similar to ChatGPT but tailored to "robot" kinematics—breaking down robotic movements into mathematical angles rather than words. The training involved hundreds of videos from wrist cameras on da Vinci robots, captured by surgeons globally. With nearly 7,000 da Vinci systems in operation and over 50,000 trained surgeons, this large data archive allows robots to imitate with precision.

Despite the da Vinci system’s known imprecision, the team overcame this by training the model to perform relative movements, rather than absolute, ensuring accuracy. "All we need is image input and then this AI system finds the right action," explained lead author Ji Woong "Brian" Kim, a postdoctoral researcher at Johns Hopkins. "Even with a few hundred demos, the model can learn the procedure and adapt to new environments it hasn’t seen before."

Krieger added, "The model learns things we haven’t taught it. If it drops the needle, it picks it up and continues—something I never explicitly taught."

According to the team, this model could swiftly train robots to handle various surgical procedures. They’re now working on using imitation learning to perform not just individual tasks but complete surgeries.

Previously, programming a robot for even basic surgical tasks required meticulously coding each movement, a process that could take years for a single procedure. "It’s very limiting," said Krieger. "What’s new here is that by collecting imitation learning across different procedures, we can train a robot in days. This accelerates us toward autonomy, reduces errors, and enhances surgical precision."