Writing code is already a tedious task, but having to program robots is even more work. The coder has to design multiple steps, such as detecting objects, triggering actuators to move the robot’s limbs, code to indicate task completion, and so on.
Google’s robotics researchers are now working on putting an end to this hassle. Their team of scientists has developed a robot capable of writing its own programming code based on ordinary language instructions. For instance, you could tell the robot to “pick up the yellow block” and it would take care of the rest. You would no longer have to get into the robot’s configuration files to change block_target_color from #FF0000 to #FFFF00.
This robot uses Code as Policies (CaP) developed from Google’s Pathways Language Model (PaLM) to take instructions from ordinary language and turn them into code. The robot’s AI model was trained by examples of instructions (formatted as code comments written by the developers to explain what the code does for anyone reviewing it) and the corresponding code.
Google engineers explained in a blog post published earlier this week that the robot was able to take new instructions and:
Autonomously generate new code that re-composes API calls, synthesizes new functions, and expresses feedback loops to assemble new behaviors at runtime.
It was able to come up with probable robot code on its own, using “hierarchical code generation”, from which it can develop code and repurpose it for similar instructions. Since CaP is built on a regular language-based model, it has some non-code-related features as well, like understanding emojis or languages other than English.
However, CaP’s capabilities are still very limited since it can only work with the right context and instructions that make sense to it. It cannot handle parameters it does not support or more complex instructions that require dozens of parameters in a single prompt.
There are also safety concerns since it is able to write its own code. If the robot thinks the best way to achieve its task is to flail its arm around with humans nearby then someone could get hurt.
Regardless, it is still exciting research, and it would make the lives of programmers a lot easier if done right.