Good primer on control theory and machine learning in UAVs

ArsTechnica has a good primer on UAV control theory and machine learning. The good stuff starts around page 2. Here's an excerpt (citing this Stanford research):

The helicopter is performing an entire airshow sequence, but the flights in the video are completely automated! As the helicopter flies through the various maneuvers, the computer has to deal with all kinds of complicated aerodynamics and aero-elastic effects. The rotor blades flex and twist and the helicopter has to fly through turbulence generated by its own rotor. These things are hard to measure and almost impossible to model mathematically with accuracy.

Given how nasty the aerodynamics get, the Stanford researchers couldn’t design the controller the same way as the quadrotor designs we talked about above. Instead, the Stanford team turned to the best authority they could find on helicopter flight: a skilled human radio-control (RC) pilot. They had the helicopter learn to fly by watching the pilot and flying itself using machine learning.

Again, there were two parts to this process, low-level control, and high-level trajectory generation. The researchers needed control algorithms that could keep the helicopter stable and in control during the kind of extreme flights they wanted to perform. They also had to come up with some way to represent the sequence airshow maneuvers to fly.

For the control algorithms, the team used a technique known as reinforcement learning, or RL. They started by recording the position, orientation, and speed of the helicopter, plus control data like joystick and throttle positions from a skilled human pilot. Then they took a basic model of the helicopter in simulation and paired it with a basic control algorithm, and simulated the flights thousands of times. Each time, the computer adjusted the model and the control algorithms to try to get the simulated angles and speeds closer to the real flight from the human pilot. By running the flights in simulation but with real data, they were able to learn a control algorithm that could fly helicopter at the extreme conditions for the aerobatics, as well as a model for how the helicopter behaves in those conditions.

Now, to get the helicopter to perform the extreme maneuvers shown in the video, the research team also had to specify the angles and positions of the helicopter (just like the paths generated for the quadrotors before. But this turns out to be quite complicated).

During these flights, the helicopter is performing in a highly nonlinear way. This is engineer-speak for being very unpredictable: as the helicopter spins and pirouettes, it flies through its own prop wash and turns through large angles. In this case, the aerodynamics get so complicated that it’s impossible to do what we talked about before: picking out desired intermediate points and having the computer simply connect the dots. If the helicopter doesn’t exit one maneuver with the right speed and orientation, it won’t be able to complete the next one. The whole sequence is an intricate, interdependent chain.

To get around this, the Stanford team again turned to their human expert pilot. They had the pilot take the helicopter through the entire airshow routine again and again, giving the helicopter a set of examples. However, this was just the start. The human pilot isn’t perfect; every performance differs slightly from the others. Sometimes turns are cut a little short, or a flip is faster than the pilot might have wanted. But each flight has thousands of data points, each with different angles and positions. So how do you know which part of each routine is correct?

In this case, you can’t combine the trajectories and try to average out the error. Some maneuvers were performed slightly faster or slower, and it’s almost impossible to line things up. Simply cutting and pasting different maneuvers and stunts from different flights get nonsensical results: the start of a flip may depend on the speed coming out of the loop before, and the slightest error means the difference between a completed maneuver and a crash. You can’t simply take a flip from one flight and a loop from another and just hope they work together.

I actually found the article pretty weak, and technically inaccurate in many places.

Then there is a lower level processor, often a micro-controller, which stabilizes the UAV in flight. The low-level processor acts like the involuntary nervous system, which controls reflexes like balance. It samples the gyros and other sensors several hundred times per second, adjusting the motors to keep the UAV stabilized in the air.

Yes, OK.

That’s the role of the control algorithms, the code running on the computer that samples the sensors and decides how fast to run the motors.

No, not at all. It is the low level part of the program that does this. The same one that stabilizes it. The high level stuff just figures out what angle to achieve, and the high level stuff figures out how to achieve it, by varying motor power.

The ironic thing about quadrotors is, although they may look more complicated than regular helicopters and airplanes, they’re actually simpler to control (at least for a computer). It’s this simplicity that made them so popular, and it makes possible the kind of maneuvers we see above.

Simpler in a hover, sure. But any kind of dynamic manoevers, are still really complicated with quads.

Also, who thinks quads look complicated? 4 motors, 4 props, 4 sticks. The parts count on a helicopter is 20-50 times higher.

The process itself is not simple. Because of gyroscopic precession, the lift offset is actually 90 degrees out of phase with the actual deflection of the rotor blades, so a leftward tilt of the rotor disk is actually created by increasing the blade pitch at the rear of the helicopter, and decreasing it at the front. A helicopter's controls have to automatically account for this when mapping the pilot's control inputs to the rotor system.

He's greatly overcomplicating this. The controller, or the pilot, doesn't have to know anything about gyroscopic precession. This is all handled mechanically, and this part is actually very simple. If you want to fly left, you push the stick left, the swashplate tilts up on the left, and the thrust vector goes left. Simple as that. The fact that the blade pitch is actually higher on the rear of the rotor disk than it is on the right side is pretty meaningless.

And in fact, this whole gyroscopic precession thing is debatable anyway. I've seen it argued by many that it's totally irrelevant to helicopter flight.

As you might imagine, tilting the rotor blades like this is complicated, and helicopters are really hard to fly without a lot of training. A small change in a single control input like rotor angle affects the helicopter in many different ways. And this also makes life rough for a robot designer: it’s hard to predict how changes in the controls will affect the flight of the helicopter. Human pilots train for hundreds of hours to develop the muscle memory that enables them to fly a helicopter.

I fly both. A regular helicopter is no more or no less difficult to fly than a quad. They are the same. The only reason why a helicopter has the appearance of being more difficult, is because it is possible to fly them completely manually. No computer intervention. This is difficult.

It is impossible to fly a quad without computer intervention. Therefore, they are all flown with computer aid. This computer aid makes them easy to fly. A helicopter with computer aid is also easy to fly.

Overall, I'm not that impressed with machine learning. They had to spend a lot of time to "teach" the helicopter how to do those aerodynamics. And the outcome of that work will really only fly that one helicopter, in exactly that one way. You couldn't then command it to do a brand new trick without doing more machine learning.

I would prefer the approach of getting a model of aerodynamics working, full parametrized, and use that as the basis of control. That would then allow for you to use the same program on different helicopters, just by changing the parameters appropriately.

Also, I don't see a whole lot of use to these sorts of manoevers. Yes, it's fun to figure out how to get a machine to do them autonomously. But what practical application does it serve? It is more interesting to figure out how to get them to fly in a more practical sense, fully autonomously, and based on a physical model rather than just "memorizing" a flight plan after being given 50 examples.

You need to be a member of diydrones to add comments!

Join diydrones

Comments

Rob_Lefebvre November 28, 2012 at 2:08pm

I actually found the article pretty weak, and technically inaccurate in many places.

Then there is a lower level processor, often a micro-controller, which stabilizes the UAV in flight. The low-level processor acts like the involuntary nervous system, which controls reflexes like balance. It samples the gyros and other sensors several hundred times per second, adjusting the motors to keep the UAV stabilized in the air.

Yes, OK.

That’s the role of the control algorithms, the code running on the computer that samples the sensors and decides how fast to run the motors.

No, not at all. It is the low level part of the program that does this. The same one that stabilizes it. The high level stuff just figures out what angle to achieve, and the high level stuff figures out how to achieve it, by varying motor power.

The ironic thing about quadrotors is, although they may look more complicated than regular helicopters and airplanes, they’re actually simpler to control (at least for a computer). It’s this simplicity that made them so popular, and it makes possible the kind of maneuvers we see above.

Simpler in a hover, sure. But any kind of dynamic manoevers, are still really complicated with quads.

Also, who thinks quads look complicated? 4 motors, 4 props, 4 sticks. The parts count on a helicopter is 20-50 times higher.

The process itself is not simple. Because of gyroscopic precession, the lift offset is actually 90 degrees out of phase with the actual deflection of the rotor blades, so a leftward tilt of the rotor disk is actually created by increasing the blade pitch at the rear of the helicopter, and decreasing it at the front. A helicopter's controls have to automatically account for this when mapping the pilot's control inputs to the rotor system.

He's greatly overcomplicating this. The controller, or the pilot, doesn't have to know anything about gyroscopic precession. This is all handled mechanically, and this part is actually very simple. If you want to fly left, you push the stick left, the swashplate tilts up on the left, and the thrust vector goes left. Simple as that. The fact that the blade pitch is actually higher on the rear of the rotor disk than it is on the right side is pretty meaningless.

And in fact, this whole gyroscopic precession thing is debatable anyway. I've seen it argued by many that it's totally irrelevant to helicopter flight.

As you might imagine, tilting the rotor blades like this is complicated, and helicopters are really hard to fly without a lot of training. A small change in a single control input like rotor angle affects the helicopter in many different ways. And this also makes life rough for a robot designer: it’s hard to predict how changes in the controls will affect the flight of the helicopter. Human pilots train for hundreds of hours to develop the muscle memory that enables them to fly a helicopter.

I fly both. A regular helicopter is no more or no less difficult to fly than a quad. They are the same. The only reason why a helicopter has the appearance of being more difficult, is because it is possible to fly them completely manually. No computer intervention. This is difficult.

It is impossible to fly a quad without computer intervention. Therefore, they are all flown with computer aid. This computer aid makes them easy to fly. A helicopter with computer aid is also easy to fly.

Overall, I'm not that impressed with machine learning. They had to spend a lot of time to "teach" the helicopter how to do those aerodynamics. And the outcome of that work will really only fly that one helicopter, in exactly that one way. You couldn't then command it to do a brand new trick without doing more machine learning.

I would prefer the approach of getting a model of aerodynamics working, full parametrized, and use that as the basis of control. That would then allow for you to use the same program on different helicopters, just by changing the parameters appropriately.

Also, I don't see a whole lot of use to these sorts of manoevers. Yes, it's fun to figure out how to get a machine to do them autonomously. But what practical application does it serve? It is more interesting to figure out how to get them to fly in a more practical sense, fully autonomously, and based on a physical model rather than just "memorizing" a flight plan after being given 50 examples.

This reply was deleted.