Engineering is all about calculating and predicting things well enough that you get the best outcome possible. So it follows that we’d want all of our models to be as accurate as we can manage, right? I mean, it makes sense.
But when it comes to replicating the physics of a UAV in a digital simulation environment, sometimes the model needs to actually be slightly inaccurate in order to match real-world behavior.
This can feel very wrong to do if you’re not used to it, like you’re now accepting sub-par work from yourself and your team. As with so many aspects of engineering, it takes experience to know when absolute accuracy is critical, and when you can let some things slide in service of other priorities.
Let’s talk about how with engineering models, better accuracy may not necessarily create a better model.
“Accurate” stall behavior does not mean “realistic”
One of my favorite examples of where high accuracy can be undesirable—in fact, the one that led to this newsletter—is aircraft stall behavior in a sim environment.
A while back I spoke with a team about the simulation they were building. They specifically needed airfoil behavior for angles of attack from 0 to 180 degrees, and were able to find some test data by Sandia National Labs that they could use. This is what lift of an airfoil at 0 to 180 degrees looks like:
The reason they spoke with me, though, was because their aircraft model was behaving oddly. Most notably, after stalling and with full up elevator, the aircraft would calmly float down through the air. The expected behavior would be for the nose to pitch down but that was certainly not happening.
I pointed to that extended airfoil lift curve as a possible contributor. The lift curve features nonzero lift coefficients past the initial stall at 15 degrees angle of attack—and including this in the sim environment let the computer think the aircraft could fly at those higher angles.
Key to remember is that a sim “model” is really just strings of numbers: the lift curve we give our code is but a vector of angles matched to a vector of coefficients. The lift curve does not capture all the aerodynamic phenomena at play, and this is critical.
It does not capture the increased turbulence at higher angles, or the huge change in stability when your airfoil’s separation bubble is finally large enough to trigger a stall. The model is literally just a measurement of lift generation capability, without the context to create the real behavior.
What’s very sneaky is that wind tunnel data doesn’t usually capture this well either. In a wind tunnel you force your test article to maintain a constant angle of attack. Most tunnels are set up to take hundreds of data samples at every condition, and the average of all these is the single data point output for that condition.
So if you’re collecting a few hundred data samples from an airfoil held at an angle of attack with high turbulence, the average of all those samples will say the airfoil is generating lift. But in the real world, the amount of turbulence and airflow separation present effectively make that lift generation meaningless.
When you take that wind tunnel data and directly drop it into a sim environment, you lose the key behavior that makes a stall a stall. In that original simulation, the aircraft might get pushed past its stall angle. And because the aerodynamic model says there’s usable lift past stall, the sim aircraft might keep flying at an impossible angle of attack.
So how do we make our aircraft quite literally straighten up and fly right?
Matching the accuracy to the impact
For once the solution is very simple: one option is to have lift abruptly drop off after the stall angle of attack. Beyond maybe a degree or two past stall, you set CL to zero. Another option is to set a hard-coded maximum CL that the aircraft literally cannot exceed; some autopilots or sim environments may even require this.
I’m not the biggest fan of hard-coding because if you revise your aero model (as you might do during new product development) you’ll need to remember to double-check the CLmax value. I also prefer the lift curve modification because if somehow the sim aircraft reaches a super-high angle of attack, there will still be “instructions” for what to do. Either option is valid, the choice just comes down to your setup and what makes sense to you.
How important “accurate” modeling is also comes into play in other places. I see this with Reynolds number effects models—effectively, how the coefficients change as the aircraft flies through extremely slow airspeeds.
A highly accurate way to implement this would be to have separate aero models for different key Reynolds numbers. At Re = 100,000, the sim uses one specific lift curve; at Re = 200,000 it uses a different lift curve, and so on. This provides very precisely-tuned coefficients to match behavior at each Reynolds number.
Quite frankly, this sounds like an interpolation nightmare. Let’s not do that.
What if we added Reynolds number corrections as deltas to the existing coefficients? We’ll cover our expected range in steps of 500, calculating the difference between our standard aero and the Re-impacted coefficients for Re = 50k, 50.5k, 51k, 51.5k….
Sure, that would be accurate. But that’s a lot of data to correct. And in truth, the flight time where Reynolds number effects actually matter is maybe 1-2 minutes out of the entire flight. Are those 2 minutes so important to accurately capture that it justifies spending 20+ hours to generate the data, construct the deltas, and integrate it into the sim environment? Honestly, no.
Let’s do something similar, but just use steps of Re = 50k. This model won’t be perfectly accurate. But it will be good enough for the level of its impact on the sim results.
And considering how low the aerodynamic forces already are at the airspeeds where these effects matter, you could argue modeling them isn’t necessary to begin with. Again—you don’t need, and sometimes don’t want, perfect accuracy to get realistic sim behavior.
Why I don’t trust all-in-one tools
Maybe it’s a bit of a stretch, but I’d argue that the importance of accuracy also applies to the tools you use. In this context, I’m using “tools” to refer to actual computer programs (STAR-CCM, Cameo, etc) as well as purpose-written codes and scripts.
In my view, not every single code or simulation needs to include every possible factor to handle any possible question, and so operate with the highest possible accuracy. I’m fine with my tools being really good at one job I want them to do, and only as accurate as they need to be for that job.
Any code will become bloated, unwieldy, and hard to manage if you add too much. You might want your aircraft performance code to be as accurate as you can get, so you build an entire six-degree-of-freedom model into it to account for the turns and orbits in a flight. You should account for the changes in power, lift, and drag resulting from that turn…right?
But is that a worthwhile increase in effort? As in, does the work to include the full 6DoF make your performance predictions substantially better, and in turn does it meaningfully influence the downstream outcomes? If you’re just providing a sense of the aircraft’s capabilities in a proposal, does it truly matter if a specific configuration can fly for 12 hours, versus 12.17?
You might be better served sticking with a three-degree-of-freedom model (lift, drag, pitch) for your performance code, and using that saved time to evaluate additional potential configurations, or even doing more static testing to validate your propeller choices.
I’ve talked about it plenty before, but I feel we can all use an occasional reminder: Accuracy in our modeling is awesome and something we should strive for. But what we truly need is realism, not accuracy, and all our combined data sources need to produce behavior we can believe—the definition of realistic.
In the pursuit of that realism and accuracy, we should keep in mind how much a higher level of detail really changes the final answer, and if the effort required is the most beneficial use of our precious time.