Saturday, March 5, 2011

Something's Missing - Another Installment of Cause and Effect.

The time has come for aviation to embrace a cold soul searching willingness to challenge everything we’ve been told or believe about our sport. Instead though, aviation’s leaders continue to shore up reputations, seek out quantity over quality, and treat us like mushrooms. Today’s example comes from EAA.
You may have read recently, my take on EAA’s Young Eagles Program. If so, you likely had one of three responses; he’s wrong, he’s right, or I feel he’s right but I am unwilling to let go of my desire to believe Young Eagles produces new pilots. Well, if you are the later, please remember that I did not say Young Eagles never produced a pilot. And if you said “he’s wrong”, today you are likely laughing with glee due to EAA’s release of their Young Eagles pilot certification matching statistics. Unfortunately, EAA is wrong once again.
Today, many people learned that EAA has been matching up names of Young Eagles, the only record they ever kept, to the FAA pilot database in an attempt to extrapolate information; information that would prove Young Eagles is a great success. And boy did they pull, tug, and rip information from the data they found. Unfortunately, there is a problem; they are making baseless assumptions. This is concerning. EAA flat out refuses to look at reality and instead they are seeing what they want to see; their pet program is a huge success.
Many people, when they hear me question these EAA releases, go immediately to the “you hate EAA” fallback position that allows them to discount what I am suggesting without putting any thought into it. That’s fine with me. It’s their choice what to believe. But please do not say I hate EAA, simply because I am asking for logic. To the contrary, I love EAA and I believe it needs help. And when things are not going well, I think some tough love and logic should win out over feel good “trust me” promises.
EAA’s latest feel good production is the result of their pilot certificate database search for names recorded in the Young Eagle’s logbook. In what is perhaps one of the most blatant twistings of irrelevant data I have ever seen, EAA has left me with two feelings, either they are blatantly telling us something they do not really believe, or they have people running the show who do not understand basic cause and effect relationships. Neither of which is a good indicator for the longevity of an organization.
With this recent release of “data”, EAA is making the following claims, assumptions, or inferences:
•7.3 percent of all pilots below age 35 are former Young Eagles (the oldest Young Eagles from 1992 are now reaching age 35).
•Young Eagles are 5.4 times more likely to earn a pilot certificate than those who have not flown as a Young Eagle.
•Nine percent of those pilots are female, a 50 percent difference when compared to females being just 6 percent of the current U.S. pilot population.
•Two out of every 100 young people who take their first Young Eagles flight at age 17 earn pilot certificates. The older a Young Eagle is at the time of a first flight, the more likely that young person is to become a pilot. Young Eagles ages 13 and up are especially more likely to pursue a pilot certificate.
•The more flights that a Young Eagle takes, the more likely that young person will become a pilot.
The interesting thing is that Young Eagles has been very successful with one act, giving free rides to kids. Nobody can deny that and nobody would say that these rides are patently useless. To the contrary, giving kids rides is a great way to share aviation. I just do not believe these rides just are an economical or effective way to generate NEW pilots from a population of kids who otherwise have no PRIOR interest in flying. This is the hidden flaw in all of EAA’s assumptions and inferences.
To make my position easier to understand, think of the kid who builds airplane models. I think nearly everyone would agree with the statement “Kids who build airplane models have an inherent or pre-existing interest in aviation”. But if Monogram or Revell model companies were using the same statistical analyst EAA is using, whoever that mystery person is, they would be telling us the same thing; kids who build their models are 5.4 times more likely to earn a pilot’s certificate than kids who have never built an airplane model. Do you see how laughable and yet relative this information is? And do you see how this information relates to pilot’s certificates?
If my example was successful, you now see what a good statistical analyst would see in this information; the truly useful information that is overlooked. If you had a way to track the kids that build aircraft models, you would already have a wealth of information as to who would be seeking out flight training in later years. Likewise, if you were honest with the Young Eagles information, you would understand that Young Eagles doesn’t create pilots. More correctly stated, Young Eagles is an indicator of kids who have a pre-existing interest in flight.
EAA is not alone in their mistake though, and it’s time for aviation to get serious about the future of flight.
Here is a brief explanation of EAA’s inferences above. I say they are inferences because they do, more than anything, infer that Young Eagles is responsible for all the data. Unfortunately, their statistics merely mirror what we already knew about aviators.
*EAA - 7.3 percent of all pilots below age 35 are former Young Eagles (the oldest Young Eagles from 1992 are now reaching age 35).
Explanation - This is written to infer 7.3 percent are because of Young Eagles yet it doesn’t say that does it. Like my example above, I’m sure you could generate a statistical number that showed a certain percentage of pilots in that age group built aircraft models. This does not convey model building created an interest that wasn’t already there. Young Eagles is also very good at one thing, giving free rides to kids. Therefore, if a kid had a true interest in aviation during the lifetime of Young Eagles, it is very likely they went for a free ride. It does not mean 7.3 percent of all pilots in the stated age category would not have been pilots were it not for a Young Eagles ride.
*EAA - Young Eagles are 5.4 times more likely to earn a pilot certificate than those who have not flown as a Young Eagle.
Explanation – Again, this is written to infer Young Eagles increased the likelihood of people earning their pilot’s certificate. Unfortunately, it does not do that. It merely indicates the obvious correlation that kids who were already interested in flying were 5.4 times more likely to go for Young Eagles rides.  Yet another major problem here is 5.4 times more likely than who? Where is the comparison data and how credible is it?
*EAA - Nine percent of those pilots are female, a 50 percent difference when compared to females being just 6 percent of the current U.S. pilot population.
Explanation – This increased has happened in every aspect of our culture over the same period. Are we to believe Young Eagles rides are responsible for the increase in female CEO’s, the increase in female Harley Davidson riders, females on previously all male sports teams, and female politicians? No, this merely parallels trends in all of the USA.
*EAA - Two out of every 100 young people who take their first Young Eagles flight at age 17 earn pilot certificates. The older a Young Eagle is at the time of a first flight, the more likely that young person is to become a pilot. Young Eagles ages 13 and up are especially more likely to pursue a pilot certificate.
Explanation – Again, didn’t we already know this? Wait, what is it we know? This infers Young Eagles rides increase the chance of a kid getting a pilot’s certificate if the kid is older. Unfortunately, the older a kid is, the more firmed up their desires are and the more life experience they have. Therefore, just like the older kid that is still building models at age 17, the older kid going for Young Eagles rides is naturally more likely to have a real passion for flight. The building of models, or Young Eagles, did not create that passion, it is an expression of it.
*EAA - The more flights that a Young Eagle takes, the more likely that young person will become a pilot.
Explanation – In letter count, this is the smallest inference advanced by EAA. Yet what they infer is bold. Here it is suggested that the more you fly with Young Eagles the more likely you are to become a pilot. Instead, I believe a good analyst would find those kids, who are most passionate about flight, do everything they can to get additional rides. This is the logical inference but it is nowhere to be found. And what about EAA’s one flight rule?
*EAA - They do not give a total number of pilots produced
Explanation - You figure it out.
In conclusion, what I wish to point out that if EAA, and aviation as a whole, were willing to see the true morsels of information in these programs they might be useful to the future of aviation. Just look at Sporty’s.
Hal Shevers, of Sporty’s fame, has been in business a long time and he is no dummy. He also is a big supporter of Young Eagles. By getting Sporty’s in front of Young Eagles at an early age, I’m sure he knows what few are willing to admit or maybe they don’t realize; the kids coming for Young Eagles are already tomorrow’s pilots. Young Eagles merely finds them for him.  This allows Sporty's to offer programs that help both Hal's bottom line and potential students achieve their goal of flight.
I hope you do not misunderstand. Hal has been a long time supporter of aviation. I am just pointing out that if you want to keep aviation alive, aviation businesses must be kept alive, and to keep them alive, we can no longer afford feel good baseless information over honest, hard cutting data. Of course, that kind of information doesn’t generate the level of donations EAA expects. But maybe it would; if only they would try.
With all the above considered, there is one question that must be asked:
If EAA has touched 1.6 million kids who had an interest in flight, why then have they produced so few pilots from these ranks; a number conspicuously absent from EAA's statistics.  It really is a good question.

*Reference image at top:
Cause equals the kids with a pre-existing interest in aviation
Event equals the Young Eagles program.
Effect equals the conversion to pilots.  (without total claimed pilots from Young Eagles, this cannot be determined and neither can the success of the program)

7 comments:

Anonymous said...

I understand, I think, your contention that the Young Eagles program is not as ineffective as it claims to be. What I don't understand is why this is of such great concern to you. There are lots of do-gooder programs out there that do some good but are not as effective as they could be. (Bake sales come to mine: If the people providing the cakes just donated the money it took to make them it would probably produce more revenue -- but, hey, people like to feel like they're doing something important even if it's not very effective.) So what? Unless you are in some way responsible or accountable for the program, why not just let them give their airplane rides and feel good about it? What business is it of yours?

What's your proposal and how do you plan to carry it out, if at all?

Respectfully.

Rich Davidson said...

Anonymous,

Your first question is easier to answer with an example. I will do this by changing the subject from your post. Here it is. See if you can find it. "So what? Unless you are in some way responsible or accountable for the program, why not just let them give their taxes to the government and feel good about it? What business is it of yours?"

As a member, it is my business. As a thinking person, it is also my business to ask why someone who is claiming to represent me feels the need to push deceptive figures relating to their effectiveness. And even if I were not a member, I would have to ask the same. Aviation needs help, not "let them feel good about it."

If you read my post with a clear non-defensive mind, I say giving rides to kids is a great way to share aviation. My point is that Young Eagles is not an effective enough a program to base 90% of EAA's pilot creation efforts on it.

As for what would I do, I have said this many times. But aviators are hooked on the notion that the only way to make pilots is take someone for a ride. Long term solutions based on good data are not what people want. They want, as you say, to feel good. But, look for a post in the future about programs that are effective. I have been collecting examples for a while.

And remember, aviation is not made of pilots only. For aviation to exist, there needs to be a general interest in aviation that creates, mechanics, business people, rampers, etc.

Anonymous said...

Okay I see now. I agree that as an EAA member you have a legitimate stake in the organization's actions and therefore a right to try and improve their efforts.

I disagree, though, that an outsider would have similar standing to criticize their actions if, as you say, they do some good and no harm and their membership is not in revolt over the issue.

I guess I just weary of freelance critics who are not responsible or accountable for the things they criticize. Jim Campbell's recent rants about Cirrus are a good example.

Anonymous said...

Just read your shuttle piece over at ANN.

I think the shuttle decision should be based on factors other than how it affects your personal vendetta with a former mayor. Do you really think the school children who visit the planetarium in Chicago are responsible for closing Meigs? And if by some twisted logic you think they are, just how long do you think they should be denied the opportunity to benefit from aviation in any form?

As to telling NASA how it could win back your heart, I doubt anyone there is focusing real hard on that. (They're probably too busy stifling innovation to be bothered by some crank.)

Rich Davidson said...

Response to last comment (not about Young Eagles).

Let's all be honest for a change and admit that "For the children" is the most over used white lie on the planet. What it really means is "We really want this so we'll tell you anything that will pull on your heart strings to garner your support."

As for Chicago getting an orbiter, any city that rides rough shod over its citizens desires is not in the business of inspiring anyone, children included. If you believe otherwise you are extremely gullible. Furthermore, Chicago is not the only place on the planet with children.

There are many other places, like Texas, that would be a far better fit for the orbiters. In these places they would be on display in appropriate environments surrounded by people who really do want to inspire kids to reach for the stars. Chicago is not such a place.

Finally, you should lighten up. I know they don't care about winning my heart; Syrians' maybe, but not mine.

Thanks for you response. It was fun.

Anonymous said...

You said: There are many other places, like Texas, that would be a far better fit for the orbiters. In these places they would be on display in appropriate environments surrounded by people who really do want to inspire kids to reach for the stars. Chicago is not such a place.

In your opinion. Sure, there are children everywhere and they all deserve the same shot at the shuttle -- including those in Chicago. Unless of course they should be excluded for the sins of... just who, exactly? Their parents, for voting for Daly -- if they did? An ex-mayor's sins? The children's for having chosen parents who may or may not have voted for Daly?

Anonymous said...

Texas may well be a better place for teh shuttle than Chicago, but that determination should be based on factors other than the actions years ago of a now ex-mayor.