Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: Understanding the safety of an aircraft via data rather than opinions

  1. #1

    Join Date
    Jun 2018
    Location
    ATL
    Posts
    78

    Understanding the safety of an aircraft via data rather than opinions

    I have been thinking about this since I saw this Youtube video comparing LSA accident rates many month back.

    https://youtu.be/iv_rRus-X9k

    But yesterday, I received the latest Kitplanes Magazine, where Ron Wanttaja compared accidents rates with engines as a factor, and the data basically show that 2 strokes are safer than auto conversions (which goes against most of the opinions I have heard).

    This has lead me back to something that has been bothering me since I started shopping for a plane: How safe or dangerous is the Grumman AA1?

    I have heard many opinions, most of them suggesting that the AA1 is a deathtrap, but I have not seen any data to directly compare it to the C150. Where can I find data to make the comparison between aircraft?

    I would like to see a data based comparison of many models including (but not limited to):

    C150 vs AA1
    AA1 vs AA1A/B/C (I have read that the wing makes it a very different plane)
    C150 vs Tomahawk
    C150 vs Ercoupe (how safe is the "safest plane" from a more modern perspective)
    C150 vs C172 (since the C172 is the measuring stick for GA planes)
    C172 vs C150
    1978 Grumman AA1C w/O-320

  2. #2
    FlyingRon's Avatar
    Join Date
    Aug 2011
    Location
    NC26 (Catawba, NC)
    Posts
    2,627
    Whether you can extrapolate Ron's data into an indication if a particular design is safe, is also opinion. Ron's done a lot of research, but he doesn't make the claims you're trying to make.

    Here's a statistic. I've had more certificated engines in certificated planes fail on me than I've ever had a two-stroke or VW conversion. (Mostly because I've never flown behind a VW, and only flown in a 2 stroke aircraft twice). I have had both a Lycoming and Continental crap on me in 1000 or so hours of flying the Navion.

  3. #3

    Join Date
    Jul 2011
    Location
    Clarklake, MI
    Posts
    2,461
    Quote Originally Posted by thisadviceisworthles View Post
    This has lead me back to something that has been bothering me since I started shopping for a plane: How safe or dangerous is the Grumman AA1?

    I have heard many opinions, most of them suggesting that the AA1 is a deathtrap, but I have not seen any data to directly compare it to the C150. Where can I find data to make the comparison between aircraft?
    As in accident data?

    You can dig up back issues of something like "Aviation Consumer" for a supposedly non-biased opinion of the AA-1 and C-150 then compare and draw you own conclusions.

  4. #4
    rwanttaja's Avatar
    Join Date
    Jul 2011
    Location
    Seattle
    Posts
    2,948
    Quote Originally Posted by thisadviceisworthles View Post
    This has lead me back to something that has been bothering me since I started shopping for a plane: How safe or dangerous is the Grumman AA1?

    I have heard many opinions, most of them suggesting that the AA1 is a deathtrap, but I have not seen any data to directly compare it to the C150. Where can I find data to make the comparison between aircraft?

    I would like to see a data based comparison of many models including (but not limited to):

    C150 vs AA1
    AA1 vs AA1A/B/C (I have read that the wing makes it a very different plane)
    C150 vs Tomahawk
    C150 vs Ercoupe (how safe is the "safest plane" from a more modern perspective)
    C150 vs C172 (since the C172 is the measuring stick for GA planes)
    C172 vs C150
    It's hard to make those sorts of calls, for a number of reason.

    The first is the fleet size. A lot more Cessna 150s were built than AA-1s. When you look the raw number of accidents, the 150's total is an order of magnitude greater (~3,000 vs. ~300). The same sort of thing holds for comparing it to the Ercoupe or the Tomahawk. If you're looking for the differences between AA-1As, AA-1Bs, and AA-1Cs, you're going to have even fewer samples.

    The second thing is market. Yes, the 150 and AA-1 were both marketed as trainers during the day. But few people have used AA-1s as trainers since then, they are more sporty two-seaters. This is reflected in the median pilot total time for accident aircraft; the Grumman pilots have twice as many hours. The Ercoupe is literally an antique; obviously going to be operated differently from a ratty old 150 trainer.

    So there's a lot of pitfalls if you're trying to make a comparison, and a lot of opportunities to make a mistake.

    The NTSB makes the full accident database available for download:

    https://app.ntsb.gov/avdata/Access/avall.zip

    It's an Access database.

    I have processes developed to help extract my homebuilt data from the database. I've used those same processes to extract Cessna 150 and Grumman AA-1, and have set it in the Fly Baby web page for downloading:

    www.bowersflybaby.com/events.xlsx

    There may be a few extraneous entries in the AA-1 sheet, from trying to ensure all examples were found. Delete those extras.

    And if you say, "Hey, why are a lot of columns blank?" it's because the NTSB has changed formats several times over the years, and data comes and goes.

    Ron "Welcome to my world....." Wanttaja
    Last edited by rwanttaja; 05-30-2019 at 06:13 PM.

  5. #5
    rwanttaja's Avatar
    Join Date
    Jul 2011
    Location
    Seattle
    Posts
    2,948
    Quote Originally Posted by FlyingRon View Post
    Whether you can extrapolate Ron's data into an indication if a particular design is safe, is also opinion. Ron's done a lot of research, but he doesn't make the claims you're trying to make.
    I figured I'd have the peasants coming for me carrying torches, when that article came out.

    Ideally, one would prefer the accident rates between engine times to be based on comparing the total number of accidents vs. the number of each type of engine installed. But the FAA registration database is notoriously bad, regarding recording the types of engines in homebuilt accidents. In the article, I explained how I tried to interpolate the FAA data, using the NTSB data, to come up with approximate fleet sizes for each engine type. Auto engines did come up with a higher fleet accident rate, higher even than that of the airplanes powered by two-stroke engines.

    The other way doesn't require interpolation...once can just compute how often the ENGINE was the cause of the accident, in accidents involving that aircraft type. Here's the results of that analysis.
    Total Accidents
    % of Total
    Cases Involving Engine Problems
    % of Total
    Traditional Engines
    1900
    55.1%
    338
    17.8%
    Auto Engines
    447
    13.0%
    196
    43.8%
    Non-Cert Four Strokes
    472
    13.7%
    82
    17.4%
    Two-Strokes
    414
    12.0%
    148
    35.7%
    Not Categorized
    245
    7.1%
    59
    24.1%
    What it basically says that, over the 20 year period covered by my database, almost half the accidents involving auto-engine-powered homebuilts were caused by some sort of failure related to the engine... vs. just ~18% of the conventionally-powered aircraft. As the table shows, two-stokes WERE worse than conventional engines...but better than auto engines.

    In any case, whichever way one processes the data, auto-engine conversions come out significantly worse than traditional engines.

    Ron Wanttaja

  6. #6
    Dana's Avatar
    Join Date
    Jul 2011
    Posts
    927
    It would be interesting to compare statistics for only homebuilts operating in phase II, to weed out the early test failures, but I doubt there's any data on that.

    Regarding the Ercoupe, I have no reference to back it up, but I remember reading that the anticipated safety record improvement for the "safety plane" didn't pan out as expected. You can't stall or spin an Ercoupe, but if you mush it into the ground it hits pretty hard anyway, and the supposed "safe" nature of the plane may have made pilots complacent... just like when ABS brakes came out (drivers driving more aggressively, relying on the improved stopping power).

  7. #7
    lnuss's Avatar
    Join Date
    Dec 2013
    Location
    Colorado
    Posts
    279
    have heard many opinions, most of them suggesting that the AA1 is a deathtrap, but I have not seen any data to directly compare it to the C150. Where can I find data to make the comparison between aircraft?
    I don't have data, but I do have some experience. The AA1s have a small wing area, and not much flap, and they're not exactly over powered, either. And like all those Grumman American planes, that laminar flow wing is sensitive to speed when climbing, in other words, if you're a little too slow you'll have little climb performance (behind the power curve), especially at higher elevations, unlike the C-150 and many others where the transition between a little too slow and a lot too slow is much more gradual. The original AA1 is the worst, and the AA1A and later had a slightly more forgiving wing, but still not what folks were used to.

    So there's nothing really wrong with the AA1 aircraft (they're sweet to fly), but their handling, especially at lower airspeeds, is considerably different from what most pilots trained on, whether C-150, Cub, Champ, or Cherokee. A Musketeer with its laminar flow wing is closer to that, but still not as sensitive.

    Larry N.

  8. #8
    rwanttaja's Avatar
    Join Date
    Jul 2011
    Location
    Seattle
    Posts
    2,948
    Quote Originally Posted by Dana View Post
    It would be interesting to compare statistics for only homebuilts operating in phase II, to weed out the early test failures, but I doubt there's any data on that.
    Well, there's this:



    This kind of illustrates how quickly the auto-engine issues arise:



    Ron Wanttaja

  9. #9
    Dana's Avatar
    Join Date
    Jul 2011
    Posts
    927
    So the gap gets narrower after the test period, not surprisingly. What's interesting is the rise in accidents after 40 hours... plane is sold, or the owner starts flying to new places.

  10. #10
    rwanttaja's Avatar
    Join Date
    Jul 2011
    Location
    Seattle
    Posts
    2,948
    Quote Originally Posted by Dana View Post
    So the gap gets narrower after the test period, not surprisingly. What's interesting is the rise in accidents after 40 hours... plane is sold, or the owner starts flying to new places.
    Yes, there is a rise in pilot judgment accidents after forty hours, which I call "cross-country errors." Forty hours is the typical Phase 1 period, so folks start going places...and running out of fuel on the way, landing at unfamiliar airports in less-than-ideal conditions, trying to continue VFR in IFR conditions, etc.

    However....There is also a spike in MECHANICAL issues after 40 hours:

    A couple of factors here, I think.

    First...and probably foremost...just because something holds together for the first flight or the first 40 hours doesn't mean it's correct. It may just take a number of flight hours for problems to manifest themselves.

    Second, that "blip" in the figure above is around the typical yearly utilization of a homebuilt airplane. In other words, it's happening around the first condition inspection. Builders receive Repairman Certificates that allow them to perform the inspection because of their familiarity with the construction of the airplane. However, the building experience does not acquaint the builder with how the aircraft systems wear. That's one of the big advantages of a professional (A&P) inspecting the aircraft: He or she may not be as familiar with the construction of that exact homebuilt, but they have experience on how systems *degrade*. I've always had to have A&Ps for my condition inspections. Once, the man arrived, walked up to the wingtip, wiggled the aileron up and down, and announced there was a problem, somewhere. Turns out it a broken weld on the control stick.

    From this, I recommend that builders get a second set of eyes for the first condition inspection. An A&P if you can, or at least another owner experienced in the design.

    Ron Wanttaja
    Last edited by rwanttaja; 05-31-2019 at 11:42 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •