8
$\begingroup$

What is the standard of evidence that we should require in answers?

Do we simply rely on the upvotes and downvotes to moderate what is true and untrue? What about myths that are widely believed that will gather upvotes regardless of their truth? Or should we be striving to provide authoritative sources for the claims that we make in answers?

I'll take one question as an example: Why tap the brakes on take-off in fixed gear aircraft?

The answers make the following claims:

Spinning wheels are not a problem in fixed undercarriage aircraft, there's no good reason to tap the brakes after takeoff if the gear is nailed down.

  • Says who? Why are we to trust this answer?

I'd say you are probably safe to ignore that in a TB-10, however that is just my opinion and it may be worth contacting a TB-10 club and asking them what they think.

  • Is speculative opinion useful in an answer?

This answer to another question has a link suggesting, that the spinning wheels may be sufficiently large gyroscopes to affect handling of the aircraft.

  • Do we care what a link "suggests"? Isn't what is actually the case what matters? Are the spinning wheels in fact sufficiently large gyroscopes to affect handling of the aircraft?

You have a decent amount of angular momentum fighting any maneuver that the aircraft wants to make.

  • Does any reliable source say that the additional angular momentum of the wheels is relevant to any flight maneuver? Does it warrant tapping the brakes?

On a Cessna 172, spinning wheels do create vibrations.

  • If this is true, why not provide a reference for this?
$\endgroup$
3
  • 1
    $\begingroup$ See also: meta.aviation.stackexchange.com/questions/328/… $\endgroup$
    – Lnafziger
    May 5, 2014 at 11:19
  • 1
    $\begingroup$ You, like me, have spent too much time at skeptics. I find myself backing up every answer with official sources on aviation. $\endgroup$
    – Jamiec Mod
    May 13, 2014 at 15:04
  • $\begingroup$ @Jamiec Haha.. it's part of me now. $\endgroup$
    – user2168
    May 13, 2014 at 15:04

2 Answers 2

Reset to default
6
$\begingroup$

This answer has been dramatically edited. See the edit history for context.

Required standards of evidence for answers should be defined locally (in the question) rather than globally (site-wide).

Rationale:

1. Not all questions can be answered to an arbitrarily high standard of evidence.

Some questions have verifiable, authoritative answers. Some do not. We already have a minimum standard close reason for questions - those that fall below it are closed as "Too broad" or "Primarily opinion-based". This is all that's required.

2. Not all questions need to be answered to an arbitrarily high standard of evidence.

The person who asked this question was looking for an answer with sources. This one was not. The person who asks the question is best qualified to and should be the one to set the standard for what sort of answers are acceptable.

3. There is nothing wrong with implicit standards.

Question askers should not be required to explicitly define the evidence standards for their answers. If they want to, that's fine, but if not, the voting system will tend to bubble better answers to higher positions in the stack (which is as it should be).

$\endgroup$
4
  • $\begingroup$ Another stackexchange (Skeptics) manages to demand a higher standard of evidence without any problems. Authoritative is a hierarchy: with personal opinion at the low-end and meta-analyses and systematic reviews at the high end. $\endgroup$
    – user2168
    May 5, 2014 at 3:59
  • 1
    $\begingroup$ @user2168 - Another stackexchange (Arqade) manages to demand a lower standard of evidence without any problems. $\endgroup$
    – Steve V.
    May 5, 2014 at 4:20
  • $\begingroup$ True. That just means that my question is one that each site needs to answer on its own and not simply point to "This is the internet" as an apparent answer to my question. $\endgroup$
    – user2168
    May 5, 2014 at 4:34
  • $\begingroup$ @user2168 - Agreed, I am guilty of typing without thinking. I'm editing my answer now. $\endgroup$
    – Steve V.
    May 5, 2014 at 4:37
4
$\begingroup$

Simply put, the level of evidence required is equal to the evidence desired by the community.

Generally speaking, we expect answers to be defensible. So if you write something which strikes a user as wrong or improbable, they may ask you for a citation. If your response is something like "I heard it from a guy on [random forum]" then that's not going to cut it. We'll either ask you to remove the claim, edit it ourselves, or you're likely to receive down votes.

This exact scenario happened today when I challenged a claim in an answer, and the only evidence supporting the claim was an anonymous comment on another site. After a short back and forth, the author removed the claim and I deleted the comments. That's the way it's supposed to work. (I'm not mentioning the answer because there's no reason to draw attention to the user after he already did the right thing).

The question: What do the terms 'holding out' and 'common carriage' mean? inspired a very lively debate when a user challenged an already accepted answer. The debate created a long stream of comments before eventually moving to chat, and, after about a day, resulted in everyone coming to an agreement that the accepted answer was indeed wrong, and then corrected. We all learned something that day thanks to a user who spoke up when something seemed wrong, and again, that's how it's supposed to work.

So I guess my advice to you would be: if you are skeptical of an answer, feel free to leave a comment asking for clarification or citations. However, don't get into a long comment thread war with someone - take that sort of thing to a chat room. If you feel like it's simply a bad answer in general, consider down-voting it instead of leaving a comment. At the same time, be reasonable in your requests. It's perfectly acceptable to draw on personal experience for many answers, or portions thereof, and there may not be any way for them to create a citation for that.

In other words, try to use good judgment.

$\endgroup$
11
  • $\begingroup$ Okay, so to make sure I understand your position, you're saying that we should have no specific standard of evidence at Aviation? $\endgroup$
    – user2168
    May 5, 2014 at 4:08
  • $\begingroup$ Could you explain further what you mean by "it's perfectly acceptable to draw on personal experience for many answers". There is no way to know how individual anecdotes generalize to other cases, or even if the individual anecdotes have been remembered and reported accurately. $\endgroup$
    – user2168
    May 5, 2014 at 4:09
  • $\begingroup$ @user2168 we are a very different community from Skeptics, and the nature of the questions are very different. Much of flying, from a pilot's perspective, is either non-scientific, or varies so widely from person-to-person or airplane-to-airplane that there is no way to give a comprehensive answer. So, instead, people will give an answer based on their own experience with the airplanes they've flown, or the companies they've worked for. $\endgroup$
    – Bret Copeland
    May 5, 2014 at 4:23
  • 3
    $\begingroup$ We're a community largely consisting of fellow pilots, so if someone writes something that doesn't seem right, we call bullshit. Beyond that, yes, I'm saying there's no specific standard for evidence. One notable exception, however, is that when the question deals with regulations, you're generally expected to include an excerpt and link to the relevant regulations. $\endgroup$
    – Bret Copeland
    May 5, 2014 at 4:26
  • $\begingroup$ Okay thanks. I understand that many answers are subjective. You point out that regulations, however, are not, and should have references. But, what about claims that are about how the physical world works? Those don't seem subjective either. I would think that pilots would want claims about how the physical world works to be backed up scientifically. But, if this community doesn't want that, that's cool. I can deal with a lower standard of evidence. $\endgroup$
    – user2168
    May 5, 2014 at 4:36
  • 1
    $\begingroup$ Perhaps there is some miscommunication. The point is that there is not one standard which can apply to all of the types of questions we get here. I didn't say personal experience is acceptable for all answers. $\endgroup$
    – Bret Copeland
    May 5, 2014 at 4:42
  • $\begingroup$ @Articuno Physics in flying still has an unusual amount of subjectivity. Take for example the gyroscope bit: We know a rotating wheel acts as a gyroscope, that's basic physics. Logically it follows that "sufficiently large" rotating wheels could have a noticeable impact on airplane performance, which begs the question What is 'sufficiently large'? - the answer to that may differ by pilot: A high-time Connie pilot might notice the difference, someone doing initial training on the aircraft might not, or might unconsciously correct for it having never known different characteristics. $\endgroup$
    – voretaq7
    May 5, 2014 at 22:35
  • $\begingroup$ @voretaq7 Well, it may be so small an effect that not even the most unstable aircraft with a beginner pilot behind the controls would notice. Or, it could be so large that certain manuals recommend using the brakes to stop the wheels from spinning. Or, it could fall somewhere in between. A good answer would state exactly where within this spectrum the spinning wheels on a general aviation aircraft lie, along with a reference establishing that. None of that is subjective. $\endgroup$
    – user2168
    May 5, 2014 at 22:44
  • $\begingroup$ You could test the "noticeability" by monitoring the control inputs with and without spinning wheels and see how they differ for a 10 hour pilot, a 100 hour pilot, and a 10,000 hour pilot. $\endgroup$
    – user2168
    May 5, 2014 at 22:45
  • $\begingroup$ And, if we just don't know the answer, no need to speculate. $\endgroup$
    – user2168
    May 5, 2014 at 22:46
  • 1
    $\begingroup$ @Articuno It's not that the answer is "unknown", it's that there are many possible reasons, all with some level of validity/correctness: Gyroscopic rigidity? Totally plausible with big enough wheels (Tundra tires on a Cessna 150? Maybe ; Standard gear on a PA28? Probably not). Vibration from the gear turning? Sure - I've felt it myself with a dry bearing or an off-balance tire. Re: the idea of doing controlled experiments, it's an interesting one, but there are a lot of challenges (stop by chat and I'll be happy to talk about them) $\endgroup$
    – voretaq7
    May 5, 2014 at 23:09

You must log in to answer this question.