Wednesday, December 8, 2010

Permission to model: denied!

Chaeli Judd and Kate Buenau sent along the following criteria for deciding if you are permitted to use a statistical or other modeling method. The answer to all three questions must be yes, preferably with concrete proof.
  1. Can you, personally, get a computer to do it?
  2. Can you explain the method to a person that doesn’t already know how to do it?
  3. Do you understand when not to use it?
Criterion #1 was inspired in part by this quote from a paper by Carl Walters and co-authors:

"As we tell participants in introductions to Adaptive Environmental Assessment and Management (AEAM) workshops, things you can get away with on paper have a nasty way of coming back to haunt you when you try to represent them clearly enough that a computer can reproduce the steps in your reasoning."

I think #3 is really important self-discipline - we should ask ourselves this all the time. Why would we not use _____ for this problem?

3 comments:

  1. These criteria appeal to me! I have always sought to use quantitative tools that I understand from first principles. I've phrased it as needing to understand the underlying assumptions, which helps me deal with #3.

    But I sometimes wonder if I am unfairly elitist in my judgement of who should be "allowed" to use quantitative methods and tools. I derive a lot of satisfaction from the modelling process but I am in the minority. For many scientists (including social scientists), their passion lies elsewhere and they just want to understand the stats/software/black box enough to get an answer. I think us modellers need to be careful not to send a "DON'T TOUCH!" message to these people, and work out constructive ways to promote collaboration and quality modelling.

    ReplyDelete
  2. Great points! Of course, applicable to more than just ARM, but I think #2 is especially important for adaptive management implementation. If you can't explain the computer-based decision process to (1) a class of senior undergraduates or (2) state wildlife ageny personnel, then you might face the real chance your work will never be implemented after you (and your computer) walk away.

    ReplyDelete
  3. Cindy - I agree that we don't want to tell people they shouldn't come to grips with models and quantitative methods. I have spent a lot of time conducting "rapid prototyping" workshops with all kinds of people developing simple models, and I regularly teach a course in statistics. However, I've become aware of a kind of moral hazard when teaching cutting edge tools to people that, well, don't want to take the time to be able to do #2 and #3. They often end up doing inappropriate things with those powerful tools.

    LAP - too right!

    ReplyDelete