A few questions to colleagues more knowledgeable than me.
- Context: software / systems design & development.
- Assumptions: I'm focused here on pure misestimation risk. I know that projects rarely die because of misestimation risks, but political and business stuff is not usually quantifiable.
- We all know that our duration/cost estimates are usually off by a wide margin. I wonder what real-world quantitative data are available on the distribution of the error. Is it Gaussian with a definite offset? Or non-Gaussian with a definite and stable positive skew? In Taleb's terms, are we in Mediocristan or Extremistan here?
- On the nature of distribution again. Let's forget for the moment the productivity differences between individual contributors (well known to be huge). Do we observe significant outlier events in software development, e.g. an estimation error of several orders of magnitude? — provided that we implement the basic process hygiene, i.e. our stories are small enough, we use consensus techniques like planning poker, we're on a familiar ground technologically, etc.
- Has anyone ever tried to apply the math usually used in various financial pricing models (either poor old CAPM or some more sophisticated stuff like Black-Scholes and all its descendants) to project estimates? Any references?
- In the same vein, but more general, have there been any attempts to analyze projects using analogies with derivative instruments (e.g. viewing a fixed-price bid as a swap of some kind)?
Thanks all for any answers.