Monday, May 4, 2009
Use of proxies in science
I am working on Chicago economics of the 1940s and 50s. One interesting methodological feature of the program is that it was eager to connect empirical-statistical research to theoretical development. This meant it was suspicious of (among other things) overly formal mathematical (general equilibrium) models and over reliance of econometric technique. Now when faced with the (large) gap between general theory and messy or theoretically malformed data, they did not turn (primarily) to modeling (the focus of much recent philosophy of science). Instead, the Chicago economists developed empirical proxy measures on a case by case basis. Obviously, the application and reliance of proxies involves many complications. Yet it seems to be a standard practice in science. (I am aware of use of proxies in 17th century physics and 18th century economics.) Now my question to readers of this blog is this: can anybody recommend any philosophical work on proxies? Anybody have any interesting ideas on proxies? I would be much obliged.
Subscribe to:
Post Comments (Atom)
Eric,
ReplyDeleteAs replies are not apparently forthcoming, I'll go ahead and ask the dumb question.
What's a proxy, in this context?
-Kris
This comment has been removed by the author.
ReplyDeleteI guess another terminology would be, indirect measurement. A measure is constructed that can give indirect access to the thing you are interested in. (I think of FMRI's as an important example of this, although people are getting so used to the bright pictures that it may become seen as measurement.) The measure often is 'derived' from theory, but these inferences are in the cases at hand not straightforward (and involve all kinds of assumptions [this is also true in FMRI software). If faced with necessity of indirect measurement, one would try to construct several such indirect measures (or proxies) that one hopes are independent from each other and mutually illuminating.
ReplyDeleteHi Eric,
ReplyDeleteI wonder if the distinction you seem to be drawing between direct and indirect measurements is sufficiently robust. I cannot think of any example of genuinely direct measuerments. At most I can think that there are more or less indirect ways of measuring a certain quantity.
Thermometers, for example, do not seem to measure temperature "directly", as thermometry relies on a number thermodynamical assumptions and even the seemingly innocent measurements of the length of objects by dint of a ruler seem to be subject to a number of far from innocent theoretical assumptions.
(Bas van Frassen's new book Scientific Representation has an excellent chapter on this)
Just as an aside, an area where proxies are used to great effect is paleoclimatology. In order to reconstruct past surface temperatures, one must use ice cores, tree rings, satellite measurements, etc. and integrate them statistically. The infamous "hockey stick" controversy was in part over the reliability of proxies and how to statistically combine them.
ReplyDeleteEric-
ReplyDeleteNot sure I know what exactly you have in mind, but you might look at Megan Delehanty's paper ""Perceiving Causation Via Videomicroscopy" (2007) Philosophy of Science 74(5):996-1006.
In relation to proxies, it might be worth mentioning Goodhart's law.
ReplyDeletehttp://en.wikipedia.org/wiki/Goodhart%27s_law
In re Gabriele's point: I think we can probably construct a clear if somewhat relative notion of proxies vs. "direct measurements".
ReplyDeleteSuppose we want to know how many people are in a given room. We could go in "directly" and count them. Or we could watch the exits and entrances. (Starting either with an empty room or a head-count.)
If the value we want to measure is sufficiently dynamic (if lots of people come and go) repeated head-counts will be less precise even though they are less direct. By the time we finish counting, the value has changed, some have left after being counted, some have arrived and have not yet been counted.
We have to assume that there's no other way in or out (and no one is getting vaporized inside), of course. But such assumptions are, like Gabriele says, just part of measurement in general. The distinction between the proxy and the value we want to observe, however, seems pretty clear (as does the sense in which it is a proxy).
The utility (and necessity!) of this sort of proxy in economics and social science seems obvious to me. Though I had not put a word on it before reading this post. I, too, would be interested in some references for further reading. Not least some of the example from the Chicago school that Eric is thinking of.
I have been traveling, so have had no time to respond to these interesting comments.
ReplyDeleteSeamus--yes, that's terrific!
Jay (hi!)--I find that case fascinating. (I was introduced to it by Alison Wylie.) But I find to distinguish between the problems of measurement by proxies and statistically combining proxies. Let's call the latter issues of robustness (in Wimsatt's sense). I actually think that if one can combine more than one proxy, one is in better shape.
Thomas/Gabriele--thank you. I incline toward Thomas's position. I hope to add a bit to this later today or tomorrow.
Looking forward to it. I notice I should have said: "...repeated head-counts will be less precise even though they are MORE direct."
ReplyDeleteOkay, finally, I have some time.
ReplyDeleteOn Delahanty's paper; that's interesting stuff, but it is not very illuminating on how to think about different kinds of measurement. (I see it more as contributing to deflating claims about how certain representations give unique access to causation.)
On Gabriele's point. Once again, let me grant Gabriele that all measurement requires extra (background, simplifying, normalizing, theoritical, etc) assumptions. This can have significant impacts on the margin of error in accuracy quality of data; much science concerns itself with elimination of possible sources of error in measurement. Nevertheless (yes?), there is a qualitative difference between such practices and situations when there is no practical possibility of measurement of the (theoretically significant) desired information. So, for example, in the 1940s and 1950s, "Chicago" economists (Stigler, then at Columbia, and Warren Nutter, who became (in-)famous for his analysis of Soviet economy) were very interested in the nature monopoly. In particular, they were interested in learning if large-scale monopoly was an inevitable outcome of modern industry or if (as they suspected) government policy actually partially contributed to rise of monopoly. Economic theory tells them that they should try establish “long-run elasticity of demand," but there was no way of measuring that. In response, Nutter and Stigler develop indirect measures by estimating the level of concentration by industry (and then estimate changes over time). This involved considerable subjective judgment and no protocol or rule-following could eliminate that; moreover it is by no means obvious that estimating industry concentration gives one access to theoretically relevant factors. (Ironically, their empirical approach, which changed the terms of debate about anti-trust policy in 1950s, got undermined by the employment of highly formalized toy-models of the economy by their Chicago colleague, Harberger, but that is another story.)
Just wanted to let you know, that I've been thinking about the ramification of the use proxies in science, and how they can be problematic. I am particularly interested in statistical noise issues (I'm coming at this from the Global Climate model angle) but have started to look at it from the Goodhart/Heisenberg's principle. I've been focusing on how the act of measuring can change what one is attempting to measure, and thinking about how theoretical models can cloud judgement about data and/or the use of proxy data. The tie in to economics is fascinating to me and there may be a tie in to the philosophical concepts of "emergence".
ReplyDeleteHi David,
ReplyDeleteLook forward to your futher reflections on this.
Climate modeling has a lot of use of proxies, so it is a fruitful area to think about the topic. Much of the focus in philosophy of climate science seems to be on simulations and/or values/politics & science.