This is a joke that I heard many times and once on a big stage at the 2014 annual meeting of the Verein für Socialpolitik where some supposedly important person from a supposedly important central bank (if I recall correctly) used it as a criticism of current economic methodology (as this person understood it) and generalizing it to mean it as a criticism of any economic methodology that uses math (if I understood this person correctly).
The joke goes like this. A police officer patrols the city at night and finds a perhaps slightly inebriated person apparently looking for something under the dim light of a street lamp. The police officer approaches said person and inquires: “Are you looking for something?” The perhaps slightly inebriated person responds: “Yes, I am looking for my keys.” “Where did you lose them?” the police officer asks. To which the perhaps slightly inebriated person answers: “Over there.” loosely waving at a bunch of bushes in the distance. “Why aren’t you looking for your keys over there then?” the police officer wonders out loud. “Well I only have light here” the slightly inebriated person replies.
How does this apply to research methodology in economics? Think of the slightly inebriated person in the joke as your economic researcher (now you see why this person had to be slightly inebriated). Think of the keys as the answer to a research question and think of the light as the research methodology that the economic researcher applies. The economic researcher is thus just as unlikely to find the right answer using their methodology as the slightly inebriated person is to find their keys.
I like this joke because it does ring true. I am sure it is a valid criticism of thousands of research articles in economics every year. But I do not come to the same conclusion that I believe the supposedly important person from a supposedly important central bank came to which is that, if I understood correctly, we should abandon serious mathematical modelling in economics (in favor, I believe this person indicated, of large scale simulation studies with intricately interwoven agents who all behave mechanically according to some simple heuristic). A brief aside: I do not mind if some people try simulation as a means of getting to an answer. I did not intend to here write a criticism of simulation as a methodology, although I am not overly optimistic of its usefulness. Simulation, as I see it, can only ever provide a fairly dim light. But it could on occasion be in just the right place to find the beginnings of an answer. But I believe that when we find a mathematical model to be unhelpful we should not abandon math completely but we should try and find or develop a more appropriate math to deal with the situation.
The more math we have at our disposal the more stats we have at our disposal the more light we have and the more likely will it be that we will find answers to our economic problems.
A final note, perhaps, and again very much only my opinion: the more brilliantly creative and intuitive you are the less you may need to know the tools. But then again who is brilliant?