Remembering the ‘prophet of big data’
Statistician laid the groundwork for equations that determined modern economic policies
Theodore W. Anderson, a statistician whose work brought a new mathematical rigor to economics and social science in the postwar years and helped pave the way for modern econometrics and data analysis, has died in Stanford, Calif. He was 98.
The cause was heart failure, his son Robert said.
Anderson, the son of a Minnesota minister, had a distinguished academic career. He studied at Northwestern and Princeton, did research at the University of Chicago and taught at Columbia and, for more than two decades, at Stanford University. During the Second World War, at Princeton, he did war research work on long-range weather forecasting, gunfire strategies for battleships and explosives testing.
But his signal work began immediately after the war, when he joined pioneering efforts at the University of Chicago to help shape postwar economic policy decisions.
The work involved developing systems of mathematical equations that would reveal underlying structures in the economy.
Those systems evolved into econometric models widely used today by the Federal Reserve and other central banks.
Anderson also made advances in the analysis of data in psychology and the social sciences.
His talent for reducing complicated systems to their mathematical essence helped him strengthen tools for dealing with the sorts of large data sets common in the social sciences, in which researchers look for patterns within a broad swath of information collected on individuals.
In one instance, in 1961, psychologists applied these methods to data derived from detailed personality questionnaires and concluded that individual personalities could be summed up by an overarching “big five” traits.
Anderson’s contributions lay the groundwork for an advanced analysis of data sets containing not just dozens but thousands of variables.
“Among other things, I would say that Ted was a prophet of the big data era,” David Donoho, a professor of statistics at Stanford and a longtime colleague, told the Stanford University news office.
Anderson’s book An Introduction to Multivariate Statistical Analysis (1958) remains a classic in the field, educating generations of statisticians in the conceptual underpinnings of a particularly challenging kind of data analysis.
In multivariate data, many variables are considered simultaneously rather than one at a time. Using this type of analysis in medicine, a person’s health is gauged not just by blood pressure, for example, but by blood pressure in tandem with weight, cholesterol levels and heart rate.
Anderson’s work, including his book The Statistical Analysis of Time Series (1971), was at the forefront of this new field, John Taylor, an economics professor at Stanford and a former doctoral student of Anderson’s, said in an interview.
Time series analyses are now “very commonplace in economics,” he said, “and Ted’s work helped move that forward.”
Anderson’s early interest in economics and social science applications stemmed from his goal of “doing some good,” he said in the 1986 interview.
“While I find intellectual interest in mathematical statistical problems,” he said, “I think a final objective is to have an effect on analyzing data and decision making under uncertainty.”
Although he retired from classroom duties in 1988, Anderson continued to give talks, attend seminars and do research from his home office. He recently submitted a technical paper and, just days before his death, was responding to peer reviewers’ comments.
His son recalled him saying, “Well, they’re wrong, but that is a good idea for a followup paper.”