In the summer of 1946, at the ‘Atomic’ session of the United Nations Statistical Commission (UNSC), a representative of a British colony made an impassioned plea to set globally accepted standards for conducting large-scale sample surveys. Prasanta Chandra Mahalanobis argued that household surveys would become an invaluable data source for many developing countries that were not fortunate enough to have the kind of rich administrative datasets that claimed advanced economies.
Mahalanobis’ suggestion was accepted, and given his unique experience in conducting such surveys at the Indian Statistical Institute (ISI), he was asked to chair the UNSC’s first sub-committee on sampling. The Global Manual on Sampling, which the world would follow for the next few decades, was largely indebted to the work of Mahalanobis and his colleagues at the ISI. ISI’s innovations – such as using a pilot survey to test the survey and the use of replicate samples to ascertain the magnitude of different types of errors in the survey – were adopted around the world.
Mahalanobis’ contribution to the global statistical body went beyond the standardization of sampling criteria. When the UNSC set up a committee to prepare a manual for the computation of national accounts (through which a country’s GDP or GDP is calculated), Mahalanobis’ dependent and a national income expert, Moni Mohan Mukherjee was asked to join the task force. Mahalanobis presided over the 8th and 9th sessions of the UNSC in the 1950s, and attended all its sessions until his death. When he died in 1972, the UNSC condolence note said it had lost its “prestige”.
Mahalanobis’ intervention on the global stage was not designed to appropriate the tag of ‘Vishwaguru’ for himself or his institution. This yawning was driven by a desire to fill in the data gaps faced by India and most of the developing world, without blindly imitating the statistical norms and practices followed in the West. Ronald Fischer—arguably the greatest statistician of the 20th century—once said that what impressed him most about Mahalanobis’s work was that it was not “imitation”.
Like most intellectuals of that era, Mahalanobis had a strong belief in planning as a means of growth and development. Good plans required good data, and this prompted him to set up several data gathering institutions in the first few years after India’s independence. The National Income Unit of the Ministry of Finance, headed by Mukherjee, was divided into a separate department called the Central Statistical Organisation. In order to augment the initial surveys of the ISI and to fill the data holes in the calculation of national income, the National Sample Survey (NSS) system was established. The Annual Industry Survey (ASI) was set up to collect regular data on industrial activity.
Mahalanobis’ views on the plan have not been good. But their statistical interventions have stood the test of time. The way India and much of the world collects data is shaped by his fundamental ideas. Where Mahalanobis (and India) led, the rest of the world followed, Nobel-winning economist Angus Deaton wrote in a 2005 article.
Mahalanobis had a few things in store for him. Born into a prominent Bengali Brahmo family, his social network was an important asset. He was close to Rabindranath Tagore, at whose house he was the first to spend long hours with Jawaharlal Nehru. As chairman of the Planning Committee of the Congress, Nehru was concerned about the data gap long before India’s independence, and sought Mahalanobis’ consultation on ways to fill such gaps.
Another important factor was the unwavering support of Chintaman Dwarkanath Deshmukh. As a civil servant and central banker, Deshmukh helped Mahalanobis access government funds so that the ISI faced repeated financial crises in its early years. Later, as independent India’s finance minister, it was Deshmukh who helped fund Mahalanobis’s ambitious survey plans. Deshmukh told the professor of ways to deal with the labyrinthine British bureaucracy and later wrote its Indian version, Mahalanobis biographer Ashok Rudra.
Despite these advantages, it was never an easy ride for Mahalanobis. Rudra documents the struggle of a lifetime to stabilize the finances of the ISI. By the end of 1954, Mahalanobis felt disheartened enough to consider resigning as its director. “My struggle has been mostly against a machine that is impersonal, and unable to respond to changing needs,” Mahalanobis wrote in a letter to Deshmukh (then finance minister as well as ISI president). ,
Historian Nikhil Menon in his recent book Planning Democracy documents Mahalanobis’ struggle to get computers to handle the ‘big data’ of his era. Mahalanobis realized that without computers, NSS data could never be tabulated in time. He moved heaven and earth to get them. The first digital computer in the country was installed at ISI in 1956 to process NSS data. Soon it was processing data for other institutions as well.
As we celebrate the 75th anniversary of India’s independence, we should be proud of the statistical achievements of a poor country in the early years after independence. But we must also ask why India’s statistical system no longer enjoys the respect it had in the early years of the Republic.
This is the first of a five-part series on the founding fathers of India’s once famous statistical system.
Prameet Bhattacharya is a Chennai based journalist. His Twitter handle is pramit_b. Is