Luxembourg is not known for its world-beating exports. You could cross the country by car in an hour and hardly know it was there, unless you travelled by taxi, in which case it would leave a large hole in your wallet.

But if you stopped a stone’s throw from France and approached a cluster of colourful blocks that rise out of an old industrial site, you would be close to one of the country’s best-kept secrets.

The Luxembourg Income Study (LIS) at the Cross-National Data Center does what it says on the tin, only in a more thorough fashion than any other similar data repository in the world. The centre, founded more than 30 years ago, hosts the world’s largest income database, with hundreds of datasets from 46 countries from as early as 1967.

The microdata held by LIS includes anonymised individual entries from the respondents to each survey and affords an in-depth look at the finances of particular cohorts of the population, over time, in as broad a range of countries as possible. The database is typically only accessible to academics and is used by World Bank, the IMF, the OECD, the International Labour Organisation and many UN agencies.

After a number of discussions with Janet Gornick, the director of LIS at the Graduate Center of the City University of New York, the Guardian travelled to Luxembourg to review the database and develop a set of queries to access the numbers required for the project.

Selecting the data

Ten countries were selected for our initial analysis: Australia, Canada, France, Germany, Italy, Norway, Spain, Sweden, the UK and the US. The longest possible selection of data was taken, from 1978 to 2013, and the information was accessed and harmonised by a data team at LIS.

Although we worked with many of the hundreds of variables, we opted to narrow our focus to those that occurred most frequently. Our research focuses on equivalised disposable household income weighted (EDHIW) and personal income labour weighted (PILW).

Another consideration is the type of household to study. Typically, older surveys gathered the most detailed information about the income of the head or spouse of the household, a definition that varies from country to country. As a result, income and other variables are often reported on a household level rather than showing what each individual earns.

However the framing of the Generation Y project – which looks at the problems facing millennials, born between 1980 and 1994 – imposed additional demands on the data held by LIS. One key group we wanted to understand better was adult children living at home with their parents. In most surveys, information on this cohort is difficult to accurately assess. Their earned income is counted along with their parents and spread out equally for the whole household, which makes them appear far richer than their peers.

In order to better understand the income for this age group we requested the data in five groups, relating to the overall population by age; single households where the householder is a particular age; multi-person households where the head or spouse is of a particular age; children, related to the head or spouse of a household, defined by age; others in a household defined by age. By looking at the children in a household by age we were able to identify young adults still living with their parents.

However, in our analysis we opted to look at figures relating to the households as defined by the age of the head or spouse and the national average – a snapshot of how people are doing, regardless of their living situation.

The composition of the population by age household type and age group was requested for all the data. This allows us to analyse the typical household structures found in each age group, survey and country over time.

Sample sizes

All the data is comparable based on the purchase power parities (PPP) index using US dollars as the baseline. This takes into account variations between different currencies and inflation in the surveys over time. Therefore all the reported figures are in real current US dollar PPP terms.

In the countries selected, the number of people included in a survey sample varies from 18,000 to 220,000 for the countries analysed. In order to ensure the sample sizes remained statistically significant, only age groups with more than 30 respondents were included in any analysis.

As income in most cases is measured by household, a scale of equivalisation must be used to allocate income to all of the individuals in the household. The scale used is called the OECD modified scale and is the standard equivalence scale for Eurostat, the Department for Work and Pensions and the Office for National Statistics.

The scale allows you to divide income among all the people in a household by allocating a different weighting to the individuals living there. Single-person households are treated as the reference group with a value of 1. In multi-person households each additional person is given an equivalence weighting of 0.5 for those aged 14 and over, while each child under 14 is give an equivalence weighting of 0.3.

Data analysis

Five-year cohorts were selected for the first round of analysis. The cohorts started with those aged under 15 and increased in five-year cohorts, until the age of 80 and over, and were reported in the five categories as defined by individuals within the household and national averages.

Although the generational cohorts had already been defined, the five-year cohort approach was executed in order to retain a more fine-grained view of the changes in income over time, while retaining statistically significant sample sizes.

Household composition percentages, population samples, mean and median data were extracted for each cohort group.

The Guardian’s analysis of the data was carried out in Excel and R, using the ggplot2 package for visual analysis, linear regression and the summary function for descriptive statistics.

Click here to view original web page at

How we revealed the predicament of Generation Y
Reader Rating 4 Votes