NFP Response Rates Sound a Warning

As I wrote in my Non-farm Payroll preview, the most important result to watch is the response rate. If the response rate is below the normal level, it might imply a higher level of business deaths. If this is true, the standard BLS methodology results in a significant overestimate of payroll jobs.

And that is exactly what happened. While the just-reported July response rate was a solid 77.8%, the second and third estimate for prior months are more important. The reason is that shuttered establishments will not respond to the survey, even two months later.

Let’s check the data. For May, which started the series of rebounding numbers, the third (and final) estimate shows a response rate of 90.7%. To most that would seem quite good, but is actually the lowest level in more than ten years, and about 4.5 percentage points lower than the prior eight-year average.

The significance of this is that, unless a very unusual number of business failed to answer the survey for three months, the missing firms may well be defunct. We do not know the size of the non-responding firms, but if they were typical of the average of the 145,000 surveyed, this would represent an over count of 6.7 million jobs in May.

June data includes only the second response. That is 85.0%, the lowest in over ten years and almost as low as May’s 85.5%. June is about 7% below the average for the second response, so there is a lot of ground to make up by next month.

My analysis strongly suggests that employment is much weaker than the the last three jobs reports suggest. This may lead both investors and policymakers to misjudge the state of the economic recovery.

Addendum: It is a challenge to explain an unusual concept to casual observers. The general response to the response rate being a little low seems to be, “So what?”

Perhaps this example will help. Suppose we conducted a scientific survey of Facebook users — representative and with a suitable sample size. We asked all sorts of questions about age, other activities, reasons for using the site, gender, frequency of use, and other similar variables.

If we were then asked about the age of users, or whether it differed by gender, we could give a good answer. If we were asked the proportion of people who used the platform to stay in touch with family, we would have a good estimate. The survey sample provides a solid basis for inferences about the entire population.

But what is we were asked how many people use Facebook? Our survey cannot help us. It is a question about the size of our universe or sampling frame. We must get this information from another source.

And that is the problem with the BLS method. Surveys do not help in determining the size of the sampling frame.

You may also like

4 comments

  • Tom Summers August 8, 2020  

    Got it. Thanks for broadening my reference frame. I’ll be more careful making inferences on other statistical reports.

  • M.P. Sunday August 8, 2020  

    Thanks Jeff! You are right. I better understand it now than when you first introduce it.

  • Bruce Robinson August 8, 2020  

    Excellent analysis IMHO. That’s why you’re my go-to source for understanding what is significant in the fire-hose of data that showers us daily.

    I suspect there are many, many readers who appreciate your work, but don’t always have time to acknowledge their debt to you. So please never get discouraged……we’re all out here reading.

    Thank you very much.