Three Common Mistakes in Interpreting the Employment Report

There are three things that nearly everyone does wrong in the monthly discussion of the employment report.  The first two are comments I made earlier today on RealMoney.

Forgetting that it is a survey.
I am mystified by the apparent willingness to draw big conclusions from
a single month’s employment number. The non-farm payroll report is
based upon a (voluntary) survey. The responses come in over a
three-month period, and that is the reason for revisions. When all of
the responses are in there will still be a 90% confidence interval of
+/- 100K jobs. Think about it. The actual job growth for this month
(something we will NEVER know) could be a lot worse, or a lot better —
and this is after the two revisions. This range does not include
non-sampling issues like business creation.

And the discussion of the "internals"? These are also the result of a survey with a large margin of error.

The right way to use this report is to take several months
together and put it as part of an overall collection of data. Today’s
ISM number, for example, is consistent with GDP growth of about 3%. The
household survey showed that unemployment dropped a little. Other
elements we find to be correlated with job change are the four-week
average of initial claims and Michigan consumer confidence. The job
growth consistent with the other data is about 80K.

It is interesting to note how people highlight the wide
confidence interval when something like home sales data come out, yet
ignore it on the jobs report. The problem is that the BLS is trying to
estimate the total number of payroll jobs (about 140 million) each
month, and then subtract one month from another. Getting within 100K by
this method is excellent work, but it does not provide the level of
precision that we would all like to see.

As you watch the discussion today, you might not find anyone who even reminds you of the confidence interval on this survey.

Making Bogus Claims about Revisions.  I have now seen several pundits assert that this month’s jobs number
will be revised higher. No one seems to understand the method used or
how the revisions are made. The revisions come as more replies from
employers dribble in. Unless there is a reason to believe that late
responders differ systematically from "on time" responders, there is no
basis for predicting the direction of revisions.

Missing the Point on Business Creation.  This is the time that the BLS does the annual benchmark revision, where they square up their own data series with actual job counts from state agencies.  This is a good opportunity to check up the critics of the BLS and the Birth/Death adjustment.  I think that they will ask a question like the following:

Did the BLS, using the Birth/Death model overestimate the number of jobs?

They might also ask the following:

How much of the net job change for the year (some will erroneously call this "job creation" which is something else altogether) was due to the Birth/Death adjustment?

The questions that we all should be asking are the following:

Did the Birth/Death adjustment improve the estimate of the number of payroll jobs?

If the Birth/Death adjustment had not been used, would the undercount of jobs have been even larger than this year’s overcount?

It is the job of the BLS to use the best possible methodology.  Anyone who wants to criticize those methods should start by asking how well they worked, and what would be the alternative.

These are the right questions to ask, but you have not seen them anywhere else.  We will follow up, and provide the answers, when we have more time with the data.