Critics of the BLS Birth/Death Adjustment Proven Wrong
The actual job count is in. It shows whether the much-maligned birth/death adjustment improved the monthly job count, as the expert BLS team contended, or whether they government was creating "fictional jobs" as maintained by the many vociferous critics.
Result? A slam dunk for the BLS!
The critics were completely wrong. Month after month. Article after article. Since their efforts were so pervasive, and the government does not go on TV to refute the misinformation, the entire investing and trading world has been misled.
At this point nearly everyone believes that the BLS creates phantom jobs through this adjustment. It is time for people to learn the truth.
Background
Each year the BLS makes a "benchmark revision" to the payroll employment series based on the establishment survey. The purpose of this is to make sure the survey data are consistent with the actual count of jobs from state unemployment insurance tax records.
The state data is much better, of course, but it is not available in a timely fashion. The benchmarking is a reality check. It allows the BLS to see how well they did with the monthly estimates. Each October, along with the report on September employment, the BLS releases the preliminary version of these benchmark revisions.
This is the report card for the BLS.
The Vocal Critics
Each month a chorus of highly vocal critics, mostly people who are neither economists nor experts in modeling, tells us what is wrong with the BLS method. For several years the focus of their criticism has been the birth/death adjustment. Briefly put, the critics do not see why an adjustment should ever add jobs at a time when the economy is weakening. Their language is very colorful and persuasive, often featuring the claim that these are "phantom jobs."
We have repeatedly explained the job creation estimates. Nearly 2.5 million jobs are created every month. The BLS method has a process that estimates most of this job creation in new businesses from the behavior in old businesses. It captures economic effects pretty well. The birth/death adjustment is relatively minor when compared to the primary method of estimating job creation.
While we have written many articles on this topic, interested readers should check out this piece from July, 2007. We realize that most readers do not click through to read prior work. This time you should. You will see why and how most analysts got this wrong.
And the result……?
The aggregate error in estimated job change for a full year period ending in March, 2008 was 21,000 jobs. That's right, only 0.02% of the labor force. Without the birth/death adjustment the error would have been 761,000 jobs.
For some time we have challenged critics to show any year in which the birth/death adjustment made the results worse. This report, How the Business Birth/Death Model Improves Payroll Employment Estimates, shows both the results and some excellent analysis. While anyone interested in analyzing government data should read the entire report, here is the key summary in a handy chart:
This chart makes it easy to interpret. The blue line is the actual count. Just compare the red line to the green line. The red line shows what the estimate would have reported without any birth/death adjustment. The green line shows the effect of birth/death.
UPDATE on EXPLANATION
While readers got the point of the original article, our original explanation of the chart was not very good. The BLS takes the actual count from unemployment statistics and makes a benchmark revision. The smaller the revision, the better the original estimate. The original estimates use the birth/death adjustment.
With this in mind, the red bars show the revisions using the adjustment. The green bars show what the revision would have been without the adjustment. Since the red bars represent a smaller adjustment in all cases, this shows that the B/D adjustment improved the real-time estimate. The blue bars show, for reference, the amount of the B/D adjustment.
Conclusion
It is obvious that the birth/death adjustment has improved the estimate of total jobs in every time period since it has been implemented.
This conclusion is dramatically different from the current market perception, and it deserves further consideration.
We are looking forward to comments from the misguided critics of this approach, to see whether they 'fess up. Anyone who talked about "phantom jobs" and who wants to maintain credentials to comment on payroll employment should have an explanation about this result.
A Hint
The error made by everyone was not realizing that the BLS estimate of job creation is mostly looking at current firms and imputing that to new businesses. The birth/death adjustment was not and is not the main estimate of job creation. It is an adjustment factor, applied after other methods, including those reflecting cyclical changes, have been applied. Once again, check out this post for the explanation.
Thanks for putting this up. It ends my questioning. My main concerns had come from how they developed the annual estimate, which this answers. Less important is how they apportion it over the year, which still seems squishy to me, but if it is right over a one year period, then I don’t care much about the month-to-month.
The Birth Death Model used to be a minor adjustment in BLS NFP data. It has been around for many years. Since ’06/07, it has become a major statistical nuisance.
The theory behind it was that small businesses are under represented in the Establishment survey. Much of the job creation comes from these smaller firms. The 2001 changes (effective 2003) BLS significantly modified the B/D adjustment. They went from a minor statistical adj to overweighted.
There were two methodological problems with this adjustment: 1) It took a primarily MEASURED data series and introduced a significant amount of extrapolated/modeled data into it (deducing new NFP job from new state Incs).
2) The model improved accuracy somewhat in the beginning of the economic cycle, but at the cost of making BLS data much less accurate at the end of the economic cycle.
Example: In 2007, about 75% of the new jobs created were due to the B/D adjustment.
~~~
Since someone challenged my integrity about the motivations for my B/D adjustment criticism, allow me to point this out: I run money, and don’t care about the ideological or political battles between economic theories or politicians. I only want to protect my clients capital and generate positive returns when opportunities arise.
I cannot say the same of the people who were the staunchest defenders of the BD adjustment.
Indeed, many of the people who took the BLS data at face value, who did not delve beneath the headlines, and approached such data interpretation in a no-critical fashion got mangled in the markets.
Those “analysts” who did not skeptically approach BLS inflation data, NFP jobs info, B/D adjustment — were way too bullish going into 2008. They believed the official data as well as the spin, if you followed the advice of less critical thinkers, you lost tons of money . . .
Barry – Thanks for taking the time to stop at “A Dash.” I am not sure who is questioning your motives, but I do not. If I did not think you were seeking the truth and serving investors, you would not be on my list of featured sites.
On the B/D adjustment, and lets stick to that for the moment, I think you are missing an important point. The BLS needs to account for about 2 million new jobs created each month. I have explained on several occasions how they do this — mostly by looking at behavior in existing firms and extrapolating to potential new companies through a function.
When they are finished with that — and it is the most important method of imputing job creation — they do the B/D adjustment. They warn you not to look at it by itself.
I really think that if we discussed this I could convince you and you could use your strong voice on the side of truth.
Meanwhile, I have a simple challenge to you and other B/D critics. I made this challenge well over a year ago, and no one has taken it up.
Just show us a single period where the B/D adjustment made the job measurement worse. We have the state employment data, so we can see who was right. The evidence comes in six months later.
I think it shows that your early criticism was was wrong, but we all make mistakes.
It is a simple challenge. Go to the state employment data. Show where you were right and the BLS was wrong. If you cannot do that, admit your error.
It is about data analysis and evidence.
I look forward to your reply.
And thanks again for your comment!
Jeff