NHTSA "CODES"
 
Study Crash Outcome Data Evaluation System Analysis by the AMA
From: Sean Maher - American Motorcyclist Association Date: January 3rd, 1996
 
The Crash Outcome Data Evaluation System (CODES) Study: Comments and
Observations from the American Motorcyclist Association. The National
Highway Traffic Safety Administration (NHTSA) recently released a Report to
Congress, The Benefits of Safety Belts and Motorcycle Helmets: Based on Data
from The Crash Outcome Data Evaluation System (CODES). NHTSA was provided
with five million dollars to fulfill the requirement for this report
contained in Section 1031 (b) of the Intermodal Surface Transportation
Efficiency Act of 1991 (ISTEA): "(1) In General. -- The Secretary shall
conduct a study or studies to determine the benefits of safety belt use and
motorcycle helmet use for individuals involved in motor vehicle crashes and
motorcycle crashes, collecting and analyzing data from regional trauma
systems regarding differences in the following: The severity of injuries;
acute, rehabilitative and long-term medical costs, including the sources of
reimbursement and the extent to which these sources cover actual costs;
government, employer, and other costs; and mortality and morbidity outcomes.
The study shall cover a representative period after January 1, 1990." This
study extends beyond fatalities to include an analysis of non-fatal injuries
and their related costs for treatment. In addition to reporting the average
inpatient charges associated with the treatment of injuries, NHTSA provides
'effectiveness' measures as an overall assessment of how the use of a safety
belt or helmet can reduce injuries or prevent death. Voluntary helmet use
advocates will undoubtedly be confronted with elements of this report as
they preserve or work to restore voluntary helmet use within their
respective states. It is important to be familiar with the full study,
rather than the parcels of information that the press is repeating. Doing so
reveals that NHTSA's best attempt to build a case for mandatory helmet laws,
has fallen well short of the mark. Strategies The following comments are
offered for your consideration as you develop strategies to respond to
CODES: Perhaps one of the most important facts to remember when evaluating
CODES is that it is not nationally representative. NHTSA states on page 14
of the CODES Report to Congress, "The reader should note that the overall
results presented in this report reflect only the 7 CODES states and the
case selection criteria for the safety belt and motorcycle helmet analyses.
They are not intended to be nationally representative." New York contributes
nearly forty percent of the motorcycle rider data. It is likely that this
over-representation increases the average inpatient charge estimates. Only
six of the seven CODES states are used for the helmet analyses: Hawaii,
Maine, Missouri, New York, Pennsylvania and Wisconsin. Utah participated in
the safety belt study, however their data is excluded from the helmet study.
Of the states contributing data to the helmet study, New York's data
represents nearly forty percent of the study sample. This likely results in
considerable bias in the reported economic results. NHTSA concedes on page
14 of the CODES Report to Congress, "Because these methods of computing
averages are strongly influenced by number of cases, data from larger states
may disproportionately influence the overall weighted average." To
illustrate the limited applicability of this report and the effect that one
state can have on the results consider the following: The draft Report to
Congress did not include data from Pennsylvania. The draft Report to
Congress indicates motorcyclists were less dependent (19 percent) upon
public sources for medical costs than motor vehicle operators (21 percent).
The final Report to Congress includes Pennsylvania's data. The final Report
to Congress indicates motorcyclists are more dependent (23 percent) upon
public sources for medical costs than motor vehicle operators (16 percent).
The addition of one state causes a significant shift in the outcome. This
underscores the wide margin for error inherent in this report, reinforces
the fact that states are unique and that transportation issues should be
evaluated within that context. The 'effectiveness' ratings for helmet use
are inflated. NHTSA uses two methods to calculate 'effectiveness'; one for
seat belts another for helmet use. In order to increase the sample size for
the helmet portion, NHTSA did not control for contributing risk factors, as
they do with the seat belt portion. This leads to inflated 'effectiveness'
ratings for the helmet study. NHTSA states on page 15 of the draft Report to
Congress, "Estimates of 'effectiveness' without these factors incorporated
were generated as part of the CODES analysis. These estimates were higher in
most of the states, an indication of the influence of these risk factors on
crash outcomes." In our comments to the draft Report to Congress, the AMA
notified NHTSA that their methodology inflates 'effectiveness' ratings for
helmet use. NHTSA's response was to change the final Report to Congress on
page 16 to read, "Estimates of 'effectiveness' without these factors
incorporated were generated as part of the CODES analysis. These estimates
of 'effectiveness' with and without risk factors were not significantly
different in most of the states." No change of methodology was used between
the draft and final reports. However, NHTSA estimates changed from "higher
in most of the states" to "not significantly different in most of the
states." We can only assume this is an intentional misrepresentation of the
facts. NHTSA acknowledges the minimal effectiveness of helmets in preventing
any type of injury, indicating that helmets prevent head injury, not all
injuries. In a separate analysis that addresses only head injuries and more
specifically, brain injuries, NHTSA found that helmets are effective in
preventing brain injury. However, the same analysis indicates that the
average inpatient charges for unhelmeted riders were less than those of
helmeted riders. NHTSA found that the average inpatient charge for a
helmeted motorcyclist receiving a brain injury was $26,985, compared to
$26,805 for an unhelmeted motorcyclist (see chart). NHTSA also found that
the average inpatient charge for a helmeted motorcyclist not receiving a
brain injury was $12,736, compared to $11,730 for an unhelmeted
motorcyclist. This same analysis found helmets to be 67 percent effective in
preventing brain injuries. Although this estimate is impressive on its own,
brain injuries represent the top of a pyramid in terms of motorcycle injury
distribution. NHTSA has acknowledged the minimal effectiveness of helmets in
preventing most injuries. The Technical Report supplement to CODES shows
inconsistencies for helmet effectiveness, which were not included in the
Report to Congress. In the Technical Report, NHTSA calculates odds ratios in
addition to 'effectiveness' ratings, as an alternative measure for
effectiveness. Utah's odds ratios indicate a non-protective effect for
helmets in preventing death. Pennsylvania's odds ratios are termed
"non-significant" in the prevention of injury, as are Hawaii's. The odds
ratios are not presented in the final Report to Congress. NHTSA attributes
these inconsistencies to the small sample size, different patterns of helmet
use and the type of injury. However, the same sample, the same patterns of
helmet use and the same types of injury are present in the estimates of
'effectiveness,' which are presented in the final Report to Congress. In
addition, NHTSA excludes Utah's data from the helmet portion of the study.
NHTSA's rationale for this decision is because Utah did not include a code
for not wearing a helmet on their traffic report. The code Utah uses is
'helmet use unknown.' The decision to exclude Utah's data was made after
Utah had been awarded a grant, based on its capabilities to participate in
this study, and after Utah collected its data. Had the 'unknown' factor been
a legitimate concern, it stands to reason that Utah would not have been
selected as a CODES participant state. Conclusion The AMA recognizes helmets
as an effective piece of safety equipment and encourages their use. However,
the determination to use a helmet should be that of adults, free from
governmental mandates. The results of the CODES study indicate that helmet
use has no impact on the cost of an injury after it has occurred. In
addition, this study shows that helmets are minimally effective in
preventing any type of injury after an accident has occurred. NHTSA was
provided with five million taxpayer dollars to develop a compelling case for
mandatory helmet use laws. Having generated a report with no national
applicability, using questionable methodology for the helmet study and with
mixed results for helmet use, NHTSA has fallen well short of the goal. In
their limited applicability, the draft and final Reports to Congress show
that motorcyclists are no more dependent upon public sources for medical
costs than motor vehicle operators. These reports show that helmets are
minimally effective in preventing any type of injury from occurring,
underscoring the importance of accident prevention. Furthermore, one
analysis showed the inpatient charges for unhelmeted motorcyclists to be
slightly more than helmeted motorcyclists. While another analysis, more
specific to head and brain injury, showed the inpatient charges for
unhelmeted motorcyclists to be less than helmeted motorcyclists. It remains
clear that the most effective way to mitigate the cost of motorcycle
injuries is through awareness and education aimed toward accident
prevention, a strategy widely recognized and accepted by the motorcycling
community. The NHTSA, however, continues to embrace a failed strategy of
imposing mandatory helmet use laws upon all motorcyclists, a strategy that
has been repeatedly rejected by the motorcycling community.

 
For further information: Copies of the CODES report to Congress on the
Benefits of Safety Belts and Motorcycle Helmets and the CODES Technical
Report may be obtained from the National Center for Statistics and Analysis,
National Highway Traffic Safety Administrations, 400 7th Street, SW, Room
6125, Washington, DC 20590 or send a FAX request to 202-366-7078. Copies of
the draft CODES report and Technical Report can be obtained from Sean Maher
at the American Motorcyclist Association, 33 Collegeview Rd., Westerville,
OH 43081. Please send a 9x12, SASE w/$2.00 of postage. If you have questions
or comments, please call Sean Maher at the American Motorcyclist
Association, (614) 891-2425.