Veteran Affair’s Automated System Leads to 27% Inaccuracy in High BP Claims

The Veterans Affairs (VA) automated system, which was intended to expedite claims decisions, has unintentionally contributed to inaccuracies in approximately 27% of high blood pressure claims. This issue was brought to light through a recent report from the VA Office of the Inspector General (OIG), which found that more than a quarter of the 60 high blood pressure claims reviewed were processed inaccurately through the Automated Benefits Delivery System.

This development underscores the complexities of automating intricate decision-making processes and emphasizes the crucial role of human oversight in AI-driven systems. The Automated Benefits Delivery System was introduced in December 2021, with the VA anticipating a substantial increase in disability applications following the enactment of the PACT Act.

The Act prompted Vietnam-era veterans to file high blood pressure claims due to their exposure to Agent Orange, a known contributor to hypertension. The automated system’s purpose was to extract blood pressure readings and other relevant data from VA treatment records, generating a summary for review by VA staff who would make the final decisions on these claims. However, the system’s compilation of incomplete and inaccurate data resulted in a series of incorrect decisions. 

The VA OIG report recommended that the VA make improvements to the technology and the quality assurance process, emphasizing that the lack of oversight increases the risk of veterans not receiving the benefits they are entitled to. Larry Reinkemeyer, the VA assistant inspector general for audits and evaluation, expressed concern that investing additional funds in a process and technology that doesn’t deliver the intended outcomes is a real possibility. 

Christopher Alexander, chief analytics officer of Pioneer Development Group, shed light on the challenges associated with humans using artificial intelligence tools without fully understanding their capabilities. He emphasized that AI tools are only as effective as the data they receive and the quality of human oversight. Alexander argued that human error and poor data quality control led to inaccurate data being processed by the AI, which, in turn, couldn’t identify the faulty information. 

Phil Siegel, founder of the Center for Advanced Preparedness and Threat Response Simulation (CAPTRS), suggested that the VA needs to address data collection and data entry issues before implementing automated models. He stressed the old adage, “garbage in, garbage out,” which holds true for software systems, including AI models. Siegel pointed out the need to ensure the quality of data input to achieve reliable and accurate outcomes. 

VA officials acknowledged most of the recommendations from the OIG report in their official response. They clarified that the automated system never made final claims decisions and that, during the inspection, the system was still in the early stages of implementation. The VA noted that the OIG was reviewing cases in the verify phase, with prototyping and a pilot phase not yet complete, and ongoing development efforts. 

In response to inquiries from Fox News Digital, VA Press Secretary Terrence Hayes explained that the VA has begun using automation in limited cases to expedite claims processing and ensure veterans receive their entitled benefits promptly. Hayes emphasized that the claims reviewed by the OIG were processed during the prototype stage of this new ability before it was fully operational. Subsequently, the VA made significant improvements to the automation process. 

Some of these improvements include the implementation of a repeatable process designed to ensure quality and effectiveness on a small scale before nationwide deployment. The VA also established a dedicated quality review process that incorporates independent reviewers. Additionally, the VA collaborated with tech industry leaders to gain insight into process enhancement and introduced a new dashboard to closely monitor trends. 

Hayes highlighted that, on average, veterans receive initial decisions approximately 20 days faster with the use of automation. The VA is committed to modernizing and expanding its business processes and technology, in addition to enhancing communication with veterans, family members, caregivers, and survivors. The VA’s introduction of an automated system to expedite claims decisions faced challenges, with approximately 27% of claims receiving incorrect ratings.

The VA recognizes the need for enhancements, and it has undertaken efforts to improve data quality, the automation process, and oversight. Despite initial setbacks, the VA remains committed to providing veterans with the benefits they deserve in a timely manner, while learning from the pitfalls of automating complex decision-making processes. 

Overall, this situation highlights the importance of careful planning, data quality control, and continuous monitoring when implementing automation in critical processes, especially in government agencies that serve a vulnerable population. The VA’s willingness to adapt and improve its systems is a promising step toward providing better services to veterans. 

Reference  

Fox News, “Veterans plagued by errors in health benefit system due to computer mishap” Veterans plagued by errors in health benefit system due to computer mishap | Fox News.