- Widely adopting validation and testing tools for the Fast Healthcare Interoperability Resources (FHIR) can help to reduce variation in implementation and ensure the deployment of more standardized methods of health data exchange, says the ONC and several other industry stakeholders in a new journal article.
The paper, published in JMIR Medical Informatics, asserts that validation tools are key for preventing developers from deviating too far from HL7 FHIR standards, a common byproduct of implementation decisions when navigating healthcare’s fragmented health IT environment.
Creating FHIR-based solutions that are similar enough to work together seamlessly will be essential for creating a truly interoperable healthcare industry that enables appropriate data access for authorized stakeholders.
“Despite the relatively rapid nationwide adoption of electronic health records (EHRs), the industry’s ability to successfully exchange computable health data has not kept pace,” said authors from the ONC, MITRE Corporation, and AEGIS.net.
“A recent study found that less than 35 percent of providers report data exchange with other providers within the same organization or affiliated hospitals. The exchange of data across organizations is even more limited, with less than 14 percent of providers reporting they exchange data with providers in other organizations or unaffiliated hospitals.”
FHIR is becoming an increasingly common way to bridge those fundamental gaps in health data exchange.
The ONC recently released statistics showing that EHR vendors covering approximately 80 percent of the nation’s healthcare providers are using FHIR to support some component of their health data access and exchange capabilities – a promising development in light of new requirements for application programming interface (API) access within EHRs starting in 2019.
Yet each vendor may have interpreted the FHIR standards slightly differently during development and deployment, stated the authors, despite the guidance of industry groups such as the Argonaut Project.
“Although the ONC’s 2015 Edition Health IT Certification Criteria includes an API certification criterion…that is well suited for FHIR implementation, no government regulations require health IT developers to conform to any published version of the FHIR standard,” explained the team.
Validation and testing tools may be able to bring developers into closer alignment, the ONC, MITRE, and AEGIS.net hypothesized. Such toolkits have been available since FHIR began its meteoric rise in popularity, but have not always been used consistently.
In order to test whether more stringent testing could reduce variation and improve interoperability, the team examined the impact of two different testing suites – Crucible from MITRE and Touchstone from AEGIS.net – both of which are open source and readily available to FHIR developers.
“During the study period, software developers executed tests using both projects either autonomously or as part of a FHIR Connectathon. Both projects automatically collected usage data on the tests that they execute,” the authors said.
The projects collected metrics including the version of FHIR used, how the tests map to the FHIR specification, and if the tests passed, failed, or skipped components of validation.
“The results of our data analysis indicate that as the frequency of testing or number of tests increases, the performance of a server against those tests increases,” said the team. “This should not be surprising as software developers will address issues and fix defects in order to pass the tests, so long as they are discovering these issues and defects by repeated testing.”
“Assuming the tests accurately and adequately cover the depth and breadth of the FHIR specification, then FHIR servers developed and tested using these tests in a test-driven manner should more accurately adhere to the FHIR specification.”
More frequent testing can also speed up the development process, the study indicated. By conducting daily tests and integrating continuous improvement principles into the development cycle, one vendor was able to accomplish its tasks twice as fast as other entities, the study said.
“Crucible and Touchstone have proven to be valuable tools for the FHIR developer community,” the authors wrote.
“These tools can aid in the transition to an interoperable health care infrastructure by providing open reference implementations for FHIR testing and support future Cures Act requirements. Our research shows that developers that use testing and validation tools tend to produce more compliant FHIR implementations.”
In order to further the use of effective testing and validation tools, the ONC recently released a new option for the development community. The Inferno testing suite “offers a rich and rigorous set of testing tools to help health IT developers make sure the FHIR standard is consistently implemented,” wrote Steven Posnack, MS, MHS, Director of the Office of Standards and Technology at ONC, who is also an author of the JMIR study.
“Inferno has been designed to verify that developers’ FHIR servers meet the leading industry standards most often implemented as part of a FHIR deployment (e.g., Argonaut Implementation Guide, OAuth 2.0, Open ID Connect, and other best-practices contained in the SMART App Authorization Guide),” the ONC added.
“It also includes some bells and whistles, such as testing for Dynamic Client Registration, and has a step-by-step approach for testing each of the standardized interactions needed for apps to register and securely connect to a FHIR server.”
Making validation and testing tools more available to the industry will be an important part of ensuring broad interoperability and meaningful health data exchange, the study authors concluded.
“The test data collected by MITRE and AEGIS.net during the study period shows that when it comes to health data interoperability, ‘practice makes perfect,’” they said. “This gives us hope that a future with ubiquitous health care information interoperability is possible.”