{"id":2000,"date":"2021-02-13T17:38:42","date_gmt":"2021-02-13T16:38:42","guid":{"rendered":"https:\/\/www.medsyscon.com\/?p=2000"},"modified":"2021-04-06T17:07:30","modified_gmt":"2021-04-06T15:07:30","slug":"blog-benefit-risk","status":"publish","type":"post","link":"https:\/\/www.medsyscon.com\/en\/blog-benefit-risk\/","title":{"rendered":"ISO 14971:2019 \u2014 Clarifying Benefit, Risk, & Benefit-Risk"},"content":{"rendered":"\n

Edwin L. Bills, member, ISO TC 210 JWG, wrote in the Guest Column on Med Device Online from February 8, 2021: \u201cDiscussion of the term \u201cbenefit\u201d first appeared in the first edition of ISO 14971, which appeared in 2000. The requirement of controlling risk allowed the use of \u201cbenefit\u201d as an alternative method of releasing a product when a residual risk was evaluated as \u201cunacceptable.\u201d Residual risk is that risk remaining after risk controls are applied, so, in this case, all possible risk controls were applied, and the residual risk still could not reach acceptable levels. The product should not be put on the market if the benefit does not outweigh the risk in a documented benefit-risk analysis. It would be up to regulators to ask the question, but you should not move product forward if the benefit does not outweigh the risk or until further risk reduction could be made.<\/p>\n\n\n\n

One of the dramatic examples of the use of risk-benefit evaluation was the original mechanical heart, which had an associated mortality rate of 40% when it was used. This risk was considered unacceptable until measured against the benefit. The benefit was that the mortality rate was 70% without the use of the device. Clearly, the improved benefit outweighed the risk, and the device was ultimately cleared for use by FDA. However \u2014 and here is a consideration that must be part of postmarket review (see previous production and post-production article<\/a>) \u2014 in less than a year, what was considered \u201cstate of the art\u201d (another ISO 14971:2019 defined term in 3.28) changed and a competing product was released with an associated 30% mortality rate. Now, the original product was no longer acceptable \u2014 not because it was defective or the risk changed, but because the state of the art changed and a safer product was available. The original product had to be withdrawn until changes could be made to reach the new state of the art.<\/p>\n\n\n\n

Note that the term \u201crisk-benefit\u201d was used in the 2000 edition and the 2007 edition of the standard. In the 2019 edition of the standard, you will find the term is now \u201cbenefit-risk\u201d because regulators felt that manufacturers were only looking at benefit as an afterthought and thus wanted to emphasize benefit first instead of risk. Nowhere in regulation or standards will you find this term defined; however, you can find definition of \u201crisk\u201d in ISO 14971 (now at 3.18 in the 2019 version) going back to the original edition. In the standard, the definition is and has been<\/em> \u201ccombination of the probability of occurrence of harm and the severity of that harm.\u201d This definition was adopted by the regulators and appears in the EU\u2019s Medical Device Regulation (MDR) and In Vitro Diagnostics Regulation (IVDR). <\/p>\n\n\n\n

The FDA Quality System Regulation (QSR), 21 CFR 820, appeared several years before ISO 14971 and did not have \u201crisk\u201d as a defined term in the regulation. However, FDA was aware that the state of the art was changing. In the Preamble to the QSR in Comment 83, FDA indicated it was participating in ISO Technical Committee 210, working in the area of medical device risk. The FDA is currently considering updating its regulation to align with ISO 13485, in which the term \u201crisk\u201d carries the ISO 14971 definition.<\/p>\n\n\n\n

In Essential Principles of Safety and Performance of Medical Devices and IVD Medical Devices, <\/em>IMDRF\/GRRP WG\/N47 FINAL:2018, the International Medical Device Regulators Forum (IMDRF) uses the same \u201crisk\u201d definition as ISO 14971 in 3.35. <\/p>\n\n\n\n

So far, we have half the equation, \u201crisk,\u201d well defined. The term \u201cbenefit,\u201d however, had been undefined by regulators or in standards until ISO 14971:2019. Here, a couple of things happened:<\/p>\n\n\n\n

  1. \u201cBenefit\u201d is now defined, in 3.2, as \u201cpositive impact or desirable outcome of the use of a medical device on the health of an individual, or positive impact on patient management or public health.\u201d A Note 1 to Entry expands information to help clarify the implications of \u201cbenefit.\u201d<\/li>
  2. Since then, regulators, including FDA (see the four guidance documents in Refs. 1\u20134) and the European Commission (EC), adopted the term \u201cbenefit-risk\u201d instead of \u201crisk-benefit\u201d as was previously used in the standard. ISO TC 210 JWG1, the authors of the risk management standard (ISO 14971) and technical report (ISO TR 24971), updated the standard and technical report to the new term for the new releases in 2019 and 2020, respectively.<\/li><\/ol>\n\n\n\n

    We now have defined terms updated to current regulatory thinking, in the ISO 14971 standard and ISO TR 24971 technical report, so we now understand what we are talking about. Of course, we defined \u201crisk\u201d and \u201cbenefit\u201d individually, and not as a combined term, but we have some understanding of the terms.<\/p>\n\n\n\n

    The 2019 edition of the standard indicates in 7.4 Benefit-Risk Analysis that \u201cbenefit-risk analysis\u201d may be used to address unacceptable individual residual risks. It refers to ISO TR 24971\u2019s discussion (via guidance, not requirements) of some aspects and possible methods of this analysis. This section only deals with the individual risks, and the standard (7.4) only requires that individual unacceptable<\/em> residual risks undergo this analysis. Some regulatory bodies may require that all<\/em> individual risks, regardless of acceptability, undergo this analysis.  This is not in conflict with the standard, so you must apply this regulatory requirement in addition to the requirement of the standard if you are serving a market where this additional requirement exists. How to perform these analyses is beyond the space allowance we have here, but the discussion referred to earlier in 24971 is more than we could successfully perform here.<\/p>\n\n\n\n

    For individual risks, then, the 7.4 requirement in the standard only addresses individual unacceptable risks, not all risks. Regulators may impose additional requirements. However, we need to go further in the standard to uncover additional requirements for \u201cbenefit-risk analysis.\u201d<\/p>\n\n\n\n

    How To Evaluate Overall Residual Risk<\/h2>\n\n\n\n

    In ISO 14971\u2019s Clause 8, Evaluation of Overall Residual Risk, we find a universal requirement applicable to all devices, regardless of risk acceptability. The standard requires, \u201cthe<\/em> manufacturer <\/em>shall evaluate the overall<\/em> residual risk <\/em>posed by the medical device<\/em>, taking into account the contributions of all residual risks<\/em>, in relation to the benefits <\/em>of the intended use<\/em>\u2026\u201d (emphasis mine). This means for every device, an overall residual risk evaluation shall consider the benefit compared to the risk of using the device. The benefit should outweigh the risk.   Fortunately, ISO TR 24971:2020 includes 2.5 pages of guidance on methods you can use to perform this analysis. The technical report in 8.3 a)\u2013f) provide some possible approaches, indicating there is no preferred approach. The evaluators should be experienced in the use of the device and the environment of use, which are both critical aspects of such an evaluation. One of the key tools I have used in this type of risk evaluation is a visual tool presenting all of the risks of the device simultaneously on a risk chart so that it may be easier to understand the device\u2019s risk profile in such an analysis. ISO TR 24971 8.3 b) explains this approach to using visual representations of residual risks to understand where problems may exist in the overall residual risk of the device and where efforts can be applied to give the best results on reducing residual risk. <\/p>\n\n\n\n

    ISO TR 24971\u2019s Clause 8.1 discusses that there is no preferable way to perform overall residual risk analysis, and that judgment is involved. As a result, it is essential that professionals with knowledge, experience, and authority (including application specialists and clinical specialists or users) perform this evaluation. <\/em>In my experience, a team that includes medical professionals current with device experience, product liability insurance experts, experienced design members, and risk management experts not associated with the project<\/strong> would provide the needed expertise to perform this evaluation. Adequate preparation, including review of the entire risk management file for the device prior to conducting the overall residual risk evaluation and review of documentation of comparable devices, is necessary and may provide valuable information. Additionally, the overall residual risk acceptability criteria provided for the device might be different from the individual residual risk acceptability criteria. This criteria must be included in the risk management plan for the device prior to conducting the evaluation. ISO TR 24971 Clause 8.2 provides information on the inputs for the evaluation.<\/p>\n\n\n\n

    An important aspect of the requirements in Clause 8 is the requirement that \u201cthe manufacturer shall inform the user of significant residual risks, and shall include the necessary information in the accompanying documentation.\u201d A limited discussion of the rationale about what is significant for this disclosure appears in ISO 14971 Annex A.2.8. <\/p>\n\n\n\n

    It is important to understand that only Clauses 1\u201310 of ISO 14971:2019 contain requirements.  All other parts of the standard and all parts of ISO TR 24971 are only guidance to provide aid in constructing your risk management system. Those sections are not<\/em> requirements. This misunderstanding has caused a great deal of problems for those implementing the standard in the past.<\/p>\n\n\n\n

    That covers the risk side. Now for the benefit side of the equation.  <\/p>\n\n\n\n

    How To Evaluate Benefit<\/h2>\n\n\n\n

    While engineers love numbers and want to assign values to such an analysis, it is not possible to do this numerically for benefit, because it consists of many independent factors. This effort should involve medical\/clinical risk management team members, as they are more qualified to perform benefit analysis. Additionally, the evaluation team must consider any alternative treatment modalities, including pharmaceuticals and biologics, when considering benefit-risk. If the alternative treatments provide the benefit at a lower risk, then the necessary benefit-risk may not be achieved to place the device on the market. ISO TR 24971:2020\u2019s Clause 7.4 discusses benefit-risk analysis, with some additional information is provided in Clause 8.3 a) and c).<\/p>\n\n\n\n

    First, for a more complete understanding of benefit, the definition of the term in ISO 14971:2019 Clause 3.2 is essential: <\/p>\n\n\n\n

    benefit<\/em>
    positive impact or desirable outcome of the use of a medical device on the health of an individual, or a positive impact on patient management or public health<\/em><\/p>\n\n\n\n

    Note 1 to entry: Benefits can include positive impact on clinical outcome, the patient\u2019s quality of life, outcomes related to diagnosis, positive impact from diagnostic devices on clinical outcomes, or positive impact on public health.<\/em><\/p>\n\n\n\n

    ISO 24971:2020 7.4.2 mentions factors necessary to consider when evaluating benefit, such as:<\/p>\n\n\n\n