5 common mistakes in SaMD and AI Quality Management Systems
A quality management system (QMS) is integral to the correct functioning of a software medical device company, not to mention being a strict requirement for regulatory approval in many jurisdictions. There are five common mistakes that companies regularly make, which they are perhaps unaware of, or that often go unnoticed until spotted by an external auditor.
Standards for medical devices
A QMS is simply a structured system of procedures and processes covering all aspects of design, manufacturing, supply, risk management, management responsibility, customer-related processes, and corrective and preventive actions (CAPAs). Its purpose is to have an optimal end SaMD or AIaMD product and, more importantly, minimise risk for end users.
Under the EU’s Medical Devices Regulation (MDR), you are required to have a quality management system as per Article 10(9). The current international standard for a quality management system is ISO 13485:2016 Medical devices – Quality management systems – Requirements for regulatory purposes. The FDA’s requirements are now also harmonised with ISO 13485, so it makes sense to have a QMS certified to this standard if you are aiming for global market access for your SaMD or AIaMD.
Common QMS mistakes by SaMD and AIaMD developers
Not validating software
The first common mistake we at Hardian come across is SaMD and AI companies not validating all software within the scope of their QMS. Although software validation is conducted by most companies that we audit, they don’t always fully understand the scope within which a software is deemed part of a QMS – hence the need for validation. Examples include complaints portals, which may simply be a ‘contact us’ page on a website that sends a notification – via email or, perhaps, a software system – to the company. This system needs to be validated to ensure feedback or complaints are being received and responded to accordingly, as per the company’s internal complaints procedure. It is surprising how often we come across systems that have faults within them, meaning complaints or feedback are not received.
2. Lack of supplier evaluation
Another issue we find is supplier evaluation. Medical device companies always make sure they validate physical suppliers, but they often fail to evaluate service suppliers, such as file-sharing systems, project management tools (if in the scope of the QMS) and, especially, external consultants.
One of the first questions we at Hardian ask a company when we are brought in as external auditors is, ‘Have they conducted a supplier evaluation on us?’ Why? Because they need to ensure our regulatory team have the relevant qualifications and experience to be able to conduct an audit to their requirements. If they have not, they are not compliant and this can be flagged up as a non-conformity during an external audit.
3. Not keeping it simple
Extremely complicated procedures that only quality staff understand is another issue we come across. Procedures should be written by those who are to conduct the processes, and then amended by a quality staff member to ensure they are compliant. If a procedure is so complex that it is not understood by process operators, they will not be able to follow it correctly, eventually leading to non-conformities. The complexity of a procedure should be matched to the competency of those who will be using it. If procedures are not understood by company staff then, in most cases, quality is to blame, not the staff.
4. Self auditing
Internal audits are required to be impartial, as in auditors should not audit their own work (a common mistake companies make). But what does this mean? In short, the quality team cannot audit any work in which it is involved; likewise, any other department. So how do we get around this?
Human resources, or any other non-quality-related department, may audit quality and vice versa. Internal auditors must be appropriately trained, ideally with some form of internal auditor qualification(s) and relevant experience. Alternatively, some companies use external consultants to conduct internal audits on their behalf. As mentioned above, you must ensure a supplier evaluation is conducted on them, that they are trained on your auditing procedures, and that your procedure mentions somewhere that you may use external consultants in lieu of internal staff for such audits.
5. Omitting customer data processes
A lesser-known mistake that we see is almost always exclusive to Software as a Medical Device companies. They state that ‘customer property’ is not applicable within the scope of their QMS, their justification being that they are not handling physical product. However, intellectual property and patient data, which may be stored on their internal systems, can be deemed to be customer property, and therefore there should be appropriate documented processes for handling this within the QMS.
Summary
As medical devices are becoming more and more complex, so too is the risk that can be associated with their use. We see a lot of software as a medical device and AI companies making these mistakes within their QMS, but in most cases, they are relatively easy to fix. Hardian is always on hand to help ensure you optimise your QMS, because this helps to ensure devices remain safe and keeps the potential for harm or injury to a minimum.
Hardian Health is clinical digital consultancy focused on leveraging technology into healthcare markets through clinical strategy, scientific validation, regulation, health economics and intellectual property.
