How we assess apps
Our Digital Assessment Questions make sure only safe and secure apps and digital tools are hosted in the NHS Apps Library. App providers may need to show evidence that their products pass our tests in areas such as clinical safety, data protection, security and usability, and how anyone using them could see benefits to their health and wellbeing.
The questions have been designed by our expert reviewers who use them to see how an app performs in nine key areas. Any major updates made by an app provider will require their product to be reassessed to make sure it still meets the required standards following the changes.
App providers must answer our pre-qualification questions before they can put their product forward for full assessment and be considered for the NHS Apps Library.
We currently focus on patient-facing apps that help with social care, cancer, maternity, mental health or long-term conditions such as asthma and diabetes.
Our eligibility questions are used to find out what an app actually does. We ask how it helps users to treat or manage their condition and who might use it as part of their treatment.
All apps must have approval from the correct regulatory body, for example the Medicines and Healthcare Products Regulatory Agency, which makes sure that medical devices work and are safe to use.
Evidence of Outcomes
These questions make sure all apps are doing what they are supposed to do, and we ask app providers to show us how their product improves health and wellbeing. For example, if an app is designed to help patients with their mental health, the app provider must give examples of how it could — or already has — helped people.
We also ask how an app can assist the NHS in areas such as reducing waiting times or reducing the need for doctor’s appointments, as well as making sure there is a demand for the services it offers.
In December 2018, the National Institute for Health and Care Excellence (NICE) published new standards for evidence of clinical and cost effectiveness in digital health technologies, including health apps. NHS Digital is working to include these standards in the next version of the DAQ.
Our clinical safety questions make sure app providers have taken all appropriate action to keep safe any patients using their product. For example, with an app that reminds patients to take their medication, app providers must give evidence that shows any risk of these reminders being incorrect has been completely removed or made as low as possible.
Providers of any app that could put a user at risk must meet the safety standards required by the Health and Social Care Act 2012. This would mean producing Hazard Logs and Safety Case Reports, which would be reviewed by experts at NHS Digital.
Our data protection questions are designed to make sure that any personal information collected or shared by an app is handled in a safe, fair and lawful way.
App providers must give details of where the data collected is stored and tell users what rights they have to control how their information is used.
These questions seek to understand the security controls in an app that are relied on to protect your data and device. Among other things, we check a product only asks for the information it really needs to work for you, that installing it will not put your device at risk and that your information is handled and stored securely.
App providers must prove that their products follow good security practices, so we can be sure they are safe to use.
Usability & Accessibility
Our usability and accessibility questions help us make sure apps are easy to understand and use. Text must be easy to read, and action buttons must be easy to press, clearly marked and do exactly what users expect them to do.
Apps are assessed to make sure they provide access to as many people as possible, including older users, younger users and those with disabilities. This might involve them being able to increase text size where needed and work with voice software to help visually impaired people.
Our interoperability questions test how well an app exchanges data with other systems. For example, how it connects with a patient’s medical record or collects readings from another device such as a smart watch or blood pressure monitor. This process helps app providers use data within their products to build new functions, benefiting users.
Not all apps exchange data, but providers of those that do must meet NHS England’s rules on access to data. These rules make the sharing process simple, while also keeping it safe and secure.
The technical stability questions are used to understand how an app has been tested and how testing will continue over time. App providers must show how patients can report any problems with a product and how they will work to sort out those problems.
These questions also cover what happens to any patient information an app has collected if the patient stops using it or it is shut down by the app provider.
Apps featured on the NHS Apps Library will either have an 'NHS Approved' badge, a 'Being tested in the NHS' badge, or no badge.
An app meets NHS quality standards for clinical effectiveness, safety, usability and accessibility and has a supportive evidence base.
Being tested in the NHS
An app meets NHS quality standards for safety, usability and accessibility. It is being tested with NHS patients to see if there is evidence of clinical effectiveness.
An app meets NHS quality standards for safety, usability and accessibility. It is not currently being tested by the NHS for clinical effectiveness.