Over the past year, ICAEW has undertaken a project to review the business risks associated with using developing and emerging technologies – such as blockchain, VR, or the main case study of our research, AI and cognitive automation. We have been seeking to understand better the key risks that these powerful and valuable technologies carry, and how organisations seeking to benefit from their strengths can work to reduce and contain those risks. This has led us to describe a combined response to the risks of emerging technologies, based on design principles followed at the point of creation, controls put in place during operation, and a role for internal or external assurance providers to help check that the risks are properly controlled and the technology is working as intended.
For AI, which is usually developed from data using machine learning or similar approaches, a key risk is that the system can become incomprehensible to a human reviewer. This not only makes spot-checking the system’s decisions – or explaining them to the subject of an automated decision – harder, it also means that any errors or biases in the system may be difficult or impossible to fix.