Avoid these AI risks in your business

Generative AI has its uses in business but getting it wrong can be damaging. Katherine Evans explores the potential pitfalls

Generative AI can automate repetitive tasks in your business, such as data entry, report generation or scheduling, freeing up employees to focus on more strategic and value-added activities, leading to the potential for improved operational efficiency and productivity.

Compared to hiring additional staff or outsourcing certain tasks, use of generative AI can be a cost-effective solution, eliminating the need for additional internal resources or external service providers.

However, generative AI models have the ability to create unexpected and unintended outputs. Even with rigorous training and testing, these models can produce content that is offensive, inappropriate and/or illegal. Organisations must have mechanisms in place (a) to prevent the dissemination of unintended outputs and (b) to deal with any fallout which escapes those protective processes.

Recruitment and HR

Monotonous tasks like candidate sourcing and screening can increasingly be completed using AI software. The companies which promote such software may tell you that AI has a 6 per cent error rate versus a corresponding human error rate of 11.3 per cent, but in such an apparently definitive statement, much is left unsaid. An employer knows what instructions it gave to a human employee in terms of which websites or databases to check and what kind of parameters to be taken into account during a particular search. With generative AI, it is much harder to establish that the data used in any search was properly sourced in a manner which was compliant with data privacy and security regulations.

Businesses also need to be aware that generative AI systems may well fail to identify candidates who would be no less suitable for a particular role but who may be excluded from the success criteria by virtue of having done less well academically and/or having had a less impressive career history, as a result of a wide range of possible factors including (without limitation) having a physical disability, being neurodivergent, having caring responsibilities or having grown up in a home where English was not the first language. If these biases go unchecked, they can perpetuate existing societal biases and discrimination.

Note also Article 22(1) of GDPR and UK GDPR:

“The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significant effects on him or her.”

In short, while having AI software may reduce the need for human involvement in the recruitment and HR processes, somebody still needs to carry out checks and balances on the outcome of any automated decision making processes.

Customer data

AI can be a useful tool for database creation, being faster and more efficient at capturing and recording data more consistently than a corresponding human could achieve. It can flag when details have not been checked in the previous twelve months and therefore when they may need to be updated. It can record in a consistent fashion whether data subject requests have been made and the outcome of those requests. All this can save time and resources and even support legal compliance but nothing can avoid the need for manual cross checking of data accuracy. The risk otherwise can be that AI software is verifying its own data in an unhelpful circle of false trust.

Businesses can use generative AI to build chatbots that understand and respond to customer queries, reducing the need for human intervention. This can help businesses improve response times, provide 24/7 support and handle a higher volume of inquiries without significantly increasing staff numbers. However, a percentage of the interactions between chatbots and customers need to be monitored closely by customer service employees to identify glitches and sources of irritation, which may otherwise lead to further customer complaints and ultimately to an increase in customer attrition rates.

The tension with data privacy legislation is likely to arise in the context of sourcing of new customer leads. Businesses need to satisfy themselves that they understand how the customer lead data was sourced and that it was sourced in a manner which was consistent with privacy laws. Information derived from generative AI software should always be checked against publicly available material for consistency and accuracy.

Intellectual property

Generative AI can assist small businesses in creating high-quality content at scale. Whether it’s blog posts, social media updates, product descriptions or marketing materials, AI can generate text that closely mimics human writing. This capability saves time and resources for small businesses that may not have dedicated content creation teams.

However, it also has the potential to create content that infringes upon existing intellectual property rights. This risk is particularly relevant in industries such as media, publishing, and the creative arts, where the ownership and protection of intellectual property are paramount.

Generative AI can also produce content that is offensive, discriminatory or controversial. If content is not properly vetted by appropriately qualified individuals, its release into the public domain can severely damage a company’s reputation and brand image.

Product liability and AI

The AI Act is a proposed European law on artificial intelligence (AI) – the first law on AI by a major regulator anywhere, which proposes three risk categories for AI. Firstly, there are applications and systems that are deemed to create an unacceptable risk, such as government-run social scoring of the type used in China. These will be subject to an outright ban. Secondly there are, high-risk applications, such as a CV-scanning tools that rank job applicants. These will be subject to specific legal requirements (to be determined but see above the risks already identified with such applications). It seems that applications not explicitly banned or listed as high-risk will largely be left unregulated.

Businesses need to ensure that they understand this fast-moving area of the law in all jurisdictions in which they operate, and that the AI tools that they employ remain compliant with the ever evolving legal and regulatory landscape.

As an aside, the UK is not following the EU’s approach in terms of adopting new legislation or creating a new regulator for AI. Rather it is requiring existing regulators, including the UK Information Commissioner’s Office (ICO), to take responsibility for the establishment, promotion, and oversight of responsible AI in their respective sectors. Regulators’ activities would be merely ‘reinforced’ by the establishment of new support and oversight functions within central government. It remains to be seen how well this attempt at joined up government might work.

Conclusions

As with any business tool, your business needs to satisfy itself that the AI tools it chooses will perform in a way which can be verified on an ongoing basis as being compliant with applicable laws, unlikely to infringe a third party’s intellectual property rights and unlikely to result in reputational damage to your business. There need to be human checks and balances at each stage to verify that your business remains as protected as it can reasonably be.

Finally, you should always remember that unlimited indemnities against an AI business will not protect your business if the AI business either does not have the law on its side or does not have the financial substance or wherewithal to defend you. It is your due diligence which will protect your business and not the reassurances of a third party.

Further reading

Why your business needs chatbots to improve customer experience – How can a chatbot improve your customers’ experience? And, if you are going to make an investment, what steps should you take to ensure implementation runs smoothly? Here, Chris Crombie finds out

Voice search for small businesses – The way that people are researching information is changing. Follow this guide to get ahead and optimise your voice search for your business

The key qualities to look for in a candidate’s CV – Here are some pointers for small businesses on identifying the best candidates through their CVs

Related Topics

Artificial Intelligence