AI Photo Trends Raise Fresh Concerns Over Facial Data, Voice Cloning and Cyber Fraud

Pune, 12 September 2026: The growing trend of using artificial intelligence (AI) tools to transform personal photographs into retro, cinematic or stylised images has raised concerns over how facial data is processed, stored and potentially used by AI platforms.

Saurav Chauhan, who has more than 15 years of experience in the software industry and works on AI applications in an MNC, highlighted the data privacy and cyber-fraud risks associated with uploading personal photographs to generative AI platforms such as ChatGPT.

Chauhan said that unlike conventional photo filters that generally process an image on a device, generative AI services require users to upload photographs to remote servers for processing. Depending on the platform and a user’s settings, uploaded content may also be retained or used for improving AI models.

He said users should also be aware that photographs can contain metadata, including information related to the device, date, time and, in some cases, location.

The issue becomes more significant when facial images are considered alongside the growing use of AI for impersonation and fraud, Chauhan said.

AI-generated voice scams have emerged as a concern in India, with fraudsters using cloned voices to impersonate relatives and demand money during supposed emergencies. Chauhan referred to a McAfee study that reported that 83 per cent of Indians who encountered an AI-generated voice scam lost money, while nearly seven in 10 people were unable to reliably distinguish an AI-generated voice from a real one.

Police cases reported in India have also involved callers allegedly impersonating relatives. In one reported case in Kolkata, a man lost nearly ₹96,000 after receiving a call involving a supposed medical emergency. Another reported case in Hyderabad involved a woman losing more than ₹1 lakh after a caller allegedly impersonated her nephew living abroad.

According to Chauhan, the combination of publicly available photographs, voice recordings and personal information on social media can provide criminals with material that may be used for increasingly convincing impersonation attempts.

The Federal Bureau of Investigation (FBI) has also warned about criminals using generative AI to create fake explicit images of people and subsequently using them for harassment or blackmail.

A major corporate fraud case linked to deepfake technology was reported in Hong Kong in early 2024, when an employee of engineering company Arup transferred about $25 million after participating in a video conference in which criminals allegedly impersonated senior executives and colleagues using deepfake technology.

Chauhan said users who have already uploaded photographs to AI platforms need not panic but should review their privacy settings and remove old conversations containing personal images where appropriate.

He recommended switching off the setting that allows uploaded content to be used for model improvement. In ChatGPT, the relevant option can be accessed through Settings → Data Controls → Improve the model for everyone, although the wording or location may vary between app versions.

He also advised users to delete old chats containing uploaded photographs and, when appropriate, use screenshots instead of original image files because screenshots generally remove much of the metadata associated with the original photograph.

Chauhan advised against uploading identity documents, bank-related papers or photographs of children to generative AI services.

He also recommended that families establish a private code word that can be used to verify the identity of someone making an emergency call.

“If a panicked caller cannot provide the agreed safe word, the call should be treated with caution,” he said.

He further advised people to independently call the family member back using an already saved number rather than relying on the incoming call or continuing the conversation with the caller.

People who lose money in an online financial fraud can report the incident through India’s cybercrime helpline 1930 or the national cybercrime reporting portal. Chauhan also reminded users that banks and financial authorities do not ask customers to disclose their OTPs or PINs over phone calls.

Chauhan has more than 15 years of experience in the software industry and currently works on AI applications in production at an MNC IT company. He has also spent more than four years teaching AI to data scientists and AI engineers.