How to use “law” to defeat “magic” when facing a “AI face-swapping” fraud that resulted in a loss of 4.3 million?

Defeating AI face-swapping fraud with law instead of magic to recover 4.3 million.

As the most vital application of artificial intelligence, deep synthesis technology will inevitably unleash great productivity in the future, and therefore compliance issues in this field must be taken seriously.

Written by: Sa Jie team

Key points:

• The person in the news “AI Face Swap Scams 4.3 Million in 10 Minutes” who directly uses AI face swap to defraud money constitutes the crime of fraud . If the AI face swap technology provider conspires with the perpetrator, the AI face swap technology provider (natural person) constitutes the joint perpetrator of fraud. If there is no conspiracy between the two parties, the AI face swap technology provider may constitute the crime of assisting in information network criminal activities.

  • Gitcoin COO: How to Build a “Web3 Anti-Scalping” System
  • How does the Meta-favored Magic Leap achieve a huge leap in the metaverse?
  • What are the advantages and disadvantages of mainstream arbitrage schemes on the LSD track?

• The technology mentioned in the news “AI Face Swap Scams 4.3 Million in 10 Minutes” is the deep synthesis technology of AI, which is a part of the broad group of AIGC (Artificial Intelligence Generated Content) technologies. Currently, deep synthesis technology in China is directly regulated by the “Internet Information Service Deep Synthesis Management Regulations”. In April of this year, the National Internet Information Office solicited opinions on the “Administrative Measures for Generative Artificial Intelligence Services (Draft for Soliciting Opinions).” After the final version of the draft is released, deep synthesis technology will be further regulated.

• In addition to criminal offenses, the unauthorized use of deep synthesis technology such as AI face swap may also constitute civil infringement disputes such as infringement of portrait rights and reputation rights. There are already judicial precedents in which AI face swap technology constitutes civil infringement.

• AIGC start-up companies involving deep synthesis technology need to comply with 20 compliance obligations such as information release review and personal information protection to ensure their own business compliance. The specific compliance content can be seen at the end of the article.

AI Face Swap New Scam, China Internet Association Issues Reminder

A few days ago, a new AI face swapping scam appeared in cities such as Baotou, Inner Mongolia. The victim’s friend suddenly contacted the victim through WeChat video and claimed to be bidding in another city, requiring a deposit of 4.3 million yuan and a transfer to a public account, and that it was necessary to borrow the victim’s company to transfer the money. Based on trust in the friend and having already confirmed their identity through video chat, the victim transferred 4.3 million yuan to the friend’s bank card in two installments. The victim only found out that they had been cheated after calling their friend afterwards. In this case, the scammer used AI face swap technology to disguise themselves as the victim’s friend in the WeChat video, and then synthesized the voice of the victim’s friend using voice synthesis technology, and then defrauded the victim.

As a response to the increasingly rampant AI face-swapping fraud, multiple regions have issued warnings to remind the public to be aware of the corresponding fraud risks. On May 23, the Anti-Telecom Network Fraud Center in Mianyang, Sichuan issued an early warning to remind the public to be vigilant against such fraud. Yesterday, the China Internet Association issued a warning to the public that, with the opening up of deep synthesis technology, the number of deep synthesis products and services has gradually increased, and illegal activities such as fraud and defamation using false audio and video such as “AI face-swapping” and “AI voice-changing” have been frequently seen.

Currently, the first reported case of fraud using deep synthesis technology occurred in 2019 worldwide.

First Case of Deep Synthesis Fraud

In March 2019, the CEO of an international energy company received an unknown call. Soon, he recognized the person on the other end of the line—the CEO of the German parent company and his immediate superior. In this call, the parent company CEO said that the company was currently in operational crisis and urgently needed 220,000 euros or 243,000 US dollars in funding from branch offices worldwide. Later, this parent company CEO provided a bank account in Hungary and claimed that the funds needed to be paid directly to a Hungarian supplier. The parent company would reimburse the branch office for the expenditure after the funds had turned over. Although the instructions sounded wrong and suspicious, and involved such a large amount of money circulation, the victim still chose to execute the order after careful consideration. The reason was simple: the voice on the phone was indeed that of his immediate superior, and even the iconic slight German accent and tone of voice were indistinguishable from those of his boss in normal conversation. Thus, although the victim still had doubts, he still completed the transfer to the Hungarian account provided in the “boss’s” phone call. In this case, the transferred funds flowed from Hungary to Mexico, and then were dispersed elsewhere. As of May 2023, the funds involved have not yet been recovered.

Because of the frequent occurrences of such cases on a global scale, the traditional neutral view on technology has little support in the legal field, and countries have gradually strengthened the regulation of deep synthesis technology. The “Regulations on the Administration of Internet Information Services for Deep Synthesis” is a normative document specially regulating deep synthesis technology in China. In addition, the “Cybersecurity Law,” “Data Security Law,” “Personal Information Protection Law,” “Administrative Measures for Internet Information Services,” “Regulations on the Governance of Network Information Content Ecology,” “Regulations on the Management of Internet Information Service Algorithm Recommendations,” and “Regulations on the Administration of Network Audio and Video Information Services” also involve the regulation of deep synthesis technology. Of course, the “Criminal Law” also has many charges that can regulate criminal acts using AI deep synthesis technology.

AI Face-Swapping Service Providers Face Criminal Risks of Aiding and Abetting Crimes

It is undeniable that deep synthesis technology is a “sunrise industry” at present. With the rapid reduction of the cost of technology iteration, related enterprises can even achieve huge profits through “volume-based” transactions. According to investigations, the current cost of producing a set of “face-exposed and spoken” videos is as low as tens of yuan, and only a frontal photo is needed to generate the video. The generated video not only has moving eyes, mouth, and head, but also matches the mouth shape with the words and voice content provided. All of this can be completed in just a few hours. Some merchants claim that creating a 1-minute video using photos only costs 30 yuan, and a few yuan can be done with low quality requirements.

It is worth noting that such behavior carries huge criminal risks. Article 287.2 of the Criminal Law of the People’s Republic of China stipulates the crime of aiding and abetting information network crime. The crime of aiding and abetting information network crime refers to the act of knowing that others use the information network to commit crimes, providing technical support such as Internet access, server hosting, network storage, communication transmission, or providing advertising promotion, payment settlement, and other assistance, and the behavior is serious. The “knowing” here can often be determined by objective evidence in practice, rather than relying on the subjective statements of the perpetrator. At the same time, providing AI face-swapping technology to customers can also be interpreted as “providing… technical means”. In other words, in the context of the rampant telecom fraud cases caused by AI face-swapping technology, if a customer requests an AI face-swapping service provider to use photos to generate videos without justifiable reasons and without significant evidence proving that they have obtained the consent of the person in the photo, and the relevant technology provider makes the video, which is then used by the customer for fraud, the AI face-swapping service provider is at high risk of being involved in the crime of aiding and abetting information network crime.

Unauthorized AI face-swapping may also constitute serious civil infringement

Aside from the criminal risk of constituting an information network crime as mentioned above, unauthorized AI face-swapping technology may also constitute serious civil infringement, and in fact, such judgments have already appeared. In the (2022) Shanghai 0116 Min Chu No. 13856 Portrait Right Dispute Judgment, the defendant Shanghai X Technology Co., Ltd. used the ancient costume model Liao’s ancient costume styling video template in its “AI face-swapping” software without permission. In this case, the plaintiff Liao has the right of portrait, and the defendant, although replacing the facial image of the character in the plaintiff’s video, can still identify the corresponding subject as the plaintiff Liao through other unmodified corresponding scenes and details. The plaintiff argues that the defendant has infringed the right of portrait. As a typical technology related to portrait, AI face-swapping technology destroys the identity between portrait and subject. On the one hand, the portrait subject has the right to refuse his or her physical image being crowned with someone else’s face; on the other hand, the portrait subject also has the right to refuse his or her facial image being grafted onto someone else’s body. Combining the constitutive elements of general portrait infringement, the constitutive elements of infringement of portrait right by AI face-swapping technology are: 1. Natural persons have the right of portrait for specific portraits; 2. The actor uses information technology means to tamper with, forge, and automatically generate the portrait of the right holder without the consent of others, regardless of whether or not he or she has a profit-making purpose, resulting in difficult-to-discriminate effects.

In addition to infringement of portrait rights, using AI face-swapping technology to produce maliciously caricatured videos and images of others and widely disseminate them on the Internet may constitute infringement of reputation rights. In actual life, there are many situations where face-swapping videos are used to ridicule stars and Internet celebrities, such as making face-swapping parodies to ridicule the skill of a certain star in singing, dancing, or playing basketball, which may constitute infringement of reputation rights.

Finally

Deep synthesis technology, as the most vital application in the field of artificial intelligence, will inevitably burst out with great productivity in the future. Therefore, compliance issues in this field must be taken seriously, which is a prerequisite for the healthy development of deep synthesis technology. In today’s legal community, pure technological neutrality theory is rarely advocated, and the value orientation of technology itself has penetrated into people’s hearts. Based on various risks of deep synthesis technology, Sa Jie team has summarized the compliance boundaries and 20 compliance obligations of relevant practitioners to assist the healthy development of the deep synthesis industry and provide reference for practitioners:

Compliance boundaries for generated content

1. Not engaging in political or pornographic activities;

2. Not generating fake news, and non-licensed entities are not allowed to disseminate news from units outside the scope of national regulations;

3. Not violating public order and good customs (such as how a third party takes away the original family property);

4. Not generating illegal and harmful information, and establishing a corresponding “feature library” to screen healthy information;

5. Not maliciously slandering the reputation of others and the commercial reputation of enterprises (the legal consequences will be sued for infringement of reputation rights);

6. For uncontrollable creative content, delete it in time to reduce social harm.

Specific compliance obligations

1. Improve the user registration system;

2. Algorithm mechanism review;

3. Scientific ethics review;

4. Information release review;

5. Data security management system;

6. Personal information protection system;

7. Anti-telecom fraud system;

8. Emergency response management system;

9. In apps or websites, publicly manage rules, platform conventions, improve service agreements, and prominently remind all parties to assume information security obligations;

10. Improve the feature library used to identify illegal and harmful information, and retain logs;

11. When discovering illegal and harmful information, there is an obligation to report to the competent authority;

12. When discovering illegal and harmful information, take disposal measures in accordance with the law and the contract with the user;

13. Full rumor refutation mechanism;

14. Set up convenient complaint and reporting channels and provide timely feedback on results;

15. Strengthen training data management;

16. Regularly review, evaluate, and verify the mechanism of synthetic algorithm generation and recommendation management;

17. Carry out security evaluations in accordance with the law on its own or entrust professional institutions to conduct security evaluations;

18. For generated content, take technical measures to add identification that does not affect user use;

19. Not to delete, tamper with, or hide deep synthesis identification;

20. Fulfill filing, change, and cancellation of filing procedures in accordance with the “Internet Information Service Algorithm Recommendation Management Regulations”.

Explanation: Blocking all articles only represents the author’s opinion and does not constitute investment advice

Original article link: https://www.bitpush.news/articles/4458064

We will continue to update Blocking; if you have any questions or suggestions, please contact us!