Prior to this industry, I spent three years in financial crime compliance at a major bank, meaning I’ve seen more fraud and fraud than most people will ever see. This makes me instinctively wary of anything that convincingly imitates the human voice, and thus Kyutai released the Pocket TTSA sound cloning model small enough to run with a CPU, I had to try it myself rather than take videos at face value.
What I found in less than fifteen minutes, from an empty terminal to a convincing clone of my own voice, left me more disturbed than I expected. The question I am left with is whether bank security infrastructure is sufficiently prepared for what the coming wave of generative AI might bring.
Installation was pretty quick
Anyone with a laptop and internet access can do it
I’m going to be brutally honest about how I set this up. I didn’t reverse-engineer anything, and I didn’t need a working knowledge of command-line interfaces to get it to work. I asked Google Gemini I walked through the setup and it gave me the complete, correct sequence to install the package, fix the PATH problem that Windows doesn’t automatically handle, register a clean instance, and run a command. That was literally all it took. Nothing about the whole process requires technical expertise. I had to ask the right LLM the right questions and follow the instructions, which in retrospect lowers the level of expertise that bank security infrastructure generally assumes malicious actors are equipped with.
The technical bits below were just as anticlimactic, if not more so. This includes running:
pip install pocket-tts
A PATH fix followed and I was ready to clone. I recorded the twenty seconds I spoke, reduced the .WAV to five, and ran a generation command with my clip as the reference sound. The first run was aborted to download the model itself, which took about 200MB Hugging Face. At this point I was ready to run the model.
Five seconds of audio and a convincing clone
Only budget Ryzen CPU for it
Once the model was cached, the run was effectively instantaneous. My terminal reported a full operational processing time of 46 milliseconds and an average generation step time of 15 milliseconds. I typed the sentence I wanted, executed the command, and the audio was ready before I could take my hands off the keyboard. All this happened to me Ryzen 5 7600 CPU with a dedicated GPU doing nothing. This was also expected behavior, as Kyutai’s documentation also states that GPU acceleration in their tests did not make a measurable difference in development time, as the model is already small enough that a handful of CPU cores are sufficient for the task.
The overall efficiency and cost of this setup is something I can’t pass up. Most forms of fraud have always cost patience. Previously, Deepfake audio scams relied on pre-recorded clips, and the slow generation time gave victims time to hesitate, as well as respond accordingly if something went wrong instinctively. The process of doing this was also tedious, resource-intensive, and not perfectly repeatable in every case. A model capable of cloning audio in milliseconds eliminates that window and increases risk in the process.
Voice-based authentication systems are not ready for this
Security infrastructure must mature rapidly
In many retail banks, phone-based voice authentication is often classified as a high-level security verification mechanism. Clearing this level of security allows one to perform many actions that other, lighter security authentication measures cannot perform. Some of these actions pose significant risks to the customer by initiating a wire transfer, changing the account address, redirecting a replacement debit or credit card, changing authorized signatures on business bank accounts, and even changing the level of online banking capabilities. Banks often rely on this system because voice verification uses biometric authentication, which often involves higher levels of security clearance.
Compounding the risk is the fact that a five-second clip can be taken from a voicemail, social media video or recorded phone call without consent. All of these samples are now sufficient to produce a suitable sound, as this form of generation is clearly possible within milliseconds. If that isn’t bad enough in itself, what comes next is just as disturbing. Once the security process (often handled by automated IVR systems) is cleared, the caller can go directly back to their voice for the rest of the conversation.
Voice ID verification should now be less relied upon in the public interest
Many institutions that handle sensitive data, such as banks, insurers and government agencies, rely on forms of voice-based verification systems and define them as trusted because they are based on unique biometric characteristics. For a long time, these institutions were thought to require resources to believably fake the human voice cyber threat actors and cheaters just don’t have it. This assumption may no longer hold, and if so, it is important for those institutions to reconsider the verification systems around which they have built their risk appetite.
