There’s a lot of money in voice cloning.
Case in point: ElevenLabs, a startup developing artificial intelligence tools to create and edit synthetic sounds, today announced the completion of an $80 million Series B round of financing from well-known names such as Andreessen Horowitz, former GitHub CEO Nat Friedman, and entrepreneur Daniel Gross. The investors co-led the investment. .
This round of financing also received participation from Sequoia Capital, Smash Capital, SV Angel, BroadLight Capital and Credo Ventures. ElevenLabs’ total financing reached US$101 million, and the company’s valuation exceeded US$1 billion (approximately US$100 million in June last year) . CEO Mati Staniszewski said the new funds will be used for product development, expanding ElevenLabs’ infrastructure and team, artificial intelligence research and “enhancing safety measures to ensure the responsible and ethical development of artificial intelligence technology.”
“The new funding we are raising is to solidify ElevenLabs’ position as a global leader in voice AI research and product deployment,” Staniszewski told TechCrunch in an email interview.
ElevenLabs, co-founded in 2022 by former Google machine learning engineer Piotr Dabkowski and former Palantir deployment strategist Staniszewski, launched in beta about a year ago. Staniszewski said he and Dabkovski, who grew up in Poland, were inspired by poorly dubbed American movies to create the sound-replication tool. They think artificial intelligence can do better.
Today, ElevenLabs is known for its browser-based speech generation applications that create realistic voices with adjustable tones of intonation, emotion, tempo, and other key sound characteristics. Users can enter text for free and get a recording of the text spoken by one of several preset voices. Paying customers can upload voice samples and use ElevenLabs’ voice clones to create new styles.
ElevenLabs is increasingly investing in versions of its speech generation technology aimed at creating audiobooks, dubbing movies and TV shows, and generating character voices for games and marketing campaigns.
Last year, the company released a “speech-to-speech” tool that attempts to preserve the speaker’s voice, cadence and intonation while automatically removing background noise and, in the case of movies and TV shows, translating the speech and matching it with the source material Synchronize. On the roadmap for the coming weeks is a new voiceover studio workflow with tools to generate and edit transcripts and translations, and a subscription-based mobile app that uses ElevenLabs voices to narrate web pages and text.
ElevenLabs’ innovation has landed startup customers in Paradox Interactive, the game developer whose recent projects include Cities: Skylines 2 and Stellaris, as well as publishing, media and entertainment companies such as The Washington Post. Staniszewski claims that ElevenLab user-generated information is equivalent to more than 100 years of audio, and that 41% of employees at Fortune 500 companies are using the platform.
But the publicity effect is not entirely positive.
The notorious message board 4chan, known for its conspiratorial content, used ElevenLabs’ tools to impersonate celebrities such as actress Emma Watson to share hateful messages. The Verge’s James Vincent was able to use ElevenLabs to maliciously clone voices in seconds, producing samples containing everything from threats of violence to racist and transphobic comments. At Vox, reporter Joseph Cox documented how clones were created that were powerful enough to fool bank authentication systems.
In response, ElevenLabs sought to root out users who repeatedly violated its terms of service, which prohibits abuse, and launched a tool to detect speech created on its platform. Staniszewski said that this year, ElevenLabs plans to improve the detection tool to label audio from other speech-generating AI models and work with unnamed “distributors” to make the tool available on third-party platforms.
ElevenLabs offers a range of different voices, some synthesized and some cloned from voice actors.
ElevenLabs has also faced criticism from voice actors who claim the company has used samples of their voices without their consent, which could be used to promote content they don’t endorse or spread misinformation and disinformation. In a recent Vice article, victims described how ElevenLabs was used in harassment campaigns against them, one example being the use of cloned voices to share actors’ private information (their home addresses).
And then there’s the elephant in the room: platforms like ElevenLabs pose an existential threat to the voice-over community.
Motherboard writes that voice actors are increasingly being asked to give up the rights to their voices so that clients can use artificial intelligence to generate synthesized versions that can eventually replace them—sometimes without corresponding compensation. The fear is that voice-over jobs – especially cheap, entry-level jobs – will eventually be replaced by AI-generated voices, and actors will have no recourse.
Some platforms are trying to strike a balance. Earlier this month, ElevenLabs competitor Replica Studios signed a deal with SAG-AFTRA to create and license digital reproductions of the voices of Media Artists Alliance members. The arrangements set out “fair” and “ethical” terms and conditions to ensure performers’ consent and negotiated terms for the use of digital voice avatars in new productions, the organizations said in a release.
However, even that dissatisfied some voice actors – including members of SAG-AFTRA themselves.
ElevenLabs’ solution is a voice marketplace. The marketplace is currently in alpha stage and will become more widespread in the coming weeks, allowing users to create sounds, verify and share it. Staniszewski said the original creator is compensated when others use the sound.
“Users always retain control over their voice availability and compensation terms,” he added. “The marketplace is designed to align AI advancements with established industry practices while also bringing diverse voices to ElevenLabs’ platform.”
However, voice actors may take issue with the fact that ElevenLabs doesn’t pay cash – at least not yet. The current setup gives creators credit for ElevenLabs’ premium service (which I bet some people find ironic).
Perhaps this will change in the future, as ElevenLabs (currently one of the best-funded synthetic speech startups) attempts to defeat upstart competitors such as Papercup, Deepdub, ElevenLabs, Acapela, Respeecher, and Voice.ai, as well as large tech companies , such as Amazon, Microsoft and Google. Regardless, ElevenLabs plans to increase its headcount from 40 to 100 employees by the end of this year, and intends to stay and make waves in the fast-growing synthetic speech market.
from Tech Empire Solutions https://techempiresolutions.com/voice-cloning-startup-elevenlabs-raises-80-million-achieves-unicorn-status/
via https://techempiresolutions.com/
No comments:
Post a Comment