Архив рубрики: Artificial Intelligence

Auto Added by WPeMatico

Twelve Labs lands $12M for AI that understands the context of videos

To Jae Lee, a data scientist by training, it never made sense that video — which has become an enormous part of our lives, what with the rise of platforms like TikTok, Vimeo and YouTube — was difficult to search across due to the technical barriers posed by context understanding. Searching the titles, descriptions and tags of videos was always easy enough, requiring no more than a basic algorithm. But searching within videos for specific moments and scenes was long beyond the capabilities of tech, particularly if those moments and scenes weren’t labeled in an obvious way.
To solve this problem, Lee, alongside friends from the tech industry, built a cloud service for video search and understanding. It became Twelve Labs, which went on to raise $17 million in venture capital — $12 million of which came from a seed extension round that closed today. Radical Ventures led the extension with participation from Index Ventures, WndrCo, Spring Ventures, Weights & Biases CEO Lukas Biewald and others, Lee told TechCrunch in an email.
“The vision of Twelve Labs is to help developers build programs that can see, listen, and understand the world as we do by giving them the most powerful video understanding infrastructure,” Lee said.
A demo of the Twelve Labs platform’s capabilities. Image Credits: Twelve Labs
Twelve Labs, which is currently in closed beta, uses AI to attempt to extract “rich information” from videos such as movement and actions, objects and people, sound, text on screen, and speech to identify the relationships between them. The platform converts these various elements into mathematical representations called “vectors” and forms “temporal connections” between frames, enabling applications like video scene search.
“As a part of achieving the company’s vision to help developers create intelligent video applications, the Twelve Labs team is building ‘foundation models’ for multimodal video understanding,” Lee said. “Developers will be able to access these models through a suite of APIs, performing not only semantic search but also other tasks such as long-form video ‘chapterization,’ summary generation and video question and answering.”
Google takes a similar approach to video understanding with its MUM AI system, which the company uses to power video recommendations across Google Search and YouTube by picking out subjects in videos (e.g., “acrylic painting materials”) based on the audio, text and visual content. But while the tech might be comparable, Twelve Labs is one of the first vendors to market with it; Google has opted to keep MUM internal, declining to make it available through a public-facing API.
That being said, Google, as well as Microsoft and Amazon, offer services (i.e., Google Cloud Video AI, Azure Video Indexer and AWS Rekognition) that recognize objects, places and actions in videos and extract rich metadata at the frame level. There’s also Reminiz, a French computer vision startup that claims to be able to index any type of video and add tags to both recorded and live-streamed content. But Lee asserts that Twelve Labs is sufficiently differentiated — in part because its platform allows customers to fine-tune the AI to specific categories of video content.
Mockup of API for fine-tuning the model to work better with salad-related content. Image Credits: Twelve Labs
“What we’ve found is that narrow AI products built to detect specific problems show high accuracy in their ideal scenarios in a controlled setting, but don’t scale so well to messy real-world data,” Lee said. “They act more as a rule-based system, and therefore lack the ability to generalize when variances occur. We also see this as a limitation rooted in lack of context understanding. Understanding of context is what gives humans the unique ability to make generalizations across seemingly different situations in the real world, and this is where Twelve Labs stands alone.”
Beyond search, Lee says Twelve Labs’ technology can drive things like ad insertion and content moderation, intelligently figuring out, for example, which videos showing knives are violent versus instructional. It can also be used for media analytics and real-time feedback, he says, and to automatically generate highlight reels from videos.
A little over a year after its founding (March 2021), Twelve Labs has paying customers — Lee wouldn’t reveal how many exactly — and a multiyear contract with Oracle to train AI models using Oracle’s cloud infrastructure. Looking ahead, the startup plans to invest in building out its tech and expanding its team. (Lee declined to reveal the current size of Twelve Labs’ workforce, but LinkedIn data shows it’s roughly 18 people.)
“For most companies, despite the huge value that can be attained through large models, it really does not make sense for them to train, operate and maintain these models themselves. By leveraging a Twelve Labs platform, any organization can leverage powerful video understanding capabilities with just a few intuitive API calls,” Lee said. “The future direction of AI innovation is heading straight towards multimodal video understanding, and Twelve Labs is well positioned to push the boundaries even further in 2023.”
Twelve Labs lands $12M for AI that understands the context of videos by Kyle Wiggers originally published on TechCrunch
Twelve Labs lands $12M for AI that understands the context of videos

Regie secures $10M to generate marketing copy using AI

Regie.ai, a startup using OpenAI’s GPT-3 text-generating system to create sales and marketing content for brands, today announced that it raised $10 million in Series A funding led by Scale Venture Partners with participation from Foundation Capital, South Park Commons, Day One Ventures and prominent angel investors. The fresh investment comes as VCs see a growing opportunity in AI-powered, copy-generating adtech companies, whose tech promises to save time while potentially increasing personalization.
Regie was founded in 2020 by Matt Millen and Srinath Sridhar. Previously a software engineer at Google and Meta, Sridhar is a data scientist by trade, having developed enterprise-scale AI systems that detect duplicate images and rank search results. Millen was formerly a VP at T-Mobile, leading the national sales teams (e.g., strategic accounts and public sector).
With Regie, Sridhar says he and Millen aimed to create a way for companies to communicate with their customers via channels like email, social media, text, podcasts, online advertising and more. Because companies have so many platforms and mediums at their disposal to speak with customers, he notes, it can be a challenge for content marketers to produce continuously compelling content to reach their customers.
“The way content is getting generated has fundamentally changed,” Sridhar told TechCrunch in an email interview. “Marketers and copywriters working in the enterprise … increasingly [need] to produce and manage content and content workflows at scale.”
Regie uses GPT-3 to power its service — the same GPT-3 that can generate poetry, prose and academic papers. But it’s a “flavor” of GPT-3 fine-tuned on a training data set of roughly 20,000 sales sequences (the series of steps to convert prospects into paying customers) and nearly 100 million sales emails. Also in the mix are custom language systems built by Regie to reflect brands and their messaging, designed to be integrated with existing sale platforms like Outreach, HubSpot, and Salesloft.
Image Credits: Regie
Lest the systems spew problematic language, Regie says that every system goes through “human curation” and vetting before being released. The startup also claims to train the systems on “inclusive” language and test them for biases, like bias against certain demographic groups.
Customers can use Regie to generate original, optimized-for-search-engines content or create custom sales sequences. The platform also offers blog- and social-media-post-authoring tools for personalizing messages, as well as a Chrome extension that analyzes the “quality” of emails that customers send — and optionally rewrites the text.
“Generative AI is completely disrupting the way content is created today. The biggest competitors of Regie would be the large content authoring and management platforms that will be completely redesigned AI first going forward,” Sridhar said confidently. “For example, Adobe’s suite of products including Acrobat, Illustrator, Photoshop, now Figma as well as Adobe Experience Cloud will start to get outdated as Regie continues to build on an intelligent content creation and management platform for the enterprise.”
More immediately, Regie competes with vendors like Jasper, Phrasee, Copysmith and Copy.ai — all of which tap AI to generate bespoke marketing copy. But Sridhar argues that Regie is a more vertical platform that caters to go-to-market teams in the enterprise while combining text, images and workflows into a single glass pane.
“Generative AI is such a paradigm shift that not only productivity and top-line of companies will go up as a result, but the bottom line will also go down simultaneously. There are very few products that can improve both sides of that financial equation,” Sridhar continued. “So if a company wants to reduce costs because they want to assimilate sales tools, or reduce outsourced writing while simultaneously increasing revenue, Regie can do that. If you are an outsourced marketing agency looking to retain more customers and efficiently generate content at scale, Regie can definitely do that for agencies as well.”
The company currently has more than 70 software-as-a-service customers on annual contracts, including AT&T, Sophos, Okta and Crunchbase. Sridhar didn’t reveal revenue but said that he expects the 25-person company to grow “meaningfully” this year.
“This is a revolutionary new field. And as always, adoption will require educating the users,” Sridhar said. “It is clear to us as practitioners that the world has changed. But it will take time for others to get their hands dirty and convince themselves that this is happening — and that it is a very positive development. So we have to be patient in educating the industry. We also have to show that content quality isn’t compromised and that it can perform better and be maintained more consistently with the strategic application of AI.”
To date, Regie has raised $14.8 million.
Regie secures $10M to generate marketing copy using AI by Kyle Wiggers originally published on TechCrunch
Regie secures $10M to generate marketing copy using AI

OpenAI begins allowing users to edit faces with DALL-E 2

After initially disabling the capability, OpenAI today announced that customers with access to DALL-E 2 can upload people’s faces to edit them using the AI-powered image-generating system. Previously, OpenAI only allowed users to work with and share photorealistic faces and banned the uploading of any photo that might depict a real person, including photos of prominent celebrities and public figures.
OpenAI claims that improvements to its safety system made the face-editing feature possible by “minimizing the potential of harm” from deepfakes as well as attempts to create sexual, political and violent content. In an email to customers, the company wrote:
Many of you have told us that you miss using DALL-E to dream up outfits and hairstyles on yourselves and edit the backgrounds of family photos. A reconstructive surgeon told us that he’d been using DALL-E to help his patients visualize results. And filmmakers have told us that they want to be able to edit images of scenes with people to help speed up their creative processes … [We] built new detection and response techniques to stop misuse.
The change in policy isn’t opening the floodgates necessarily. OpenAI’s terms of service will continue to prohibit uploading pictures of people without their consent or images that users don’t have the rights to — although it’s not clear how consistent the company’s historically been about enforcing those policies.
In any case, it’ll be a true test of OpenAI’s filtering technology, which some customers in the past have complained about being overzealous and somewhat inaccurate. Deepfakes come in many flavors, from fake vacation photos to presidents of war-torn countries. Accounting for every emerging form of abuse will be a never-ending battle, in some cases with very high stakes.
No doubt, OpenAI — which has the backing of Microsoft and notable VC firms including Khosla Ventures — is eager to avoid the controversy associated with Stability AI’s Stable Diffusion, an image-generating system that’s available in an open source format without any restrictions. As TechCrunch recently wrote about, it didn’t take long before Stable Diffusion — which can also edit face images — was being used by some to create pornographic, nonconsensual deepfakes of celebrities like Emma Watson.
So far, OpenAI has positioned itself as a brand-friendly, buttoned-up alternative to the no-holds-barred Stability AI. And with the constraints around the new face editing feature for DALL-E 2, the company is maintaining the status quo.
DALL-E 2 remains in invite-only beta. In late August, OpenAI announced that over a million people are using the service.
OpenAI begins allowing users to edit faces with DALL-E 2 by Kyle Wiggers originally published on TechCrunch
OpenAI begins allowing users to edit faces with DALL-E 2

AI is getting better at generating porn. We might not be prepared for the consequences.

A red-headed woman stands on the moon, her face obscured. Her naked body looks like it belongs on a poster you’d find on a hormonal teenager’s bedroom wall — that is, until you reach her torso, where three arms spit out of her shoulders.
AI-powered systems like Stable Diffusion, which translate text prompts into pictures, have been used by brands and artists to create concept images, award-winning (albeit controversial) prints and full-blown marketing campaigns.
But some users, intent on exploring the systems’ murkier side, have been testing them for a different sort of use case: porn.
AI porn is about as unsettling and imperfect as you’d expect (that red-head on the moon was likely not generated by someone with an extra arm fetish). But as the tech continues to improve, it will evoke challenging questions for AI ethicists and sex workers alike.
Pornography created using the latest image-generating systems first arrived on the scene via the discussion boards 4chan and Reddit earlier this month, after a member of 4chan leaked the open source Stable Diffusion system ahead of its official release. Then, last week, what appears to be one of the first websites dedicated to high-fidelity AI porn generation launched.
Called Porn Pen, the website allows users to customize the appearance of nude AI-generated models — all of which are women — using toggleable tags like “babe,” “lingerie model,” “chubby,” ethnicities (e.g. “Russian” and “Latina”) and backdrops (e.g. “bedroom,” “shower” and wildcards like “moon”). Buttons capture models from the front, back or side, and change the appearance of the generated photo (e.g. “film photo,” “mirror selfie”). There must be a bug on the mirror selfies, though, because in the feed of user-generated images, some mirrors don’t actually reflect a person — but of course, these models are not people at all. Porn Pen functions like “This Person Does Not Exist,” only it’s NSFW.
On Y Combinator’s Hacker News forum, a user purporting to be the creator describes Porn Pen as an “experiment” using cutting-edge text-to-image models. “I explicitly removed the ability to specify custom text to avoid harmful imagery from being generated,” they wrote. “New tags will be added once the prompt-engineering algorithm is fine-tuned further.” The creator did not respond to TechCrunch’s request for comment.
But Porn Pen raises a host of ethical questions, like biases in image-generating systems and the sources of the data from which they arose. Beyond the technical implications, one wonders whether new tech to create customized porn — assuming it catches on — could hurt adult content creators who make a living doing the same.
“I think it’s somewhat inevitable that this would come to exist when [OpenAI’s] DALL-E did,” Os Keyes, a PhD candidate at Seattle University, told TechCrunch via email. “But it’s still depressing how both the options and defaults replicate a very heteronormative and male gaze.”
Ashley, a sex worker and peer organizer who works on cases involving content moderation, thinks that the content generated by Porn Pen isn’t a threat to sex workers in its current state.
“There is endless media out there,” said Ashley, who did not want her last name to be published for fear of being harassed for their job. “But people differentiate themselves not by just making the best media, but also by being an accessible, interesting person. It’s going to be a long time before AI can replace that.”
On existing monetizable porn sites like OnlyFans and ManyVids, adult creators must verify their age and identity so that the company knows they are consenting adults. AI-generated porn models can’t do this, of course, because they aren’t real.
Ashley worries, though, that if porn sites crack down on AI porn, it might lead to harsher restrictions for sex workers, who are already facing increased regulation from legislation like SESTA/FOSTA. Congress introduced the Safe Sex Workers Study Act in 2019 to examine the affects of this legislation, which makes online sex work more difficult. This study found that “community organizations [had] reported increased homelessness of sex workers” after losing the “economic stability provided by access to online platforms.”
“SESTA was sold as fighting child sex trafficking, but it created a new criminal law about prostitution that had nothing about age,” Ashley said.
Currently, few laws around the world pertain to deepfaked porn. In the U.S., only Virginia and California have regulations restricting certain uses of faked and deepfaked pornographic media.
Systems such as Stable Diffusion “learn” to generate images from text by example. Fed billions of pictures labeled with annotations that indicate their content — for example, a picture of a dog labeled “Dachshund, wide-angle lens” — the systems learn that specific words and phrases refer to specific art styles, aesthetics, locations and so on.
This works relatively well in practice. A prompt like “a bird painting in the style of Van Gogh” will predictably yield a Van Gogh-esque image depicting a bird. But it gets trickier when the prompts are vaguer, refer to stereotypes or deal with subject matter with which the systems aren’t familiar.
For example, Porn Pen sometimes generates images without a person at all — presumably a failure of the system to understand the prompt. Other times, as alluded to earlier, it shows physically improbable models, typically with extra limbs, nipples in unusual places and contorted flesh.
“By definition [these systems are] going to represent those whose bodies are accepted and valued in mainstream society,” Keyes said, noting that Porn Pen only has categories for cisnormative people. “It’s not surprising to me that you’d end up with a disproportionately high number of women, for example.”
While Stable Diffusion, one of the systems likely underpinning Porn Pen, has relatively few “NSFW” images in its training dataset, early experiments from Redditors and 4chan users show that it’s quite competent at generating pornographic deepfakes of celebrities (Porn Pen — perhaps not coincidentally — has a “celebrity” option). And because it’s open source, there’d be nothing to prevent Porn Pen’s creator from fine-tuning the system on additional nude images.
“It’s definitely not great to generate [porn] of an existing person,” Ashley said. “It can be used to harass them.”
Deepfake porn is often created to threaten and harass people. These images are almost always developed without the subject’s consent out of malicious intent. In 2019, the research company Sensity AI found that 96% of deepfake videos online were non-consensual porn.
Mike Cook, an AI researcher who’s a part of the Knives and Paintbrushes collective, says that there’s a possibility the dataset includes people who’ve not consented to their image being used for training in this way, including sex workers.
“Many of [the people in the nudes in the training data] may derive their income from producing pornography or pornography-adjacent content,” Cook said. “Just like fine artists, musicians or journalists, the works these people have produced are being used to create systems that also undercut their ability to earn a living in the future.”
In theory, a porn actor could use copyright protections, defamation and potentially even human rights laws to fight the creator of a deepfaked image. But as a piece in MIT Technology Review notes, gathering evidence in support of the legal argument can prove to be a massive challenge.
When more primitive AI tools popularized deepfaked porn several years ago, a Wired investigation found that nonconsensual deepfake videos were racking up millions of views on mainstream porn sites like Pornhub. Other deepfaked works found a home on sites akin to Porn Pen — according to Sensity data, the top four deepfake porn websites received more than 134 million views in 2018.
“AI image synthesis is now a widespread and accessible technology, and I don’t think anyone is really prepared for the implications of this ubiquity,” Cook continued. “In my opinion, we have rushed very, very far into the unknown in the last few years with little regard for the impact of this technology.”
To Cook’s point, one of the most popular sites for AI-generated porn expanded late last year through partner agreements, referrals and an API, allowing the service — which hosts hundreds of nonconsensual deepfakes — to survive bans on its payments infrastructure. And in 2020, researchers discovered a Telegram bot that generated abusive deepfake images of more than 100,000 women, including underage girls.
“I think we’ll see a lot more people testing the limits of both the technology and society’s boundaries in the coming decade,” Cook said. “We must accept some responsibility for this and work to educate people about the ramifications of what they are doing.”
AI is getting better at generating porn. We might not be prepared for the consequences.

Datch secures $10M to build voice assistants to factory floors

Datch, a company that develops AI-powered voice assistants for industrial customers, today announced that it raised $10 million in a Series A round led by Blackhorn Ventures. The proceeds will be used to expand operations, CEO Mark Fosdike said, as well as develop new software support, tools and capabilities.
Datch started when Fosdike, who has a background in aerospace engineering, met two former Siemens engineers — Aric Thorn and Ben Purcell. They came to the collective realization that voice products built for business customers have to overcome business-specific challenges, like understanding jargon, acronyms and syntax unique to particular customers.
“The way we extract information from systems changes every year, but the way we input information — especially in the industrial world — hasn’t changed since the invention of the keyboard and database,” Fosdike said. “The industrial world had been left in the dark for years, and we knew that developing a technology with voice-visual AI would help light the way for these factories.”
The voice assistants that Datch builds leverage AI to collect and structure data from users in a factory or in the field, parsing commands like “Report an issue for the Line 1 Spot Welder. I estimate it will take half a day to fix.” They run on a smartphone and link to existing systems to write and read records, including records from enterprise resource and asset management platforms.
Datch’s assistants provide a timeline of events and can capture data without an internet connection; they auto-sync once back online. Using them, workers can fill out company forms, create and update work orders, assign tasks and search through company records all via voice.
Fosdike didn’t go into detail about how Datch treats the voice data, save that it encrypts data both in-transit and at rest and performs daily backups.
“We have to employ a lot of tight, automated feedback loops to train the voice and [language] data, and so everyone’s interaction with Datch is slightly different, depending on the company and team they work within,” Fosdike explained. “Customers are exploring different use cases such as using the [language] data in predictive maintenance, automated classification of cause codes, and using the voice data to predict worker fatigue before it becomes a critical safety risk.”
That last bit about predicting worker fatigue is a little suspect. The idea that conditions like tiredness can be detected in a person’s voice isn’t a new one, but some researchers believe it’s unlikely AI can flag them with 100% accuracy. After all, people express tiredness in different ways, depending not only on the workplace environment but on their sex and cultural, ethnic and demographic backgrounds.
The tiredness-detecting scenario aside, Fosdike asserts that Datch’s technology is helping industrial clients get ahead of turbulence in the economy by “vastly improving” the efficiency of their operations. Frontline staff typically have to work with reporting tools that aren’t intuitive, he notes, and in many cases, voice makes for a less cumbersome, faster alternative form of input.
“We help frontline workers with productivity and solve the pain point of time wasted on their reports by decreasing the process time,” Fosdike said. “Industrial companies are fast realizing that to keep up with demand or position themselves to withstand a global pandemic, they need to find a way to scale with more than just peoplepower. Our AI offers these companies an efficient solution in a fraction of the time and with less overhead needed.”
Datch competes with Rain, Aiqudo and Onvego, all of which are developing voice technologies for industrial customers. Deloitte’s Maxwell, Genba and Athena are rivals in Fosdike’s eyes, as as well. But business remains steady — Datch counts ConEd, Singapore Airlines, ABB Robotics and the New York Power Authority among its clients.
“We raised this latest round earlier than expected due to the influx of demand from the market. The timing is right to capitalize on both the post-COVID boom in digital transformation as well as corporate investments driven by the infrastructure bill,” Fosdike said, referring to the $1 trillion package U.S. lawmakers passed last November. “Currently we have a team of 20, and plan to use the funds to grow to 55 to 60 people, scaling to roughly 40 by the end of the year.”
To date, Datch has raised $15 million in venture capital.
Datch secures $10M to build voice assistants to factory floors