To Jae Lee, a data scientist by training, it never made sense that video — which has become an enormous part of our lives, what with the rise of platforms like TikTok, Vimeo and YouTube — was difficult to search across due to the technical barriers posed by context understanding. Searching the titles, descriptions and tags of videos was always easy enough, requiring no more than a basic algorithm. But searching within videos for specific moments and scenes was long beyond the capabilities of tech, particularly if those moments and scenes weren’t labeled in an obvious way.
To solve this problem, Lee, alongside friends from the tech industry, built a cloud service for video search and understanding. It became Twelve Labs, which went on to raise $17 million in venture capital — $12 million of which came from a seed extension round that closed today. Radical Ventures led the extension with participation from Index Ventures, WndrCo, Spring Ventures, Weights & Biases CEO Lukas Biewald and others, Lee told TechCrunch in an email.
“The vision of Twelve Labs is to help developers build programs that can see, listen, and understand the world as we do by giving them the most powerful video understanding infrastructure,” Lee said.
A demo of the Twelve Labs platform’s capabilities. Image Credits: Twelve Labs
Twelve Labs, which is currently in closed beta, uses AI to attempt to extract “rich information” from videos such as movement and actions, objects and people, sound, text on screen, and speech to identify the relationships between them. The platform converts these various elements into mathematical representations called “vectors” and forms “temporal connections” between frames, enabling applications like video scene search.
“As a part of achieving the company’s vision to help developers create intelligent video applications, the Twelve Labs team is building ‘foundation models’ for multimodal video understanding,” Lee said. “Developers will be able to access these models through a suite of APIs, performing not only semantic search but also other tasks such as long-form video ‘chapterization,’ summary generation and video question and answering.”
Google takes a similar approach to video understanding with its MUM AI system, which the company uses to power video recommendations across Google Search and YouTube by picking out subjects in videos (e.g., “acrylic painting materials”) based on the audio, text and visual content. But while the tech might be comparable, Twelve Labs is one of the first vendors to market with it; Google has opted to keep MUM internal, declining to make it available through a public-facing API.
That being said, Google, as well as Microsoft and Amazon, offer services (i.e., Google Cloud Video AI, Azure Video Indexer and AWS Rekognition) that recognize objects, places and actions in videos and extract rich metadata at the frame level. There’s also Reminiz, a French computer vision startup that claims to be able to index any type of video and add tags to both recorded and live-streamed content. But Lee asserts that Twelve Labs is sufficiently differentiated — in part because its platform allows customers to fine-tune the AI to specific categories of video content.
Mockup of API for fine-tuning the model to work better with salad-related content. Image Credits: Twelve Labs
“What we’ve found is that narrow AI products built to detect specific problems show high accuracy in their ideal scenarios in a controlled setting, but don’t scale so well to messy real-world data,” Lee said. “They act more as a rule-based system, and therefore lack the ability to generalize when variances occur. We also see this as a limitation rooted in lack of context understanding. Understanding of context is what gives humans the unique ability to make generalizations across seemingly different situations in the real world, and this is where Twelve Labs stands alone.”
Beyond search, Lee says Twelve Labs’ technology can drive things like ad insertion and content moderation, intelligently figuring out, for example, which videos showing knives are violent versus instructional. It can also be used for media analytics and real-time feedback, he says, and to automatically generate highlight reels from videos.
A little over a year after its founding (March 2021), Twelve Labs has paying customers — Lee wouldn’t reveal how many exactly — and a multiyear contract with Oracle to train AI models using Oracle’s cloud infrastructure. Looking ahead, the startup plans to invest in building out its tech and expanding its team. (Lee declined to reveal the current size of Twelve Labs’ workforce, but LinkedIn data shows it’s roughly 18 people.)
“For most companies, despite the huge value that can be attained through large models, it really does not make sense for them to train, operate and maintain these models themselves. By leveraging a Twelve Labs platform, any organization can leverage powerful video understanding capabilities with just a few intuitive API calls,” Lee said. “The future direction of AI innovation is heading straight towards multimodal video understanding, and Twelve Labs is well positioned to push the boundaries even further in 2023.”
Twelve Labs lands $12M for AI that understands the context of videos by Kyle Wiggers originally published on TechCrunch
Twelve Labs lands $12M for AI that understands the context of videos
Архив метки: Index Ventures
Social network for women Peanut raises $12M Series A amid pandemic
Добавить комментарий
Peanut, an app that began as a tool for finding new mom friends, has evolved into a social network now used by 1.6 million women to discuss a range of topics, from pregnancy and parenthood to marriage and menopause, and everything in between. On the heels of significant growth in online networking fueled by the COVID-19 pandemic, the company is today announcing the close of a $12 million Series A round of funding, led by EQT Ventures, a multi-stage VC firm that invests in companies across Europe and the U.S.
Index Ventures and Female Founders Fund also participated, bringing Peanut’s total raise to date to $21.8 million.
The round itself closed just weeks ago — arriving at a time when the coronavirus pandemic is impacting the startup world, often drying up venture capital for emerging companies. Some startups, as a result, have laid off employees to self-sustain, while others have sought exits or even folded.
Peanut, on the other hand, has seen rapid growth for its platform as women looked for a supportive online environment to discuss their own concerns over how COVID-19 was impacting their lives.
Many women participating in Peanut’s newer “Trying to Conceive” group, for example, worried about their canceled IVF rounds and how to plan for the future. Current moms-to-be wanted to hear from others about how COVID-19 would impact their hospital delivery plans. And others stuck working at home with kids looked for advice and coping strategies.
Since the outbreak, Peanut has seen engagement across its app increase by 30% and content consumption increase by 40%. Its total community also grew from 1 million users in December 2019 to now 1.6 million, as of April.
“We’re really lucky in that we’re growing and that we are, for the most part, untouched by what’s happening,” says Peanut founder and CEO Michelle Kennedy. “And actually, if anyone needed community more, it’s now,” she added.
Though the pandemic has sent the app’s usage skyrocketing, it has also readjusted Peanut’s priorities with regard to its roadmap.
Most notably, its friend-finding feature needs a rethink.
Peanut originally worked as a sort of “Tinder for mom friends” — an idea that arose from Kennedy’s personal experience with how difficult it was to forge female friendships after motherhood. As the former deputy CEO at dating app Badoo and an inaugural board member at Bumble, she brought her extensive experience in matchmaking apps to Peanut, which uses a similar swipe-based mechanism.
But COVID-19 has up-ended this side of Peanut’s business. Today, Peanut users are meeting in Zoom chat rooms to hangout or play games, but not in person.
Kennedy says the company will try to meet these users where they are with the development of more video networking features, potentially with technology built in-house. Other plans for the new capital include improvements to the social discovery aspects of its app, the development of a web version of Peanut, and the creation of more groups beyond those focused on fertility and motherhood, which have so far been core to the Peanut experience.
Specifically, the company soon plans to launch a new community focused on women living with menopause, an experience that will reach more than a billion women by 2025. Despite the fact that all women with ovaries will go through menopause, there are relatively few online communities dedicated to it — which Peanut sees as an untapped market.
Peanut’s real strength, however, is not in the types of communities it grows on its platform, but how they’re created.
There has not yet been a social network that focused on “building a platform for women, thinking about women’s needs and built by a women,” explains Kennedy. “So what we end up doing is using things that already exist — trying to twist them and mold them into what we need, and never getting it exactly right,” she says. “We can do better than that.”
One small example of this is the recent launch of Peanut’s “Mute Keywords” feature that allows women to remove certain types of discussions from their feeds and notifications. Some women used this to create a coronavirus-free news feed that focused on other aspects of motherhood. Others who were trying to conceive muted conversations around “pregnancy,” which they found emotionally triggering.
With the Series A’s close, Peanut says Naza Metghalchi from EQT Ventures joins the company’s majority-female board, alongside Hannah Seal from existing investor Index Ventures.
“Peanut’s user engagement metrics are a testament to the app’s ability to act as a true emotional companion throughout women’s journeys,” said Naza Metghalchi, venture lead and investment advisor at EQT Ventures, in a statement. “The EQT Ventures team is excited to partner with Michelle and continue to grow Peanut into a platform that serves all women at different life milestones, exploring topics beyond fertility and motherhood which have already seen such huge traction.”
The additional funding allows London-based Peanut to expand its business and hire more engineers to join its current team of just 16.
“I think having closed a round in this climate is great for the team,” says Kennedy. “It’s also great for the community because it means that we can grow the team, build quicker, build faster and develop the product more quickly,” she adds.
Social network for women Peanut raises $12M Series A amid pandemic