{"id":101670,"date":"2022-12-06T08:03:51","date_gmt":"2022-12-06T05:03:51","guid":{"rendered":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/"},"modified":"2022-12-06T08:03:51","modified_gmt":"2022-12-06T05:03:51","slug":"twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos","status":"publish","type":"post","link":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/","title":{"rendered":"Twelve Labs lands $12M for AI that understands the context of videos"},"content":{"rendered":"

To Jae Lee, a data scientist by training, it never made sense that video \u2014 which has become an enormous part of our lives, what with the rise of platforms like TikTok, Vimeo and YouTube \u2014 was difficult to search across due to the technical barriers posed by context understanding. Searching the titles, descriptions and tags of videos was always easy enough, requiring no more than a basic algorithm. But searching within videos for specific moments and scenes was long beyond the capabilities of tech, particularly if those moments and scenes weren’t labeled in an obvious way.
\nTo solve this problem, Lee, alongside friends from the tech industry, built a cloud service for video search and understanding. It became Twelve Labs, which went on to raise $17 million in venture capital \u2014 $12 million of which came from a seed extension round that closed today. Radical Ventures led the extension with participation from Index Ventures, WndrCo, Spring Ventures, Weights & Biases CEO Lukas Biewald and others, Lee told TechCrunch in an email.
\n\u201cThe vision of Twelve Labs is to help developers build programs that can see, listen, and understand the world as we do by giving them the most powerful video understanding infrastructure,\u201d Lee said.
\nA demo of the Twelve Labs platform’s capabilities. Image Credits: Twelve Labs
\nTwelve Labs, which is currently in closed beta, uses AI to attempt to extract \u201crich information\u201d from videos such as movement and actions, objects and people, sound, text on screen, and speech to identify the relationships between them. The platform converts these various elements into mathematical representations called \u201cvectors\u201d and forms \u201ctemporal connections\u201d between frames, enabling applications like video scene search.
\n\u201cAs a part of achieving the company’s vision to help developers create intelligent video applications, the Twelve Labs team is building \u2018foundation models’ for multimodal video understanding,\u201d Lee said. \u201cDevelopers will be able to access these models through a suite of APIs, performing not only semantic search but also other tasks such as long-form video \u2018chapterization,’ summary generation and video question and answering.\u201d
\nGoogle takes a similar approach to video understanding with its MUM AI system, which the company uses to power video recommendations across Google Search and YouTube by picking out subjects in videos (e.g., \u201cacrylic painting materials\u201d) based on the audio, text and visual content. But while the tech might be comparable, Twelve Labs is one of the first vendors to market with it; Google has opted to keep MUM internal, declining to make it available through a public-facing API.
\nThat being said, Google, as well as Microsoft and Amazon, offer services (i.e., Google Cloud Video AI, Azure Video Indexer and AWS Rekognition) that recognize objects, places and actions in videos and extract rich metadata at the frame level. There’s also Reminiz, a French computer vision startup that claims to be able to index any type of video and add tags to both recorded and live-streamed content. But Lee asserts that Twelve Labs is sufficiently differentiated \u2014 in part because its platform allows customers to fine-tune the AI to specific categories of video content.
\nMockup of API for fine-tuning the model to work better with salad-related content. Image Credits: Twelve Labs
\n\u201cWhat we’ve found is that narrow AI products built to detect specific problems show high accuracy in their ideal scenarios in a controlled setting, but don’t scale so well to messy real-world data,\u201d Lee said. \u201cThey act more as a rule-based system, and therefore lack the ability to generalize when variances occur. We also see this as a limitation rooted in lack of context understanding. Understanding of context is what gives humans the unique ability to make generalizations across seemingly different situations in the real world, and this is where Twelve Labs stands alone.\u201d
\nBeyond search, Lee says Twelve Labs’ technology can drive things like ad insertion and content moderation, intelligently figuring out, for example, which videos showing knives are violent versus instructional. It can also be used for media analytics and real-time feedback, he says, and to automatically generate highlight reels from videos.
\nA little over a year after its founding (March 2021), Twelve Labs has paying customers \u2014 Lee wouldn’t reveal how many exactly \u2014 and a multiyear contract with Oracle to train AI models using Oracle’s cloud infrastructure. Looking ahead, the startup plans to invest in building out its tech and expanding its team. (Lee declined to reveal the current size of Twelve Labs’ workforce, but LinkedIn data shows it’s roughly 18 people.)
\n\u201cFor most companies, despite the huge value that can be attained through large models, it really does not make sense for them to train, operate and maintain these models themselves. By leveraging a Twelve Labs platform, any organization can leverage powerful video understanding capabilities with just a few intuitive API calls,\u201d Lee said. \u201cThe future direction of AI innovation is heading straight towards multimodal video understanding, and Twelve Labs is well positioned to push the boundaries even further in 2023.\u201d
\nTwelve Labs lands $12M for AI that understands the context of videos by Kyle Wiggers originally published on TechCrunch
\nTwelve Labs lands $12M for AI that understands the context of videos<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

To Jae Lee, a data scientist by training, it never made sense that video \u2014 which has become an enormous part of our lives, what with the rise of platforms like TikTok, Vimeo and YouTube \u2014 was difficult to search across due to the technical barriers posed by context understanding. Searching the titles, descriptions and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[5871,5152,5154,10193,10231,5116,10450,10451],"tags":[1380,675,538,8773],"class_list":["post-101670","post","type-post","status-publish","format-standard","hentry","category-ai","category-artificial-intelligence","category-machine-learning","category-media-entertainment","category-robotics-ai","category-startups","category-twelve-labs","category-video-content","tag-ai","tag-api","tag-google-search","tag-index-ventures"],"yoast_head":"\nTwelve Labs lands $12M for AI that understands the context of videos - \u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439<\/title>\n<meta name=\"description\" content=\"To Jae Lee, a data scientist by training, it never made sense that video \u2014 which has become an enormous part of our lives, what with the rise of platforms\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/\" \/>\n<meta property=\"og:locale\" content=\"ru_RU\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Twelve Labs lands $12M for AI that understands the context of videos - \u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439\" \/>\n<meta property=\"og:description\" content=\"To Jae Lee, a data scientist by training, it never made sense that video \u2014 which has become an enormous part of our lives, what with the rise of platforms\" \/>\n<meta property=\"og:url\" content=\"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/\" \/>\n<meta property=\"og:site_name\" content=\"\u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439\" \/>\n<meta property=\"article:published_time\" content=\"2022-12-06T05:03:51+00:00\" \/>\n<meta name=\"author\" content=\"Mobile news chief editor\" \/>\n<meta name=\"twitter:label1\" content=\"\u041d\u0430\u043f\u0438\u0441\u0430\u043d\u043e \u0430\u0432\u0442\u043e\u0440\u043e\u043c\" \/>\n\t<meta name=\"twitter:data1\" content=\"Mobile news chief editor\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u041f\u0440\u0438\u043c\u0435\u0440\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f \u0434\u043b\u044f \u0447\u0442\u0435\u043d\u0438\u044f\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 \u043c\u0438\u043d\u0443\u0442\u044b\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\\\/\"},\"author\":{\"name\":\"Mobile news chief editor\",\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/#\\\/schema\\\/person\\\/659775c2c0130cf3d639e6e8c0aede94\"},\"headline\":\"Twelve Labs lands $12M for AI that understands the context of videos\",\"datePublished\":\"2022-12-06T05:03:51+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\\\/\"},\"wordCount\":862,\"keywords\":[\"AI\",\"API\",\"Google Search\",\"Index Ventures\"],\"articleSection\":[\"AI\",\"Artificial Intelligence\",\"machine learning\",\"Media & Entertainment\",\"Robotics & AI\",\"Startups\",\"twelve labs\",\"video content\"],\"inLanguage\":\"ru-RU\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\\\/\",\"url\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\\\/\",\"name\":\"Twelve Labs lands $12M for AI that understands the context of videos - \u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/#website\"},\"datePublished\":\"2022-12-06T05:03:51+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/#\\\/schema\\\/person\\\/659775c2c0130cf3d639e6e8c0aede94\"},\"description\":\"To Jae Lee, a data scientist by training, it never made sense that video \u2014 which has become an enormous part of our lives, what with the rise of platforms\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\\\/#breadcrumb\"},\"inLanguage\":\"ru-RU\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/phonezone.ru\\\/news\\\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u0413\u043b\u0430\u0432\u043d\u043e\u0435 \u043c\u0435\u043d\u044e\",\"item\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Twelve Labs lands $12M for AI that understands the context of videos\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/#website\",\"url\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/\",\"name\":\"\u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439\",\"description\":\"\u041d\u043e\u0432\u043e\u0441\u0442\u043d\u0430\u044f \u043b\u0435\u043d\u0442\u0430: \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0435 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0438\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ru-RU\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/#\\\/schema\\\/person\\\/659775c2c0130cf3d639e6e8c0aede94\",\"name\":\"Mobile news chief editor\",\"url\":\"https:\\\/\\\/phonezone.ru\\\/news\\\/author\\\/admin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Twelve Labs lands $12M for AI that understands the context of videos - \u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439","description":"To Jae Lee, a data scientist by training, it never made sense that video \u2014 which has become an enormous part of our lives, what with the rise of platforms","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/","og_locale":"ru_RU","og_type":"article","og_title":"Twelve Labs lands $12M for AI that understands the context of videos - \u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439","og_description":"To Jae Lee, a data scientist by training, it never made sense that video \u2014 which has become an enormous part of our lives, what with the rise of platforms","og_url":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/","og_site_name":"\u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439","article_published_time":"2022-12-06T05:03:51+00:00","author":"Mobile news chief editor","twitter_misc":{"\u041d\u0430\u043f\u0438\u0441\u0430\u043d\u043e \u0430\u0432\u0442\u043e\u0440\u043e\u043c":"Mobile news chief editor","\u041f\u0440\u0438\u043c\u0435\u0440\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f \u0434\u043b\u044f \u0447\u0442\u0435\u043d\u0438\u044f":"4 \u043c\u0438\u043d\u0443\u0442\u044b"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/#article","isPartOf":{"@id":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/"},"author":{"name":"Mobile news chief editor","@id":"https:\/\/phonezone.ru\/news\/#\/schema\/person\/659775c2c0130cf3d639e6e8c0aede94"},"headline":"Twelve Labs lands $12M for AI that understands the context of videos","datePublished":"2022-12-06T05:03:51+00:00","mainEntityOfPage":{"@id":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/"},"wordCount":862,"keywords":["AI","API","Google Search","Index Ventures"],"articleSection":["AI","Artificial Intelligence","machine learning","Media & Entertainment","Robotics & AI","Startups","twelve labs","video content"],"inLanguage":"ru-RU"},{"@type":"WebPage","@id":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/","url":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/","name":"Twelve Labs lands $12M for AI that understands the context of videos - \u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439","isPartOf":{"@id":"https:\/\/phonezone.ru\/news\/#website"},"datePublished":"2022-12-06T05:03:51+00:00","author":{"@id":"https:\/\/phonezone.ru\/news\/#\/schema\/person\/659775c2c0130cf3d639e6e8c0aede94"},"description":"To Jae Lee, a data scientist by training, it never made sense that video \u2014 which has become an enormous part of our lives, what with the rise of platforms","breadcrumb":{"@id":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/#breadcrumb"},"inLanguage":"ru-RU","potentialAction":[{"@type":"ReadAction","target":["https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/phonezone.ru\/news\/twelve-labs-lands-12m-for-ai-that-understands-the-context-of-videos\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u0413\u043b\u0430\u0432\u043d\u043e\u0435 \u043c\u0435\u043d\u044e","item":"https:\/\/phonezone.ru\/news\/"},{"@type":"ListItem","position":2,"name":"Twelve Labs lands $12M for AI that understands the context of videos"}]},{"@type":"WebSite","@id":"https:\/\/phonezone.ru\/news\/#website","url":"https:\/\/phonezone.ru\/news\/","name":"\u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0445 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0439","description":"\u041d\u043e\u0432\u043e\u0441\u0442\u043d\u0430\u044f \u043b\u0435\u043d\u0442\u0430: \u043c\u043e\u0431\u0438\u043b\u044c\u043d\u044b\u0435 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0438","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/phonezone.ru\/news\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ru-RU"},{"@type":"Person","@id":"https:\/\/phonezone.ru\/news\/#\/schema\/person\/659775c2c0130cf3d639e6e8c0aede94","name":"Mobile news chief editor","url":"https:\/\/phonezone.ru\/news\/author\/admin\/"}]}},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/posts\/101670","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/comments?post=101670"}],"version-history":[{"count":0,"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/posts\/101670\/revisions"}],"wp:attachment":[{"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/media?parent=101670"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/categories?post=101670"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/phonezone.ru\/news\/wp-json\/wp\/v2\/tags?post=101670"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}