Image of podcast

Generative AI in Video and the Future of Storytelling (with Runway CEO Cristobal Valenzuela)

ACQ2 by Acquired

Tue Aug 29 2023



Background and Introduction:

  • Cristobal Valenzuela is the co-founder and CEO of RunwayML, a company focused on generative models for film and video creation.
  • Runway has been working on synthetic media or generative models for several years, with a focus on applying these ideas to fields such as filmmaking, designing, and art making.

How Generative Models Work:

  • Generative models understand patterns and features within datasets using probabilistic modeling techniques.
  • In the case of video generation, these models can predict what frames will come after an initial frame by learning temporal consistency between frames.
  • Training large models on large datasets helps gather patterns and insights about frames, allowing for the generation of new frames based on existing images or other inputs.

Different Use Cases:

  • The flexibility of generative models allows them to be used in various creative endeavors such as future films, short films, series, internal HR videos, small creations made by individuals, etc.
  • Runway's goal is to enable new types of creative endeavors to flourish by providing a flexible tool that can be used by both professionals in the film industry and individuals who may have never considered themselves filmmakers or creatives before.

Customization and Control:

  • Fine-tuning models plays a crucial role in customizing their output based on specific styles or control mechanisms desired by users.
  • By fine-tuning models with particular art direction or style references, users can create customized systems tailored to their specific needs.

Potential Impact on Storytelling:

  • The ability to generate pixels in real-time during consumption rather than through a rendering process opens up new possibilities for personalized and variable storytelling experiences.
  • This could lead to more interactive or immersive narratives that resemble video games, allowing viewers to have an active role in generating or shaping the story.

Business Model Considerations:

  • Monetization and value capture can depend on the type of model and output being generated.
  • Customization and fine-tuning may play a role in monetizing models for large customers and enterprises.
  • New business models based on consumption rather than creation may emerge as the technology evolves, potentially opening up opportunities beyond traditional SaaS models.

Challenges in Product Building:

  • Building user-friendly interfaces, controls, and workflows is one of the most challenging aspects of product development in the generative AI space.
  • Creating metaphors and primitives for interacting with these models requires extensive research and exploration to ensure ease of use and control for users.

Fundraising:

  • Runway recently raised $141 million in a funding round led by Google, with participation from NVIDIA and other VCs.
  • The fact that the company was able to raise such a significant amount of capital demonstrates that value comes not just from capital itself but also from providing infrastructure and more than just financial support.

AI disruption in VC:

  • Venture capitalists initially feared AI would disrupt their jobs in making investment decisions, but it turns out that the real disruption caused by AI in VC is whether or not you have access to GPU clusters.
  • The platform teams that VCs actually wanted all along turned out to be GPUs, which highlights the importance of having the right technological resources.

Recommended reading on AI:

  • Karpathy's blog post from 2015 titled "The Unreasonable Effectiveness of Recurrent Neural Networks" opened up possibilities and provided historical insights into what could be achieved with AI.
  • Colin McDonald's piece on early computer vision models offers interesting ideas about where things were just a few years ago.
  • Twitter is also mentioned as a valuable source of material related to AI due to its constantly evolving nature.

Background and journey at NYU Tisch:

  • Cristobal Valenzuela has a diverse background spanning economics, business consulting, art, software engineering, and freelancing.
  • NYU Tisch's Interactive Telecommunications Program (ITP) offers a unique intersection of art and technology.

Runway Studios:

  • Runway Studios is the entertainment division within Runway that serves as a creative partner for filmmakers, musicians, and artists who want to leverage AI models to push the boundaries of storytelling.
  • They have produced short films and music videos, offer grants for content creation, and aim to redefine storytelling from both a technological and creative perspective.

Contacting Runway:

  • Those interested in working with Runway or exploring career opportunities can visit runway.com/careers.
  • Cristobal Valenzuela can be reached on Twitter. The company has its office in New York City's Tribeca neighborhood.