PodcastsThe Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai

20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai
The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The PitchWed Aug 30 2023
Entry into the World of AI and NLP:
- Noam Shazeer joined Google and worked on a spelling corrector project.
- Google had a third-party software for spelling correction, but it was not effective for web search.
- The number one issue with Google's search was spelling correction, so Noam decided to work on improving it.
Model Size or Data Size:
- The size of the model is more important than the size of the data.
- Computation plays a crucial role in training models, so training a bigger model for longer periods is essential.
- Companies need to invest in better hardware and spend more time training models to make them smarter.
- The value of proprietary data and open-source data both contribute to improving AI models.
The Biggest Barriers:
- The biggest barrier for Character.ai is building a horizontal product with multiple use cases.
- Model training is challenging and requires significant investment. Noam mentioned that they spent $2 million worth of compute cycles to train an early model.
- Releasing a product with many different use cases poses difficulties in terms of development and implementation.
- The race for AI dominance involves both startups and incumbents, but where the value accrues remains uncertain.
AI's Role on Society:
- Noam believes that AI can create greater human connections rather than worsen them.
- He is not concerned about the speed of adoption of AI tools because he sees their positive impact on society.
- Noam suggests that there are aspects of AI's impact on society that people may not see yet.
Character.ai's Mission and Vision:
- Character.ai aims to provide useful tools to individuals while respecting their agency and allowing them to decide how they want to use the technology.
- They believe in launching something general and letting users find innovative use cases for it.
Challenges Faced by Character.ai as a Product Paradigm:
- Balancing versatility and usability is a challenge when developing products.
- Character.ai wants to build something that is both versatile and usable, which can be a difficult balance to achieve.
The Future of AI:
- Noam believes that AI technology will continue to improve rapidly in the next few years.
- He compares the current state of AI to the early days of electricity or computers, where the full potential has not yet been realized.
Character.ai's Position as a Standalone Company:
- Noam believes that startups have an advantage over large companies in terms of speed and agility.
- Being a standalone company allows Character.ai to focus on making their products great and pushing AI technology forward.
Open vs. Closed Approach in the AI Community:
- Noam sees value in both open and closed approaches in the AI community.
- Openness allows for more research to be published, while closed systems can be more efficient for serving products at scale.
Misconceptions about AI:
- Noam wishes people understood that there are still many undiscovered applications for AI technology.
- The best applications have not even been invented yet, and there is much room for growth and innovation.
Noam's Transition to CEO:
- Noam enjoys being a CEO because it allows him to make decisions that push the technology forward and have a meaningful impact.
- He values usefulness over personal enjoyment when it comes to his role as CEO.
Advice for Parents:
- Noam emphasizes understanding one's responsibilities and focusing on what truly matters.
- He mentions that parenthood has taught him humility and gratitude for having the opportunity to do something important and meaningful.
The Future of Character.ai:
- Noam cannot predict where Character.ai will be 10 years from now due to rapid technological advancements.
- The company aims to remain agile and adapt to future changes.