Tue Jun 11 2024

Mike Conover's Background and Transition to Brightwave:

• Mike Conover transitioned from leading the open source large language models team at Databricks to founding Brightwave, an AI research assistant for investment professionals.
• He raised a $6 million seed round led by Alessio and Decibel, sharing insights from serving customers with over $120 billion of assets under management in the last 4 months since launch.
• Mike's background includes working on global economic structures at LinkedIn, DARPA-funded research on propaganda campaigns using Twitter data, and financial machine learning at Workday.

Evolution of Large Language Models (LLMs) Context Sizes:

• The context size of LLMs has evolved rapidly, with Dolly having a context size of 1,024 tokens just 14 months ago.
• Commercial LLMs struggle to generate very long answers due to limitations in fitting responses within specific token ranges as context increases.
• To address this issue, breaking down tasks into multiple subtasks and merging them back together is crucial. This approach helps maintain details while summarizing content effectively.

Challenges in Fine-Tuning vs. Retrieval Augmented Generation (RAG):

• The discussion highlighted the distinction between fine-tuning models like stem cells that differentiate into specific behaviors versus RAG focusing on grounded reasoning through known facts during inference.
• Fine-tuning is used when developing conviction about repeatable behaviors worth differentiating into specific subsystems rather than imbuing net new information into systems.

Privacy Concerns and Data Handling in Financial Services:

• Privacy and confidentiality are critical considerations in financial services due to sensitive data handling requirements.
• Brandon Katara's experience operating a federally regulated derivatives exchange adds confidence to Brightwave's ability to manage sensitive data securely throughout its lifecycle.

Building Systems for Temporal Data Management:

• Managing temporality in data is essential due to changing quarterly data overriding previous information.
• Retrieval systems must be aware of temporal intent in user queries to provide relevant up-to-date information without relying solely on semantic search results.

Role of Classical Machine Learning Alongside LLMs:

• Traditional machine learning plays a vital role alongside LLMs by providing statistical guarantees about outputs that LLMs cannot offer.
• Choosing the right tool for each task ensures fault tolerance and complements the capabilities of LLMs effectively.

Excel Spreadsheets vs. Partner-in-Thought Model Approach:

• Brightwave opts for a partner-in-thought model over Excel spreadsheets to foster dialogue and personalization in financial modeling instead of static formulas.
• Users prefer engaging with financial models personally, allowing them to develop their views rather than passively consuming automated outputs like those generated by Excel spreadsheets.

AI Impact on Financial Modeling:

• Tabular data in financial models is crucial for decision-making, with Excel spreadsheets being considered inadequate for advanced technologies by 2024.
• Language models have shown progress in enhancing understanding over time. However, challenges persist in generating detailed responses as context increases.
• Models like Dolly have demonstrated advancements in handling larger contexts but struggle with effectively producing lengthy answers.
• Breaking down tasks into subtasks and then merging them back together has the potential to improve model performance by providing more comprehensive responses.

Future of AI Hedge Funds:

• Current hedge fund operations are likened to potential AI-driven trading desks, highlighting similarities between systematics desks at hedge funds and machine learning teams.
• The feasibility of AI fully managing a trading desk remains uncertain, underscoring the ongoing value placed on human synthesis and decision-making abilities within asset allocation processes.

Open-source LLMs Evaluation:

• Economic incentives for companies to train their own foundation models are diminishing due to converging behaviors resulting from overlapping training data.
• The next frontier for innovation lies in developing new training data corpuses that elicit specific behaviors from models, offering opportunities for differentiation and enhanced model performance through tailored training datasets.