AI Innovations from Gemini and DeepSeek to Break Your Brain | [Sidecar Sync Episode 76]

Summary:

In this episode of Sidecar Sync, Amith and Mallory dive deep into two major AI model releases—Google’s Gemini 2.5 Pro and DeepSeek’s V3—and explore how they’re reshaping the landscape of artificial intelligence. They discuss the technical breakthroughs, from million-token context windows to open-source innovations, and what it all means for association leaders. The conversation covers practical use cases, like AI-generated comic strips and strategic content applications, and zooms out to reflect on the rapidly decreasing costs of AI paired with rising performance. This is a jam-packed episode with strategic insights for the forward-thinking association exec.

Timestamps:

🔎 Check out Sidecar's AI Learning Hub and get your Association AI Professional (AAiP) certification:

https://learn.sidecar.ai/

💡Attend the Blue Cypress Innovation Hub in Chicago: 

https://bluecypress.io/innovation-hub-chicago

📝 Join the AI Mastermind group!

https://sidecar.ai/association-ai-mastermind

📕 Download ‘Ascend 2nd Edition: Unlocking the Power of AI for Associations’ for FREE

https://sidecar.ai/ai

📅 Find out more digitalNow 2025 and register now:

https://digitalnow.sidecar.ai/

🛠 AI Tools and Resources Mentioned in This Episode:
Claude 3.7 ➡ https://claude.ai

GPT-4o ➡ https://openai.com

Gemini 2.5 Pro ➡ https://deepmind.google/technologies/gemini

DeepSeek-V3 ➡ https://huggingface.co/deepseek-ai

Midjourney ➡ https://www.midjourney.com

HeyGen ➡ https://www.heygen.com

Amith’s Comic Strip ➡ https://shorturl.at/2Khn5

👍 Please Like & Subscribe!
https://twitter.com/sidecarglobal
https://www.youtube.com/@SidecarSync
https://sidecarglobal.com

Follow Sidecar on LinkedIn

⚙️ Other Resources from Sidecar: 

• Sidecar Blog
• Sidecar Community
• digitalNow Conference
• Upcoming Webinars and Events
• Association AI Mastermind Group

More about Your Hosts:

Amith Nagarajan is the Chairman of Blue Cypress 🔗 https://BlueCypress.io, a family of purpose-driven companies and proud practitioners of Conscious Capitalism. The Blue Cypress companies focus on helping associations, non-profits, and other purpose-driven organizations achieve long-term success. Amith is also an active early-stage investor in B2B SaaS companies. He’s had the good fortune of nearly three decades of success as an entrepreneur and enjoys helping others in their journey.

📣 Follow Amith on LinkedIn:
https://linkedin.com/amithnagarajan

Mallory Mejias is passionate about creating opportunities for association professionals to learn, grow, and better serve their members using artificial intelligence. She enjoys blending creativity and innovation to produce fresh, meaningful content for the association space.

📣 Follow Mallory on Linkedin:
https://linkedin.com/mallorymejias

Read the Transcript

[00:00:00] Amith: You shouldn't think of ai, video generation, or ai, whatever, as a scarce resource that you can only use it in a tiny number of areas. You should be thinking that by the end of this year, you'll probably be able to do it across everything. Welcome to Sidecar Sync, your Weekly Dose of Innovation. If you're looking for the latest news, insights, and developments in the association world, especially those driven by artificial intelligence, you're in the right place.

[00:00:27] We cut through the noise to bring you the most relevant updates with a keen focus on how AI and other emerging technologies are shaping the future. No fluff, just facts and informed discussions. I'm Amith Nagarajan, Chairman of Blue Cypress. And I'm your host. Greetings and welcome to the Sidecar Sync Your Home for Content at the intersection of associations and ai.

[00:00:50] My name is Amith Nagarajan.

[00:00:52] Mallory: And my name is Mallory Mejias,

[00:00:54] Amith: and we are your hosts. Today, we are going to cover some crazy and awesome and exciting things that are happening at the forefront of artificial intelligence, and we're gonna tell you how they might apply to your world as an association leader. Before we do that though, let's take a moment to hear a quick word from our sponsor.

[00:01:13] Mallory: If you're listening to this podcast right now, you're already thinking differently about AI than many of your peers. Don't you wish there was a way to showcase your commitment to innovation and learning The association AI Professional or A A IP certification is exactly that. The A A IP certification is awarded to those who have achieved outstanding theoretical and practical AI knowledge as it pertains to associations.

[00:01:39] Earning your A A IP certification proves that you're at the forefront of AI in your organization and in the greater association space, giving you a competitive edge in an increasingly AI driven job market. Join the growing group of professionals who've earned their A A IP certification and secure your professional future by heading to learn.sidecar.ai.

[00:02:02] Amit, how are you doing today?

[00:02:04] Amith: I'm doing great. I just got back from DC I think I may have picked up a little bit of a cold or something on the way back. Yeah. But, uh, but here I am. I'm back in New Orleans and doing well. How about yourself?

[00:02:14] Mallory: I'm doing pretty well myself. Nothing like a little dose of sidecar sink to make you feel better.

[00:02:19] Right?

[00:02:20] Amith: That's right. It's like an adrenaline shot. It's awesome.

[00:02:22] Mallory: Exactly. I have been. Itching to find out how the innovation hub went. I was not able to attend, and I haven't really talked to you yet, Amit, about it. So this is the first time I'm hearing, but how was it? How did it go? How was your session?

[00:02:36] Amith: It was amazing.

[00:02:37] So this is the third annual DC innovation hub. We have the Chicago one coming up in two weeks, in, uh, in, in Chicago, April 8th. And that's also going to be an amazing event. Uh, it's at the American College of Surgeons office downtown. And the DC innovation hub was at the American Geophysical Union just north of DuPont Circle in dc.

[00:02:57] Beautiful location, amazing conference, wonderful host. We thank our friends at a GU for that. And we had, um. Great turnout. We had about 70 people show up. This is our small community event we do in DC and Chicago each year. Our main flagship event, as many of you know, who are listening, is digital now, uh, which will be in Chicago, uh, this fall.

[00:03:19] But back to the innovation hub in dc It was really a cool moment in time, I think in a way, Mallory, in that we. Really are seeing people build. They're past the contemplation stage. Many of the people that were in the room were there to listen and learn, but they were also there to share. That's, that's the idea behind the innovation hubs.

[00:03:37] We started those as informal community gatherings to certainly share some content and things that various folks across our family of companies are doing. But more than anything to kind of take a feel of the community and say, Hey, like. What's going on. And so people are talking about deploying ai, uh, in a lot of interesting ways.

[00:03:55] So it was super fun. I learned a ton, met some great people. I hadn't had a chance to meet in person before all around, really well worth it. And I hope those of you that are listening that are even somewhat close to Chicago, consider joining us on April 8th. We still have a little bit of room left.

[00:04:09] Mallory: Mm-hmm.

[00:04:10] Were there any key takeaways or any like challenges, any patterns that you noticed emerge? Of all the associations there said, Ugh, we're really struggling with X. Anything like that?

[00:04:22] Amith: I think more than anything people seem ready. So even last year at this time, the way I felt was that people were still nervous.

[00:04:31] They were contemplating, they were learning, but they were still kinda like, Hmm, I don't know if we should do this. I'm not sure if AI is ready to support the critical work of our association at scale. A lot of people a year ago were already. Doing personal experiments and actually in some cases, significant work with tools like Claude and Chat GPT, but to deploy it as an organization, right?

[00:04:52] Mm-hmm. To go out into the world and say, this is our association's AI for knowledge, or this is an as. This is our association's tool for search or personalization. Fewer are doing that. And this time around, what I felt was that people were saying, yeah, we are doing this. It wasn't a question of if, and I think that's exciting because, you know, AI is not gonna wait for any of us, whether we're, you know, do-gooders in the not-for-profit sector or somebody else.

[00:05:17] Like AI is not hanging out and just saying, Hey, we'll wait for you however, however long you want. So I was excited to see people taking action because that's really the learning loop, right? We talk at sidecar. All the time about the AI learning journey. It's not an event. It's a continuous and forever process, and that's true in any domain.

[00:05:34] It's definitely true in ai. And when you go and do the thing yourself, that's when you really learn and you build organizational reps. That leads to organizational strength, obviously leads to culture change. It's cool. So I was really pumped up about that. Um, we heard about people talking about a lot of different kinds of AI as well.

[00:05:50] So people, some people do an interesting analytics stuff. Some people are doing personalization at scale, some people are doing, you know, knowledge assistant work. So it was a lot of fun.

[00:05:59] Mallory: Awesome. I feel like you and I and through the podcast, through Sidecar, getting to witness this journey from its inception all the way to now, all the way into the future, I'm thinking of the AI Mastermind group that sidecar runs for association, C-Suite leaders, and thinking how in its early stages, the AI Mastermind group.

[00:06:17] We had to lead most of the sessions because there just weren't. Any use cases out there, it was more abstract. We were talking about concepts, getting your data ready, ready, strategy. And now in our AI mastermind iteration, we're having more and more participants partic uh, present because they're actually like rolling out these projects themselves.

[00:06:35] So it's been really neat to see, and 75 episodes, how far we've come.

[00:06:40] Amith: Yeah, I think we had to, with the mastermind, we had to get the party started, but now it's raising. Yeah, so it's, it's pretty cool. And, uh, that mastermind group is awesome. So, um, a good friend of ours, Mary Byers, she and I are the leads of the Mastermind.

[00:06:53] We co, we host this, uh, virtual monthly 90 minute. Session, um, with a, a small group of engaged, uh, leaders from a variety of different associations. And, uh, if you're interested in going deep on AI once a month, uh, consider joining that. We have information on our website there as well. Uh, but yeah, you're right.

[00:07:12] I mean this, this, this journey that we're on, we are witnesses to it, as you pointed out, Mallory. And we hope to be, you know, uh, in various ways, obviously educators and sources of inspiration, but really a, a conduit through which the community. Can share their experiences with ai. And so, uh, to build on the point I made with respect to Innovation Hub, what we'd love to do is hear more from our listeners, from the folks who watch us on YouTube, give us feedback on things you're doing.

[00:07:39] Uh, in some cases we may be able to bring you on the PO as a guest or feature something that you're doing in an article on the sidecar. Blog in our newsletter, et cetera. We'd love to hear from you. That's the most powerful thing. You know, we can talk about our perspective as the leaders of Sidecar and Blue Cyprus.

[00:07:55] At the end of the day, what matters is what you guys are doing.

[00:07:59] Mallory: In today's episode, we are talking about AI models. Is that a surprise? We're talking about the release of Gemini 2.5 Pro, and then we'll be talking about deep seeks, latest upgraded model. So starting off with Gemini 2.5 Pro, it's Google's latest and most advanced AI model introduced this week to the public.

[00:08:20] Gemini 2.5 Pro is part of a greater trend of thinking models or reasoning models, which are advanced AI systems designed to mimic certain aspects of human thought processes, particularly in problem solving and logical reasoning. These models use complex algorithms and techniques to analyze information, draw conclusions, and make decisions based on that analysis.

[00:08:42] We've chatted at length about Claude 3.7 open AI's oh one, and now Gemini at 2.5 Pro is joining the ranks. The model uses techniques like reinforcement learning and chain of thought prompting to simulate reasoning. This process involves the model thinking through its responses by verifying facts and logically deducing answers before providing them.

[00:09:04] With a large context window of 1 million tokens, Gemini 2.5 Pro can process extensive amounts of data and not just text. It can also process audio, images, videos, and large data sets like entire code repositories. The models AgTech coding abilities allow it to create complex applications like fully functional video games from a single prompt.

[00:09:26] I did a little brief test with 2.5 Pro in the Google AI studio before we recorded this pod, and so far I like it. It breaks down exactly what it's thinking, which a lot of the models that I mentioned do as well. You all know that we produce blogs from our Sidecar Sync transcripts. If you've listened to the podcast before, we've talked about this several times.

[00:09:47] Typically my go-to model for that would be Claude, but I decided to run a little experiment with uh, 2.5 Pro. And I was quite impressed, mostly because I, I like to start that process by asking, identify three topics from this podcast that we could write a blog about instead of just asking it to generate, you know, an entire blog at once.

[00:10:07] And it gave me some really compelling topic ideas. And then it also provided support from the podcast. So add minute 10. Amit mentioned this, this is why I included it in this topic. So I thought that was interesting. That's something that Claude does not typically do when I use it. Um, am What are your.

[00:10:24] Initial thoughts with with 2.5 Pro.

[00:10:27] Amith: I'm pretty impressed with it. I haven't personally sent it a single prompt, so I'll disclose that. Mm-hmm. I intend to over the next couple days, but I have seen several videos of demos of Gemini two five Pro. Uh, you pointed out the long context windows, so let's talk about that just for a minute.

[00:10:43] And it's a quick refresh for those who have heard this before. And for those that may not be super familiar, um. When you're talking to an ai, the context window really refers to the amount of short-term memory it has. So when you send a prompt in and kind of the history of your conversation in with that model, not, not all conversations by the way, but that specific conversation you're on in chat, GPT in Claude or in the Google Studio.

[00:11:08] Um. The aggregation of all of the back and forth you have with the model, that all has to fit into what's called the context window. And there's a variety of techniques for dealing with, um, really long conversations. But the idea is, is that the bigger the context window in theory, the more powerful the model could be because it has more of this short-term memory.

[00:11:29] Um, so. When we talk about a million tokens, a token being approximately equal to a word for our purposes, um, that means, that's a lot of words. You know, that's, that's something on the order of magnitude of 15 to 20 business books. It's a lot of content. Whereas the other models that are out there that are similar in intelligence, like Cloud 3.7 and GPT-4 oh, those tend to be limited to 128,000 tokens.

[00:11:52] So about an eighth of the total capability of, uh, of these models. So. Sorry, of this particular model, Gemini two five Pro. Now, Google has been a leader in long context models for some time since the first Gemini release. Actually, they had very large context windows. Uh, some of the ways to think about this for associations is that if we have really complex tasks, we wanna take on where we want to feed in.

[00:12:15] Many different pieces of content, let's say from our journals or transcripts from conferences, and we wanna be able to look across a lot of content at the same time, Gemini is a tool that stands on its own at the moment because these other tools can only are limited essentially to this fair, fairly small context window on the one hand, but remember.

[00:12:33] Original chat. CPT, if you recall, had 4K of context. So 4,000 tokens. It was very, very limited. Um, so in any event, um, the point I would make is, um, that by itself is cool. Um, and really more than anything that's not new, that's just a feature of Gemini that seems to be a, a key differentiator for, for. Big, complex pieces of content, uh, but really the intelligence of the model is pretty amazing.

[00:12:57] So one of the people I follow on YouTube, this guy named Matt Berman, who I, if you like, slightly more technical content, he's a great YouTuber to follow. Uh, I watch quite a few of his videos and he breaks down fairly complex topics in a, in a really nice way, in my opinion. Anyway, so, um, he had this video showcasing using Gemini two five Pro for coding.

[00:13:18] And I tend to look at those examples fairly quickly when a new model comes out because coding is both, it's, it has to represent both the ability to do like fairly complex reasoning, but also to understand pretty complex prompts. Especially with these days, you know, people are putting in requests to coding, uh, tools like this to do.

[00:13:36] Very complex things like the two examples in his videos. One was a Rubik's Cube simulator, which was essentially a three-dimensional Rubik's cube simulator of any number of dimensions. So it could be three by three, six by six, a hundred by a hundred. And the, the AI was asked to build the codes that you could visually represent this, you could spin it around, you could see it from any angle.

[00:13:59] Um, you could zoom in and out, you could pan tilt, et cetera. And it did that. And then on top of that, um. It, um, the code, it was requested of the AI to make it so that the AI itself could solve the Rubik's cube, so it could randomize the cube state and then it could solve it, and you can watch it visually solving the Rubik's cube, which is pretty cool.

[00:14:19] Um, what's impressive about this is that in this video it was as it was a single shot, right? So it was just a prompt, and then immediately had a working piece of code in a browser in a single HTML page that did this. Uh, that's a non-trivial. Bit of software development to write that, right? Even a really good developer would take quite a bit of time to build something of that order of magnitude.

[00:14:40] The other example was also similarly, three dimensional visual kind of thing in the browser to build a Lego simulator to be able to snap Lego bricks together of any sizes and shapes and colors. And what he demonstrated there was was pretty cool. So I found that. Particular example, really compelling? Uh, both because it was clearly showing the ability to do fairly complex reasoning with that level of coding.

[00:15:05] This is not a trivial coding ex exercise, like, you know, some simpler games that people have asked, like build a snake game in Python, which is, I wouldn't say that's trivial, but it's fairly simple. Comparatively speaking, this is an order of magnitude more complex. So that was impressive. I think your example was great, and when we, when we're talking about this.

[00:15:24] Just continuous evolution of these models. The thing we always have to point out is there's so many options, right? Yeah. So this is now Google getting into the game in a way that I think really puts them more on the radar for a lot of people. We talk about open ai, we talk about Claude, we talk about the open source models, but you know.

[00:15:42] Google's just been kind of behind, at least in the public perception. Uh, and in terms of usage, certainly they've been a, a distant, maybe not even third place is what I was gonna say, maybe fourth or fifth. So I think this is gonna put them back on the map for a lot of people to really consider and think deeply about.

[00:15:58] Mallory: And that was one of my follow up questions too, is, at least from my perspective, it sounds like from yours too, Google seems to be late to the party oftentimes when it comes to ai, but when they arrive to the party, right, they're well dressed. People wanna hang out with them, like this model's really impressive, but it seems late.

[00:16:16] So is that kind of Google's mo you would say? Like, do they take more time to bake things because they want to put out something really quality?

[00:16:25] Amith: I definitely think that is an element of it that I think in fairness to them, I think that's a part of what they need to do because of their scale and because of their brand, that they want to put things out that are fairly well thought out.

[00:16:37] Uh, the flip side of it is, I think they were just, I. Behind and, you know, they were not behind in terms of fundamental AI research. They've been leaders in that in many ways for years and years. And for those that aren't familiar, Google actually invented the transformer architecture, which is the type of neural network that is powered.

[00:16:55] Uh, all of language models you've been hearing about, including the original chat, GPT, and, and it still powers, uh, the vast majority of language models. Um, with a, back in 2017 they invented that architecture and so, uh, they know a thing or two about ai. These are some really smart folks. To a lot of resources, but, uh, you know, one of the things to maybe consider also is organizationally, um, these guys are the incumbents in search.

[00:17:19] And so when they saw LLM start to scale quickly, they're thinking about their own business model. And so it all of a sudden made it, you know, strategic. Strategically critical thing to be in this game. Um, so I don't know if they're thinking about it more from the perspective of how do we use it within Google search and other Google products, and that's their number one priority versus producing models for the rest of the world to use as compared to open ai.

[00:17:44] Philanthropic whose only purpose is to produce models other people use. So I don't know if it might be that, perhaps. Yeah. Um, you know, and it's also in any organization, no matter how big and how well resourced and how smart the people are, you have to, you have to make priority choices. I'm entirely speculating about this.

[00:18:01] Yeah. But I, you know, I sense that there's elements to that going on.

[00:18:04] Mallory: It seems like Google might need to displace itself, which we've talked about before on the pod. This idea of counter positioning to displace its own search function or at least greatly change it. Uh, and it's kind of already done that with AI overviews, which have gotten better, uh, in my personal opinion.

[00:18:18] But that'll be interesting to watch it play out.

[00:18:21] Amith: Definitely. Yeah. I think that, uh, you know, counter positioning oneself, it's better to be Netflix than Blockbuster. Yeah. Um, but it's a better idea to potentially say, Hey, how can we be, you know, how can we essentially be the, the company that creates the new business model that has superior customer experience ourselves?

[00:18:40] So. I definitely see Google heading in that direction. You know, my bottom line is that, um, there's so much happening in the area of models that, you know, even those of us who spent all of our time thinking about this and talking about it and playing with these models and building software on top of these models, we can't keep up.

[00:18:56] And so just this week, you know, with, aside from the two models. That we're talking about with the new Deep Seq, V three and also, uh, Gemini two Five Pro. Uh, we also had a release from Open AI with the GPT-4 oh, uh, new form of, of image generation. Mm-hmm. Uh, and that is, it's a really stunning capability. So it's, it's a capability if you have an experimented with it.

[00:19:17] Since, uh, you know, since it came out, I really recommend that you get on Chachi PT and ask it to create an image, uh, or take an existing image, drop it in and ask it to modify it. It's, it's pretty stunning and that's gotten quite a bit of attention, but, uh, you know, it's, it's hard to. Keep up with all this stuff.

[00:19:35] The advice that I always give people about that issue of how do you keep up with this is that look for the trend line, right? Look for the pattern and look for how you would like to use these models, not just the specific model. What are the problems you're trying to solve today? If it's a use case you already have, working totally fine for the last 12 months with GT four.

[00:19:56] Maybe you're not so excited by like this new model that's an even better blog when the existing blog was pretty darn good. Of course, you know, those of you listening to this podcast probably are always looking to take the next step and improve and evolve what you're doing. But what I also think is interesting and important, I.

[00:20:12] Is, think about the things you couldn't do with ai. Uh, so my example where I did have hands-on experience in the last 24 hours, Mallory, is I used the new, uh, chat PT four oh um, image generator to create a comic strip, and I threw it on LinkedIn. I. It's not perfect. It's not great, but it just talks about this conundrum that associations are in with respect to a MS replacement.

[00:20:35] Right? Our favorite topic of the pod for a lot of people, other than AI, of course, is a MS replacement. And, you know, we're not trying to, uh, you know, pick on it, but it's, it's a tough, tough thing to do and it's really long and really expensive, and the value creation oftentimes is marginal at best. And so, you know, why do that when you could do a lot with those resources experimenting with ai.

[00:20:57] So that's what I. Just basically said what I just said to open AI's model yesterday and I got a four panel comic strip. Then I said, Hey, give me four more panels that kind of com conclude. Like what happens two years later? If all you do is focus in your a MS and don't do ai, so go check out my LinkedIn if you wanna see the comic strip.

[00:21:15] But um, in the past I've had this idea to create comic strips or infographics or whatever, and none of the image generators could do it. Had I not experimented with this last night, I would still be thinking in that mindset that wouldn't it be cool if I had this creative outlet to create comic strips or infographics with just an AI model.

[00:21:33] But in my brain, I would say, oh, but they do a horrible job with text uhhuh. Oh. But they can't do different styles. Like a comic strip style is very different than the kind of, you know, we're all, we all know what these AI images have started to look like for the last couple years. They're very, very similar.

[00:21:46] Um, but now. My brain is all fired up about like all these new capabilities we have, right? So experimentation is good, but the trend line is it's not just we're excited about open AI as image generator. All the other ones are gonna be like that. Mm-hmm. Within half a minute, right? You're gonna see the same thing from mid journey.

[00:22:03] Probably within days. You're gonna see the same thing from all the other multimodal models. We know anthropic is no slouch with their CLO product. So, um, it's exciting. So the trend line is image generation really can be used for things that. Even like a day ago or two days ago, couldn't be used. And that's what I keep looking for is what are the next use cases that are the next unlocks.

[00:22:24] Mallory: Mm-hmm. Yeah, that's a great point. Your comic strip was good. I highly encourage you all to go to ames, LinkedIn. We can include that in the show notes as well for you to check out. I have never asked Midjourney, which is my preferred image generator to do a comic strip, but I'm pretty sure it wouldn't do a good job with it based on how frequently I use it.

[00:22:42] So I'm gonna use, um, GPT-4 OH'S image generator the next time I publish a blog, which might be today, and see what I can come up with.

[00:22:50] Amith: The text side of it is, is really compelling because the image image generation, even in the past when we've had really stunning images generated by ai mm-hmm. Um, when they have attempted to incorporate text, whether it's a sign not so in the background, it's always been, you know, kind of garbled up.

[00:23:05] And even when you prompted the AI to say, do not include text, yeah. It would still oftentimes include tech. So, um, this, this really does represent a pretty significant leap in image generation and that's useful in, in so many respects. You know, you think of it as well. Um, again, when we're going from something that was once scarce to something that is abundant, you know, comic strip would take a lot of work to put together, right?

[00:23:29] You need a talented illustrators, you need the idea for the comic strip, you need all this stuff. And I certainly wouldn't be attempting to do that. I can, I really can't do much more than a stick figure to save my life, you know? So, yeah. But I have ideas. Mm-hmm. And so I'd love to be able to express those ideas in different ways that are both interesting and hopefully effective in communicating.

[00:23:47] Uh, so I find, I find this really, really exciting. So they take the comic strip and you say, Hey, take an image to video generator and now animate that comic strip and add audio to it. And you know, so that on and on and on this goes right.

[00:24:00] Mallory: Wow. Talking about trendline, I wanna zoom out a little bit. It seems like we've been talking a lot lately about these thinking models that I mentioned, or reasoning models, which seem to be in general pretty large models in terms of parameter size.

[00:24:13] But in the prior few months, we spent a lot of time talking about small models. So I'm kind of wondering how you see small models fitting into these greater thinking and reasoning model conversations, and do you see. Like that chain of thought prompting and reasoning as something that will be needed in small models, or should we just leave that to the, to the big ones?

[00:24:36] Amith: Well, I think that the simulation of the chain of thought reasoning is something that some of the small models are starting to do in some ways, where they're, they're not really doing reasoning in the same sense as the bigger models. So as you mentioned, Mallory oh 1 0 3 from OpenAI Cloud three seven, and now Gemini two five Pro all use.

[00:24:57] This internal thinking process where rather than trying to give you an answer as quickly as possible, they decide that the problem is complex enough that they're gonna break down the problem into steps, and they're gonna solve each of the steps one at a time, typically sequentially, although not always, and then bring back the results from each of the steps, compile a result, and then evaluate the result, and then possibly iterate again and again until, uh, the model determines it's gotten to a good result.

[00:25:24] And, um. This process is, uh, this reasoning process, as it's called, uh, is compute intensive. It takes more time, it takes more compute, uh, and it produces remarkably improved results compared to just quickest possible answers, which is, you know, we've, we've compared it in the past to the idea of system one and system G thinking of the intuitive reactive thinking versus the, the reasoning style thinking that our biological brains are, are doing so well.

[00:25:51] I think coming back to your question about small models, will they incorporate actual reasoning processes? I'd be surprised if they didn't at some level. Okay. Um, but at the same time, part of what we are seeing is people distilling into smaller models, things that ref reflect the intelligence of larger models, and we've seen that.

[00:26:10] From big LLMs to small LMS for some time now for a couple of years where the small models keep getting smarter and the model architectures are getting better. In some ways, they're getting smarter, but it's also about how the data set that's being, that the small model's being trained on reflects back on what the big model.

[00:26:28] Mm-hmm. So for example, LAMA 3.3, the 70 billion parameter kind of midsize model that was released in December is smarter than the LAMA 405 billion parameter model that was released last April. Um, and the way they did that is there were some advancements in the tech, but. What Meta did is they used LAMA 4 0 5 B to generate a bunch of sample data that they then trained the new model on.

[00:26:51] So this distillation process is a way of really taking the essence of the intelligence of the larger model and packing it into a much smaller frame. Mm-hmm. And I think you're gonna see that with reasoning specifically. Um, you know. To foreshadow our next topic with respect to deep seek V three, that's actually a lot of what they did.

[00:27:08] Taking R one's reasoning and dropping some of that intelligence into the, a non reasoning model in V three. But, uh, you're gonna see more and more of this. What I keep coming back to, and this is particularly important for our friends in the world of associations and nonprofits, is that, you know, this is a somewhat technical topic.

[00:27:24] It's important to understand as a. Business leader because we're at this forefront of this emerging technology. We wanna know what's possible, how we can use it to better serve our constituents and on and on. But eventually, and I think that eventually might be in the next 12 to 24 months, you're not gonna be talking too much about is the model a reasoning model or is it a straight inference model?

[00:27:45] I. Most of these models will have a reasoning ability. Mm-hmm. And how much of that reasoning ability they use or don't use will be dependent on what you ask it to do. And that's essentially what Cloud 3.7 does, right? So Cloud three seven knows that it needs to go into thinking mode, just like you or I would if I'm given a really complicated math problem or so something else that takes, you know, time to reason through, I'm gonna do that.

[00:28:07] I'm not just gonna guess at the answer, which is essentially what the, you know, the fast inference mode is what? Uh, uh, language models have been doing up until these reasoning models.

[00:28:16] Mallory: Yeah, I think you're right. Looking back and again, this AI journey that we've been on since the beginning, I remember you and I would talk about multimodal models and models that can understand text and audio and images, and I feel like that's just a commonplace thing now.

[00:28:30] So maybe you're right. 12 to 24 months, we won't even be having this conversation, but. That's a good segue for Topic two of today, which is Deep seeks V three that Amme mentioned. Um, we saw this model upgrade this week as well from a Chinese AI startup called Deep Seek, which we recently covered in a prior episode for its R one model release.

[00:28:54] To give you a quick recap of that, it's our one model just cost $6 million to train. Over 89 times cheaper than open AI's. Rumored $500 million budget for its oh one model. And the release of R one led to a hundred or $1 trillion drop in the US stock market. So lots of waves were made with deep seek and the plot.

[00:29:16] Continues to thicken. So deep Seek has now recently upgraded its V three large language model to deep seek V 3 0 3 2 4. Again, we love these AI model names. This new version is seen as another competitive attack on major AI players like Open ai. And Anthropic Deep seeks V three offers enhanced reasoning and coding capabilities compared to its predecessor.

[00:29:41] The model has 685 billion parameters up from 671 billion in the previous version, and it's important to note that these models are open source. So the model is available under the MIT license on platforms like hugging face. Deep seek V three challenges the dominance of US AI companies by offering competitive performance at lower costs and the models release underscores the growing presence of Chinese AI startups in the global AI scene.

[00:30:09] Perhaps shifting the balance of power in tech development, I feel like Amee with the, with deep seek, the things to note are lower cost competitive capabilities and the fact that it's open source. Do you agree with that?

[00:30:24] Amith: I think so. You know, I, I think credit is definitely due to these folks. They are brilliant.

[00:30:29] The papers they put, they put out there detail, um, you know, with a lot of granularity how they built these models. So they've open sourced not only the, the code and the, and the weight, so the models, so anyone can run it anywhere. But they also have, uh, published a ton of research explaining how they've advanced their technology.

[00:30:47] So this is not a copycat or a clone. They have, um. Fundamentally advance the science of ai and they're contributing back to the global community. So I view this as a very positive thing, and I'm hoping to see this from a lot of other parts of the world, uh, jumping in because especially as you see, the resource constraints seem to decrease over time as people, you know, get really creative when they have limited resources.

[00:31:08] Uh, and that leads to fewer, fewer resources being needed or being perceived as being needed. Right. Um. That in turn leads to more innovation, which leads to more and more growth. And uh, that's, that's exciting. So, so to me, I think that that's one really important thing to recognize. Um, one of the things that these guys have done a really good job of is they've advanced the way mixture of expert models or MOE models work.

[00:31:32] It is a 600 plus billion parameter model, but you can still run it on fairly, um. You know, fairly reasonable hardware pretty quickly because it only has about, I think it's like 32 billion parameters are active on a per token basis. So what that means essentially is it's really behaves like a 32 billion parameter model.

[00:31:51] Um, uh, and that's, and that's not a particularly huge model. It's half the size of llama 3.3 70 that I just mentioned a few moments ago. So, um. That's important because, um, because they've been able to really optimize with this mixture of experts model and do some other things around efficiency. They have a highly performant and really intelligent model.

[00:32:12] Um, I am confident that all the other players that are out there are really paying close attention to everything Deep Seek is doing, incorporating their ideas. So probably today already we've seen open AI and, and. Google incorporate deep seeks ideas and, and on and on, right? And so the proprietary vendors don't really share what they're doing, but I guarantee they're taking advantage of open source and the open source, you know, community keeps on compounding on, on itself.

[00:32:38] So I would, I would point out the open source bit probably is the most important part of it. Okay. Uh, because these, these things are moving so quickly.

[00:32:47] Mallory: And when you say mixture of experts, architecture, that just means parts of the model. The only parts that activate within the model are the parts that are needed for that prompt.

[00:32:55] Amith: Exactly. So, you know, if I say for example, I would like deep seek to write code, maybe there's a portion of that model, 600 plus billion parameters that are focused on coding focused, and only those, you know, 30 billion parameters get activated. Uh, if I ask it to, you know, write a poem, it might be a different section of that.

[00:33:14] You know that a synthetic brain, essentially that gets activated. So it's a very large brain, so to speak, but it only uses portions of it at a time, and it's not running the whole thing. Uh, and you know, most of these AI architectures are MOE or mixture of expert experts, models. What these guys have been doing a good job with is making them more efficient, making them more performance.

[00:33:33] Um, part of what's happening is, is like. Who makes the decision about which section of the model should activate for a given token? And that's something that of course is really important because if you don't route that to the right part of the, of the net, you don't actually get great results. So, um, I would give these guys a lot of props as I've been doing in terms of their scientific advancements.

[00:33:55] My point of view in terms of open AI and, uh, and probably anthropic as well, but more so open AI, is that, you know, you have these companies that have had, I wouldn't say unlimited, but they've had substantial resources and their perception has been that they need those substantial resources to produce these, you know, world class gains.

[00:34:13] And here you have a challenger with far fewer resources, uh, not just money, but also they used, uh, equipment that's considerably less powerful, uh, due to export restrictions. They don't have access to the. Latest, um, chips. At least that's what's been reported. So, you know, that tells you a lot. Again, constraints can be incredibly powerful if you give people a very narrow timeframe to do a job.

[00:34:35] A lot of times you get a better result than if you say, you know, how much time you need, or if you give them longer timeframes. I'm a big fan of setting narrow deadlines for small chunks of work. I don't like saying, Hey, what's your, what's your priority for the next 12 months? You know, I'd, I'd rather know what you're gonna do in the next week.

[00:34:52] Not that I don't think about the next 12 months, but like it's more about how do you, how do you put a constraint that's near term and the smaller the constraints, usually the more creative people get.

[00:35:01] Mallory: Necessity breeds invention, which we've said on the pod before. Amit, I didn't realize this and maybe you had heard it, but there was a bill introduced in the house last month, ban potentially banning federal employees from using a Chinese AI platform, deep seek on their government issued devices.

[00:35:18] So I imagine there might be some organizations that would be dissuaded from using deep seek models under the threat of a potential ban, but on the flip side, the models are open source. So can you explain whether that is. A valid concern or not, or what to keep in mind there?

[00:35:32] Amith: It's totally a valid concern if you're using their website.

[00:35:35] Okay. So if you're going to deep seek.ai, I think is the website, um, that if you go to that website, that model that you're interacting with is hosted in China. Um, which is, you know, it's not an inherently bad thing to have a model hosted in another country. Um, but the point would be that if you're a government employee and you are asking a model something related to.

[00:35:56] You know, whatever it is you're working on, maybe that isn't the best thing to send overseas, right? Maybe that's something we should be running within the United States and, uh, preferably and, you know, a government cloud of some sort that's secure. Um, so I think that's one piece of it, but that should be separated from the idea of the model itself.

[00:36:14] It is an open source, open weights model. Um, it can be reproduced, um, and you know, you can run it. Anywhere you can run it in your own data center, you can run it in a public cloud infrastructure like Azure, A-W-S-G-C-P, Oracle, et cetera. Uh, you can run it. You mentioned hugging face earlier. They provide a variety of services as well.

[00:36:33] There's a lot of places you can run these models and so to ban the model, I think is. Really misinformed because the model itself is just a piece of software that's totally something you can inspect. Um, one of the really nice things about, um, AI models is that if you, you say it's open source, by the way, versus open weights, they sound like similar things.

[00:36:52] Mm-hmm. The model itself, the actual number of lines of code behind these models is remarkably small. It's some, it's in the single digit, thousands typically, or even smaller. And so the models themselves don't have a lot of code. It's all about the weights. So the weights are of. Like harder to understand, right?

[00:37:10] They're just a bunch of numbers, but you, if like someone was worried about like a backdoor existing or like a phone home, you know, phones home and sends your data back, that's not a thing. Like you can, you can verify that the software is not doing that. Uh, you can also contain these models in ways when you inference them in your own hardware so that there's, there's no possibility of them doing that.

[00:37:29] So I think that. The mindset should be that we care deeply about where we inference these models that should be done, thought about from a security perspective, that should be thought from a vendor trust and vendor reliability perspective. Uh, and we, we should also deeply care about which models we use, but.

[00:37:46] Whereas I probably would not sign up to do any workloads for any of our products, um, in China right now for a variety of reasons. One of which is just whether or not that'll continue to be available, but also it's a question of sensitivity, obviously, of, of the data. But I'd be totally happy to run these models as long as they're inference.

[00:38:04] In places where I believe there's, you know, a better, a better degree of transparency, visibility, control, et cetera. So, you know, we've talked about gr, we've talked about AWS and Azure Foundry. You can run most of these models in most of those places.

[00:38:18] Mallory: Okay, so it wouldn't make sense for the government to ban source code or like the actual weights.

[00:38:24] Amith: I don't believe that's what's being discussed. I think it's access to the website, so. Gotcha. That's why I think it's important to separate using the model on your own equipment or an equipment that's run by someone you trust versus connecting to deep seeks website and using it as a consumer. I believe that is what the bill is intending to ban.

[00:38:41] Mallory: Mm-hmm. I wanna zoom out again to the trend line because as you mentioned with the first topic, that's what's important to keep in mind. This competition as a whole seems to be driving AI costs down, down, down really quickly. On the flip side, it doesn't necessarily seem like vendor prices, for example, are going down, down, down.

[00:39:05] So I, I'm thinking like HubSpot for example, their AI features, HubSpot is no cheaper for us than it ever has been. In fact, it's just the prices are going up. So I know I'm kind of talking about two different things there, but from your perspective, how do you see associations actually benefiting from reduced AI costs overall?

[00:39:24] Like directly benefiting.

[00:39:26] Amith: Sure. Well, I think there's a couple things. First. Having the knowledge that the costs are going down at such a rapid rate should open your mind to the possibilities of applications that you couldn't afford previously. So if you were to say, Hey, we want to go through every piece of content your, your organization has ever published, and we want to auto generate a taxonomy from it, or we want to, we wanna do other things with it, right?

[00:39:48] Um, in the past, maybe that would've been prohibitive from a cost perspective. Maybe the quality wasn't great enough either. But let's just say that you think the quality's awesome now, but the cost. Might have been a factor. We were like, oh, that would cost us $3 million to go through all that content. Um, and now it doesn't.

[00:40:04] Right now maybe it costs you $3,000 or something like that. So these shifts in costs should open your mind up to at scale applications, doing things at scale, or what about all the unstructured data that you have in your email, um, or elsewhere? What can you do with that kind of stuff? So. The way I look at it is the applications when, when cost is going down and performance and speed are going up, it opens up new possibilities.

[00:40:30] So that's important conceptually because a lot of what we're talking about on this pod and in the rest of our work at Sidecar is plotting your, your course, really your strategic direction, understanding the tools, understanding the technology, understanding how they all fit together, but thinking about where you should go with this.

[00:40:45] So clearly we want to optimize current business processes. That's the obvious part, right? We wanna make what we do. Happen faster, better, cheaper, right? Traditionally, you pick maybe, maybe one or two of those, but you don't get all three. Now you can get all three. Mm-hmm. But the bigger question is, what should you be doing as opposed to how do you do what you currently do better?

[00:41:05] And the what should you be doing is informed by constraints. And the constraints are the capabilities of these ais, but it's also the cost. So knowing that the cost keeps relentlessly going down should open your mind to the possibilities that six months, 12 months, 24 months from now. Video generation will be close to free.

[00:41:22] For example, right now, if you want to do, you know, a lot of hours of video generation with, hey gen, you have to sign up for plans that'll cost you in the five or even six figures. That's a constraint. Um, we're by the way, extensively using Hagen in a variety of our content production for our AI learning hub and for other things.

[00:41:40] Uh, and we're making the investment because we think it's worth it. But we also know that it's not really a long-term recurring investment because. There's competition in every one of these dimensions. So that's part one, is that I think you can think bigger and think a little bit longer term, something that, you know, you wouldn't say, oh, well I was gonna invest a million dollars in a new a MS and next year it's gonna be a hundred thousand dollars in the year after it's 10,000.

[00:42:02] That's not a thing, right? That's not the way people think, and that's not the way these systems tend to work. But with ai, that's literally what's happening with the fundamental models because of competition, because of the amount of capital being thrown at it. So that's true for what I'd call kind of the raw materials of ai.

[00:42:16] It's the, you know, the, the fundamental building blocks, the, the really, the models, um, at that layer. It's also happening with inference competition because so many people are going after being your cloud provider for ai. Uh, whether it's the hyperscaler major players like AWS and Azure, um, or if it's people who have a new approach to hardware, like our friends at the gr uh, which is GROQ to always repeat that for clarity.

[00:42:41] Uh, or a number of other folks that are out there doing cool things of the difference. That's a hypercompetitive market. You're gonna keep seeing costs come down there. Um, but to come back to the second part of your question about like. Well, how come we aren't seeing costs for the, the application layer?

[00:42:55] Mm-hmm. Consuming, which consumes these AI is for one, it's just, it's, you know, it's market economics. There's fewer com competitors, right? Than there are at the model level. Uh, there's also more complexity in terms of integration with systems. There's kind of the nuance of what a lot of times is referred to as the last mile of the solution actually solving the problem.

[00:43:15] Uh, so you can take the model that has Gemini 2.5 pro caliber thinking, but then to wire it up into your. Ecosystem and make it work just right for you. Still today. There are specialized pieces of software that have to be integrated. There's a lot of labor that's required. There's domain knowledge in terms of what the best processes are.

[00:43:33] So the actual end solutions, I think you are gonna see them come down in cost overall. Um, but it's, it's slower. Um, it's kinda like if we said, Hey. Let's just imagine that the price of steel was close to zero, all of a sudden, would that mean that the price of cars immediately goes close to zero as well?

[00:43:51] You know, when you're further down, uh, in the supply chain, you know, yeah, there, there that can happen, right? Cost of materials can go up and down. Sometimes it has a ripple effect. Um, and sometimes it takes a lot longer to, uh, work its way through the system. But ultimately, this is a really good thing, the component costs of the ultimate solution.

[00:44:09] Obviously are a major factor in that solution. Um, but more than anything, it's this abundance mindset that I'm trying to really hammer into people's minds that you shouldn't think of ai, video generation, or ai, whatever as a, as a scarce resource, that you can only use it in a tiny number of areas. You should be thinking that by the end of this year, you'll probably be able to do it across everything.

[00:44:31] So every single blog post that you have. Let's have a great video on that post every single time completely AI generated today. That might cost you tens of thousands of dollars. By the, of the year, it'll probably be, uh, considerably less. Hmm.

[00:44:45] Mallory: I love that I call my version of that. So you said dream bigger.

[00:44:48] I call it often breaking your brain because it's constantly, every day, all day having to say like, what else could I do with this model? Uh, breaking these barriers of what you thought was possible, especially when we talked about member services, right? Having a member service agent that doesn't just answer basic questions, that actually has access to your whole knowledge repository and can answer domain specific questions.

[00:45:09] That's something that you have to really. Open your mind to, so I agree totally. I think that's essential. And actually, now that I'm thinking about it, that it could be an interesting take maybe for, for digital now, 2025, I don't know if we have a theme hammered out, but kind of the idea of dream bigger, uh, breaking your brain.

[00:45:26] What I, I like that. I like, I like breaking

[00:45:28] Amith: your brain. That sounds fun.

[00:45:30] Mallory: I don't know. So maybe, maybe you all are the first to hear our theme for this year. But with that, everybody, thank you for tuning into today's episode. I know Amis mentioned Grok with aq. We do have a special interview coming up. Not sure when we're gonna post it, but in the next few weeks or so.

[00:45:46] So be on the lookout for that and we will see you all next week.

[00:45:52] Amith: Thanks for tuning into Sidecar Sync this week. Looking to dive deeper. Download your free copy of our new book, ascend. Unlocking the power of AI for associations@ascendbook.org. It's packed with insights to power your association's journey with ai.

[00:46:08] And remember, sidecar is here with more resources from webinars to bootcamps to help you stay ahead in the association world. We'll catch you in the next episode. Until then, keep learning, keep growing, and keep disrupting.