The Open-Weight Wave & the Push for AI Referees | [Sidecar Sync Episode 144]

Summary:

In this episode, Amith Nagarajan and Mallory Mejias unpack a rapidly shifting AI landscape where powerful, low-cost open-weight models from Chinese labs are challenging the dominance of major U.S. players. They explore Moonshot AI’s Kimi K3 and Alibaba’s latest model announcements, breaking down what “open” AI really means and why it’s driving costs down while increasing competition. The conversation also dives into the growing geopolitical tension around AI, including potential U.S. government intervention and the risks and realities of using models developed abroad. Finally, they discuss a proposal from DeepMind’s Demis Hassabis to introduce an industry “referee” for AI safety—and what it could mean for innovation. The big takeaway: intelligence is becoming abundant and affordable, and association leaders must rethink strategy now to take advantage of what’s coming next.

Timestamps:

👥Provide comprehensive AI education for your team

https://learn.sidecar.ai/teams

📅 Register for digitalNow 2026:

https://digitalnow.sidecar.ai/digitalnow

🤖 Join the AI Mastermind:

https://sidecar.ai/association-ai-mas...

🎀 Use code AIPOD50 for $50 off your Association AI Professional (AAiP) certification

https://learn.sidecar.ai/

📕 Download ‘Ascend 3rd Edition: Unlocking the Power of AI for Associations’ for FREE

https://sidecar.ai/ai

 🛠 AI Tools and Resources Mentioned in This Episode:

Demis Hassabis has a Plan to Harness AI Safely ➔ https://shorturl.at/88ugt
US Threatens Sanctions Against Chinese AI Models Over IP Theft ➔ https://shorturl.at/oQUnw
Claude ➔ https://www.anthropic.com
ChatGPT ➔ https://chat.openai.com
Gemini ➔ https://gemini.google.com
Kimi K3 ➔ https://www.moonshot.cn
DeepSeek ➔ https://www.deepseek.com
Qwen ➔ https://qwen.aliyun.com
Microsoft Azure ➔ https://azure.microsoft.com
Amazon Web Services ➔ https://aws.amazon.com
Google Cloud Platform ➔ https://cloud.google.com 

👍Please Like & Subscribe!

https://www.linkedin.com/company/sidecar-global

https://twitter.com/sidecarglobal

https://www.youtube.com/@SidecarSync

Follow Sidecar on LinkedIn

⚙️ Other Resources from Sidecar: 

• Sidecar Blog
• Sidecar Community
• digitalNow Conference
• Upcoming Webinars and Events
• Association AI Mastermind Group

More about Your Hosts:

Amith Nagarajan is the Chairman of Blue Cypress 🔗 https://BlueCypress.io, a family of purpose-driven companies and proud practitioners of Conscious Capitalism. The Blue Cypress companies focus on helping associations, non-profits, and other purpose-driven organizations achieve long-term success. Amith is also an active early-stage investor in B2B SaaS companies. He’s had the good fortune of nearly three decades of success as an entrepreneur and enjoys helping others in their journey.

📣 Follow Amith on LinkedIn:
https://linkedin.com/amithnagarajan

Mallory Mejias is passionate about creating opportunities for association professionals to learn, grow, and better serve their members using artificial intelligence. She enjoys blending creativity and innovation to produce fresh, meaningful content for the association space.

📣 Follow Mallory on Linkedin:
https://linkedin.com/mallorymejias

Read the Transcript

🤖 Please note this transcript was generated using (you guessed it) AI, so please excuse any errors 🤖

[00:00:00:14 - 00:00:09:17]
Mallory
 Welcome to the Sidecar Sync Podcast, your home for all things innovation, artificial intelligence and associations.

[00:00:09:17 - 00:00:26:12]
Amith
My name is Amith Nagarajan.

[00:00:26:12 - 00:00:28:05]
Mallory
 And my name is Mallory Mejias.

[00:00:28:05 - 00:00:40:19]
Amith
 And we are your hosts and I am here in New Orleans awaiting the arrival of Tropical Storm Bertha. But I'm in the office so if the power goes out or if I just disappear, you know why.

[00:00:40:19 - 00:00:53:18]
Mallory
 And you'll just hear me talking through the rest of the episode everybody. So sit back and enjoy. We might miss some technical concepts but we'll still be here. I mean, I didn't really, I don't know a ton about Tropical Storm Bertha. Is it coming in today?

[00:00:53:18 - 00:01:37:18]
Amith
 Yeah, it's apparently just a little bit southeast of New Orleans. It's not expected to be anything terribly enormous but that's a lot of times actually when these storms aren't categorized as being really big. I mean, tropical storms are below the hurricane threshold obviously but they do carry with them quite a punch both in terms of wind speed but also the water. And so that's what people in New Orleans have to be particularly worried about is flooding. And so, you know, the city should be prepared for this and hopefully everything will be all right and the storm passes us by without too much damage but we will see. So yeah, I'm gonna be heading home in a little bit here and you know, get back before things get too crazy.

[00:01:37:18 - 00:01:54:08]
Mallory
 Yeah, well, I'm hoping all goes well with that. And for anybody listening to this episode impacted by tropical storm Bertha, stay safe. We're thinking about you. Amit, you were fresh off of a really fabulous trip it sounds like to Scotland or how are you doing? Are you still jet lagged? Are you ready to be back into the world of AI?

[00:01:54:08 - 00:02:38:23]
Amith
 I am totally ready. I've been back for six days. So, you know, I think they say that it's like one hour adjustment per day roughly to be fully recovered and it's been six days and it was six hours of time zone change. But really actually I always find coming, going from East to West or heading West tends to be the easier trip, particularly for Europe because it's, you know, anywhere from six to eight hours that kind of thing. It's not too bad because you can kind of force yourself to stay awake on the flight over the Atlantic and then it's a long, long day. But then you get back to, in my case, New Orleans and you pass out around eight or nine o'clock or whatever it is and you're pretty tired. So it still takes a minute to adjust but I've always find going the other way is a lot harder to deal with.

[00:02:38:23 - 00:02:42:19]
Mallory
 A hundred percent. Yeah, did you watch any World Cup games while you were there?

[00:02:43:22 - 00:03:15:02]
Amith
 While we were there, we did not. We kept meaning to, but there's just so much to do in Scotland and we had such a fantastic time. This is my first time visiting Scotland, had an amazing time there. And we didn't get around to watching World Cup but it was quite fun talking to people all over Scotland about the World Cup as the different games progressed because first and foremost, of course, they were rooting for Scotland. And my understanding is back here in the States, I believe that Tartan Army in Boston drank the city out of beer. That's not an AI headline, but I find that quite cool.

[00:03:16:07 - 00:03:30:21]
Amith
 And I can see how because on a per pound or per kilogram basis, I think Scottish people certainly know how to enjoy their beer, amongst other things. So we had a great time doing that. But what was interesting was first and foremost, they're rooting for Scotland.

[00:03:31:23 - 00:04:00:02]
Amith
 Then they're rooting for whoever's playing England. So it was quite fun. You kept seeing flags up in different places. So you'd see the Norway flag up. And then I don't know how they felt about the final ultimately, or not the final, but the semi-final with France. I think Scotland has a pretty strong traditional alliance with France. So probably rooting very strongly in favor of the French. So that was quite fun to hear all the different people around the country talk about that.

[00:04:00:02 - 00:04:28:24]
Mallory
 Yeah, I asked about it because I was watching a lot of World Cup games this year. I'd mentioned on the podcast before, but I lived in Spain for two years. So I was very excited to see Spain win the World Cup. But in prepping for this episode, I used Claude of course, and Claude actually titled the episode, "The Open Weight Wave and the Push for AI Referees." And I think that was just a coincidence, but I was thinking referees, the World Cup, referees in AI. It seems all related today.

[00:04:28:24 - 00:04:35:03]
Amith
 Yeah, for sure. And I think Scotland might eat some referees when it comes to whiskey tasting.

[00:04:35:03 - 00:04:35:22]
Mallory
 And that.

[00:04:37:03 - 00:04:44:13]
Amith
 That's definitely a thing over there. And I've always enjoyed Scotch. Well, not when I was a little kid or something, but like since then, you know, whatever.

[00:04:44:13 - 00:04:44:24]
Mallory
 We're bonding, I mean.

[00:04:44:24 - 00:05:16:07]
Amith
 But I've enjoyed Scotch my entire adult life, and I've always endeavored to learn more and more about it. So this trip was both a whiskey tasting expedition and a really fun time playing some golf and taking in the history and the beauty of the natural world in Scotland. But I say what I said about referees and the whiskey world in Scotland, because there's lots of points of view. And it's fun to go to a bar and ask different people what they think of different scotches. And there's very strongly held beliefs about like level of Pete and all these other things that it's quite interesting.

[00:05:16:07 - 00:05:27:10]
Mallory
 Yeah, were you able to really tell, were you able to tell the difference in the Scotch you try? I feel like every sip of Scotch I've ever had tastes the same and bad in my opinion, but I guess to a common store.

[00:05:27:10 - 00:05:30:12]
Amith
 If you don't like Scotch, you definitely,

[00:05:31:15 - 00:06:02:21]
Amith
 you're gonna feel that way probably about all of it. But there's definitely tremendous variation. It's like wine, there's so many different ways to do all the different steps in the process. There's differences in regions, there's differences in where the grain is grown and how they do the malting and all sorts of different variations. So it's an almost unlimited number of ways that you can alter the flavor profile of the whiskey. So it's pretty amazing. And so I'm a particular fan of heavily peated Scotch, which you'd probably hate even more. That's the stuff that's extra smoky.

[00:06:04:00 - 00:06:22:21]
Amith
 And so I went to an island called Isla, it's I-S-L-A-Y, but it's pronounced Isla. It's an absolutely beautiful place. Even if you don't enjoy whiskey, it's worth visiting. Played some golf there and went to, I think six or seven of the, I think they had nine operational distilleries. I went to six or seven of them. So it was a good time.

[00:06:22:21 - 00:06:31:16]
Mallory
 Wow, so lots of unique ways to make Scotch, just like there are lots of unique ways to utilize AI in your association. So I feel like that's a good segue.

[00:06:32:16 - 00:06:59:11]
Mallory
 In about a week this summer, the ground under AI, the AI world has been shifting quite a bit. Chinese companies are beating the big American names at their own game. Washington started arguing about whether to step in. And one of the most respected leaders in AI proposed building a referee for the whole industry. So we're gonna walk through it all because underneath it's the same question, who is in control of this technology and what does that mean for the work that you are doing at your association?

[00:07:00:15 - 00:07:19:03]
Mallory
 So the models most people know think Chat GPT and Gemini and Claude come from big US companies. Today we're talking about a model that came from a Chinese startup, Moonshot AI that we have discussed on the podcast before. And they released a model called ChemE K3 on July 16th of this year.

[00:07:20:06 - 00:08:02:16]
Mallory
 Here's the thing, it's roughly as capable as some of the best models out there and about a third the price. Against Claude Opus 4.8, the most capable public model when it launched in late May, ChemE K3 came out ahead on four of five major industry tests losing only on a broad general knowledge one. Another differentiator with ChemE K3 is that it is open. So Moonshot says it'll give away the finished model itself and its weights so you could run it on your own computers and fine tune it on your own data instead of renting intelligence from a company that can change or shut it off. So Amis, a capable model at a third of the price, what is your take on ChemE K3?

[00:08:02:16 - 00:08:31:05]
Amith
 I mean, it's the latest, you know, latest blip on the radar, so to speak, but it's the directional trend line we've been talking about pretty much for 144 episodes Mallory. We've been talking about how models keep getting smarter and faster and cheaper, but also how the frontier keeps pushing forward and there's progression there on cost as well. And so this is not quite frontier. If you consider frontier to be the very best level of intelligence, that would be GPT 5.6

[00:08:32:07 - 00:08:40:03]
Amith
 and also of course, Claude, Fable and Methos, but this is right below that. This is Opus 4.8 caliber intelligence.

[00:08:41:03 - 00:09:32:01]
Amith
 It is one of the smartest open source models available. And the fact that you can very soon download the weights and read it on your own hardware means it's not only a third of the cost in ChemE's, their cloud essentially, but it could be dramatically less expensive than even that if you were to run it in your own hardware, which most associations will not ever run their own models, but it means there's going to be competition. There will be a lot of providers, including fast inference providers such as Cerebrus. There are a number of larger scale providers that are out there like even Microsoft with Azure that are experimenting with ChemE K3 to look for opportunities to lower their inference bill. There was a note posted on Reddit somewhere, so of course it has to be true, that Microsoft believes they can save $600 million a year just by switching some of their co-pilot workloads over to ChemE K3.

[00:09:33:05 - 00:10:17:23]
Amith
 And so that's a very interesting thing to be thinking about considering that Microsoft obviously owns a big chunk of open AI, is an investor, I believe also in anthropic, but most notably, as we've covered recently in the pod, is actually developing their own models. They're very open-minded. They want to use the model that's best for the job at the lowest possible cost. It's kind of like saying, "Hey, where are you going to buy your gas? Do you really have loyalty to Shell or Exxon or whichever gas station?" If the gas was appreciably less expensive for the literal same product nearby, you'd probably go to that gas station. And so that's the classic commodity-based thinking. So that's really my main kind of point on this is that intelligence is commoditized, not being commoditized, but basically is commoditized if you think about it.

[00:10:17:23 - 00:10:36:07]
Mallory
 Hmm. Well, Amith, we have some new listeners to the Sidecar Sync podcast. And so for someone, this might be their first episode tuning in, if so, welcome. But for someone hearing open weight or open source in terms of AI models, can you just give a high level of what that means and how it compares to a "closed" model?

[00:10:36:07 - 00:10:42:16]
Amith
 Yeah, and welcome to the Sidecar Sync. If you haven't been with us for a long time, we're very happy that you're here.

[00:10:43:17 - 00:11:43:06]
Amith
 Malory, an open model, typically called open source sometimes, more correctly termed open weights, is a model that is capable of being run in your own environment. So you can download the weights and you can run the model on your own hardware. Now you have to have sufficient hardware, which oftentimes means hundreds of thousands of dollars of dedicated hardware, in the case of a model as large as Kimmy K3. But other models that are smaller in nature can run literally on a MacBook Pro. And so it is possible to run models locally, and that has several advantages. One of the advantages is cost. So if an open weights model is published under permissive terms, meaning that the license allows you to do basically whatever you want, which almost all of these model releases are, it means you don't have to pay royalties. You can create derived versions of the model if you wanted to fine tune it or create a competing model that's based on it. You can do all those things. It's very much like open source software. It's the same ethos as that.

[00:11:44:06 - 00:13:32:22]
Amith
 In addition to lower cost, the other thing that it does is it really gives you more choice. So you can run that same model in lots of different places. So will most companies and certainly most associations ever run their own AI models in production? I would venture to guess that most will not. But what it means is that there are literally dozens and dozens of large scale inference providers. And an inference provider is a company that hosts and runs the AI model for you. So in the case of Moonshot, which is, I love the name by the way, Moonshot AI in China, they are offering inference on their own model. They're both a model developer and also they are at the moment, the only inference provider for Kimmy K3. They are about to release the open weights. By the time you listen to this pod, they may already be out. There's a slight delay and there's lots of speculation on why that is normally. Weights are released along with the model announcement. But regardless, very soon, anybody in the world will be able to run Kimmy K3 and offer it as a service, which is going to create downward pressure, right? Because that ultimately arbitrages all the profit out of it over time, because there's gonna be people looking for different ways to find ultimate benefit from it. And then the other thing too is, this is a little bit of a precursor to the next beat, but we have other models coming out right behind Kimmy K3 that are just as good and this will keep going. So open weights to summarize the explanation you asked for Mallory just means that it's like open source software. You can download it, you can run it. There's no cost involved other than owning the hardware. If you choose to have someone else run it for you, there is cost, but ultimately that squeezes most of the profit out of the mix because everyone's competing to deliver exactly the same service, AKA the definition of a commodity.

[00:13:34:21 - 00:13:56:18]
Mallory
 It sounds like there's a distinction between utilizing open models as owning intelligence and then utilizing closed models as renting intelligence, but then there's also the distinction of perhaps using an open model, but as you said, Amith, using an inference provider to run it for you. What do you think about the idea of owning versus renting intelligence? Is it that simple? What do you think?

[00:13:56:18 - 00:14:26:13]
Amith
 Well, I mean, I think ultimately because these models are changing so fast, you're probably not gonna stick with the same exact model for very long. Even consumers using chat GPT are getting a different model every few months. And so, most consumers using the free version of chat GPT now, I think they're all getting GPT 5.5 instant. Maybe they've been upgraded to the smallest GPT 5.6, but you don't really notice, but you just get better and better version of the car essentially, you keep getting an upgrade.

[00:14:27:13 - 00:15:04:19]
Amith
 And so because the models are changing so rapidly and they're still upside to taking advantage of these newer models, which I'll come back to in a second. Like I think letting other people run the models probably makes sense for a while, but I do think there's an inflection point coming related to the level of practical intelligence that you need. So imagine a world where you can hire a real, you need to hire a member services professional. So you want someone who has great communication skills. Obviously you want someone that's sharp, learns quickly. Someone who has a high level of EQs, they have empathy for the customer

[00:15:05:21 - 00:15:38:21]
Amith
 and has a generally good skillset, right? They're good at using a computer, using AI tools, working with your database, et cetera. And they need to learn your domain pretty quickly too. They need to learn all of your events and all of your membership offerings and so forth. So let's say you had the choice between someone who met the criteria and they were kind of a, I would say an above average, but still typical member services type person, or you had Albert Einstein, who may not fit some of the criteria I just mentioned, but do you need Albert Einstein's intelligence

[00:15:40:04 - 00:16:40:18]
Amith
 to do that job? I would argue you probably don't. And the point is more of a crude approximation, but like clearly an ultra genius like that, do you really need that to solve most of your day-to-day problems? Maybe some of your work, you do have things that are like, hey, we don't know how to solve this problem. We have a member attrition problem. We just can't think of ways to solve it. But for most of your work, you don't need that level of intelligence. So my argument is this, Opus 4.8, is a lot smarter than I am about most things. And I would tell you that I think that level of intelligence is probably all you need for most business tasks. Now I could be wrong about that. And there could be a need for, let's say fable level intelligence, or maybe even what comes after fable. But at some point, the curve kind of flattens in terms of the need for intelligence. So the upward trajectory of demand in terms of how smart the model is for a lot of the tasks we do, will start to level out.

[00:16:41:20 - 00:16:45:00]
Amith
 Now that's an important thing to understand because so far we haven't experienced that.

[00:16:46:04 - 00:16:56:04]
Amith
 So if I'm right about that, that means that the demand will grow in terms of volume, but then costs will become a much bigger factor if the level of intelligence is assumed to be good enough.

[00:16:57:18 - 00:17:25:06]
Amith
 So that's the thing I think we're really on the brink of that, we haven't experienced that yet because Mallory, if I were to say to you, hey, you don't get Claude Opus 4.8 or fable in your work day to day anymore, but rather you gotta go back to working with Claude Sonnet 3.7 from last summer, or whenever that was, right? 18 months ago, which at the time, I think we were all saying, wow, this is an amazing model. But if you were to go back in time and use that model today, you'd probably be pretty underwhelmed compared to the intelligence you take for granted.

[00:17:26:09 - 00:18:13:15]
Amith
 I'm not suggesting we're there yet in terms of we have everything we need, but I do think most of the tasks people want to run through most AI models were very close to that point. And that's certainly true for agentic workloads, where if you're interacting with AI directly in chat GPT or whatever, your usage represents a tiny sliver of AI demand. The AI demand that's growing the fastest is what's called agentic workloads, which is where there's a computer program running a loop that keeps basically asking a prompt over and over and over again to make decisions and do other things. That's a radically larger category of inference demand and therefore cost. So those actually, we've already utilized kind of the same level of intelligence for that key decision-making process in our products for some time, because you don't need unlimited intelligence for those things.

[00:18:15:00 - 00:18:52:10]
Mallory
 That's a good point. I wanna shift gears a bit and talk about an even more powerful model. Amith right after moonshot Alibaba, which is roughly China's Amazon, announced its own giant model, Quinn 3.8 Max, claiming it was second only to Anthropic's top model, Claude Fable 5. It's a claim with nothing behind it just yet. Alibaba touted 2.4 trillion parameters, but published no benchmarks and never said how many of those parameters actually fire per task. I wanna take a brief pause here, Amith. I know we're talking lots of technical terms. What are parameters and what do our listeners need to know about model size?

[00:18:52:10 - 00:20:12:21]
Amith
 The most important thing to understand is that there generally has been a truth to the statement that the bigger the model, the smarter it is, the more capable it is in solving complex problems. So when you hear about a model that has three trillion parameters versus a model that has three billion parameters, generally speaking, the three trillion parameter model is going to be more capable. That's actually not always true because there's certain highly specialized tasks that you can make a smaller model that's really focused on. It's just like specialization of labor. You may have someone that's really good at welding, but isn't as good at something else. So models are kind of like that too. So it's a lot more nuanced than saying the bigger model is always better, but generally speaking, larger models have more capability. And so there's this concept called scaling laws. And we've talked about that in the pod in the past where in around 2017 timeframe, people were starting to say, well, what happens with the transformer architecture, which was first published then, and subsequently the scaling laws papers said, well, if we just make the transformer bigger and bigger, we will get smarter and smarter capabilities as well as so-called emergent properties, meaning things that the model can do that you didn't expect it to be able to do. Like the initial models were not trained to write code, for example, coding was initially kind of a surprise, right? And emergent properties, like the fancy way of saying, what, this thing can do what?

[00:20:14:02 - 00:21:08:23]
Amith
 That's pretty cool. So you get more emergent properties out of these bigger models because they have these connections that you wouldn't necessarily expect. Another quick point I'd make for our newer listeners, or just as a refresh for those who've been with us on this journey for a while, is that parameter count to give you a slightly more technical explanation is roughly akin to the synapses or the connections between biological neurons. So it essentially is telling us, how much the weights are between neuron one and neuron two. So that's what's important. You have parameters and then you have the weights and then the two combined to make the model complexity essentially. So that's the way to think about it is, and you don't really need to know a whole lot about it beyond that, but bigger models tend to be smarter, but they also come with a disadvantage, which is they tend to be much more expensive to run, and they're also more costly for you to use as a user over the internet.

[00:21:08:23 - 00:21:11:21]
Mallory
 And they tend to be slower as well to use, right?

[00:21:11:21 - 00:21:12:18]
Amith
 That's correct.

[00:21:12:18 - 00:22:14:13]
Mallory
 I wanna shift gears though to the open source angle. Alibaba is one of the most open players in AI. Most of its models ship under a permissive license anyone can use, but it's very top max models have always been the exception. So they are closed in API access only. This time Alibaba says it'll open even this flagship model, a real reversal, except it's a promise for now with no date and no license just yet. So if we zoom out a little bit, it's not just two stories, but one. So actually four major Chinese labs, Moonshot and Alibaba plus DeepSeek and Minimax have now shipped or announced enormous models in about a month. And what used to be the turf for a few US companies is fast becoming a crowded, largely open field. So Amith, I wanna ask you, Alibaba giving away most of its models, but keeping its best one locked up until maybe now, what do you see happening when the world gets its hands on an open model almost as powerful as Claude Fable 5? What do you think about that?

[00:22:14:13 - 00:22:54:10]
Amith
 Well, I think in the immediate term, nothing will change because it takes a while for this type of stuff to diffuse, but this is a geopolitical and strategic thing at the business level too. It's a big deal because we say, look, over here, we're very carefully vetting these new models in terms of the release process within these companies, at least that is what is said to be done at Entropic and OpenAI and Gemini. I generally believe that's true. These safety teams have a really important job to do. And then the government's even stepped in and said, "Hey, before you release something like Fable or GPT 5.6, you gotta come ask for our blessing essentially, which is both good and bad in different ways."

[00:22:55:16 - 00:23:13:05]
Amith
 But we have no control over what other countries do here in the United States. We have no control over open source models released by China or by anybody else for that matter. So it's a really interesting problem. The overall trend line is a very good trend line, which is that there's more powerful AI available.

[00:23:14:05 - 00:24:09:23]
Amith
 But whereas Fable and Methos and GPT 5.6, there was all this talk about the cybersecurity risks, the bioweapons risks, these other downsides of AI, and these companies trying to work really, really hard to prevent those kinds of misuses, that will not be the case with an open source model. So an open source model that you can download, you can tune it, you can change it, you can do whatever you want to it. So all that power is available essentially without those restrictions. So it's concerning for a lot of reasons. As a public citizen of the world, it is noteworthy because we should be paying attention to this. What do you do about it? Well, ultimately, even models like Claude Fable, you can't really count on those things being behind any meaningful lock and key for very long, because the models get smarter so fast, like we're talking about. But it's just something worth noting, I think. My general view is this is gonna keep happening faster and faster.

[00:24:11:00 - 00:24:55:19]
Amith
 There are a lot of Chinese labs doing really good work. There's actually American companies like Thinking Machines Labs introduced a, I think, Opus 4.8 caliber open source model they're putting into the world. And they also have a unique approach to it because they're really focused on making it easy for companies to fine tune the model on their own data, which has typically been a very difficult thing to do. So that's an interesting attack angle on the market. There's a lot of people playing in this space. There's big prizes out there, and there's a lot of people motivated to go and drive this. So I have my opinions on it. I wish it could be slowed down in terms of certain types of model releases. I've said that for a long time, but I don't think that's a realistic thing that you can really put in place with the global players that are out there.

[00:24:55:19 - 00:25:04:09]
Mallory
 Is it possible to release an open model with guard rails bolted on, or because it's inherently open, is it just easy to remove those?

[00:25:05:11 - 00:25:33:24]
Amith
 It's pretty easy to remove those. You can build into the model training process certain things. So like, for example, Anthropic is well known for their constitutional AI approach, which simply means that the model is continuously trained from the very beginning on this idea of a constitution. What's good and what's bad, right? It's a value system, essentially. And so the open source developer could certainly do that. I think they do, in fact, have things similar to that because I don't think these folks want to put something terrible into the world.

[00:25:35:06 - 00:25:59:08]
Amith
 I don't think that's the case at all, actually. I think they want to put good models that are safe into the world. However, once you own the model weights, you can do your own additional RL, you can do fine tuning. There's a lot of things you can do to change the behavior of the model where it will quickly forget even those types of guard rails that were trained into it. So that is the concern. Whereas, Claude Fable, nobody has access to it other than Anthropic.

[00:25:59:08 - 00:27:22:00]
Mallory
 Well, with powerful Chinese models, suddenly free to download and available and open and powerful. The US government, like you said, Amith, had began weighing in on whether to step in. So that's limiting agencies from using them, issuing security warnings, et cetera. To be precise, though, this is just being debated right now. It has not been decided. KEMI K3, which we just covered, is actually what tipped it from abstract worry into an active question, a genuinely capable model from a Chinese lab that anyone can pull down and run. The tools on the table are the usual policy levers, so restrict federal procurement, ad labs to trade blacklists, issue security advisories. But some of those on the receiving end argue the real motive is US incumbents leaning on Washington to slow cheaper competitors. The hard part here is once a model is released for free, it obviously spreads across the internet within days and can't really be pulled back. You could stop a chip shipment at the border, but you can't un-download a file already on thousands of machines. Even the White House's own AI advisor has warned new rules could cause more problems than they solve. Now, Amith, I'm realizing a lot of this episode, the bulk of this episode really is about Chinese labs and these open models, but I'm sure we have listeners that are thinking, "Mallory and Amith, I would never use Chinese AI at my association. It's not safe. Why are we even discussing this?" Can you speak to those people, Amith?

[00:27:22:00 - 00:27:31:21]
Amith
 I think the first thing I would say is it's very good that you're thinking through this because the models you use and where you choose to run those models are very important decisions.

[00:27:33:03 - 00:27:51:24]
Amith
 I would caution people from having a generic view like that though and holding it too closely because first of all, not all of the Chinese labs are the same. There's many different labs. There's actually a bunch more that aren't really being discussed publicly, but are producing a ton of models. It's an extremely large community out there that's very well-funded. And then secondly,

[00:27:53:06 - 00:30:36:07]
Amith
 you need to think about the difference between where the model was created and where it's being run. So should you go to the DeepSeek website and use it for sensitive data in your association? My answer to that would be no because that website is run on servers outside of this country and not subject to any laws that we have any ability to have any say into or enforcement around. So if you were to upload your membership directory or membership database to deepseek.ai or to the QWEN site, which is Alibaba, or any of these other ones, you are essentially providing your data to those providers. Now they would in turn argue by the way that they have a terms of service just like anthropic and open AI that prevents them from doing anything malicious with your data, training models. It's a question of trust. And you may be comfortable with that, or you may not be. I think most association leaders I talk to in the US and also in Europe and in Canada and in Australia are pretty concerned about that idea because it's known that generally speaking, if the Chinese government says they want access to the data within a Chinese private enterprise, they can get access to it. So that's probably a concern aside from whether or not the model training company will use your data to train the next model. So that is one part, but the other part is, well, what if Microsoft on Azure where you probably have all of your critical business data already, what do they offer, KIMI K3? Because actually Microsoft has offered every KIMI model since version two. They have K2, K2.5, K2.6, they have all the KIMI models running on their hardware. And that is subject to Microsoft terms of service, your enterprise agreements with Microsoft. By the way, the same thing is true for Amazon with their AWS cloud service, and also with Google with GCP. All three major cloud providers offer some type of AI inference service that is subject to the same cybersecurity, the same contracts, the same terms of service that you entrust with your business critical data. So my argument there would be, well, maybe there you should consider it because if the model is from anthropic or open AI or another US lab versus a Chinese lab that's running on the hardware of Microsoft or Amazon in the United States, you're probably in pretty good shape. Concerns people may have might be things like is there possibly a backdoor in KIMI K3 where the data can be exfiltrated back to a server in China? And there is a ton of testing that goes on and that's extraordinarily unlikely.

[00:30:37:11 - 00:30:40:19]
Amith
 It is, nothing is ever impossible, but it is extraordinarily unlikely.

[00:30:41:21 - 00:31:05:21]
Amith
 And one of the nice things about AI models is the code that actually defines like the possibilities of what the model can do is actually very small. There's not a ton of code behind these AI models. It's thousands of lines of code, not millions. So it's actually very observable and very definable what the potential risk is. Now the model's intelligence and what it can choose to do if you give it tools. So for example, if you say, I'm gonna take K3

[00:31:06:22 - 00:31:59:12]
Amith
 and I'm going to give it access to the internet, I'm gonna give it access to my database, I'm gonna give it access to all these other things, can it do stuff you don't want it to do? And the answer is possibly, but that's also true for Fable. It's also true for open AI's tools. Will the model somehow bake deep into the neural net, the weights of this model be thinking it should find a way to exfiltrate your data to China? Extraordinarily unlikely. Again, I'm not saying it's impossible and I'm not giving anyone advice here. I'm just trying to provide you some goalposts to think about in terms of how to measure your distance from each of these targets, right? So it isn't an absolute, but it depends on your risk profile. I would use Chinese models just for Blue Cypress workloads and we do all the time. I just wouldn't run them on Chinese servers. I would run them on servers at companies that I trust, whether it's someone like a fireworks, who's a major inference provider with servers here in the States or someone like a GCP or an Azure.

[00:31:59:12 - 00:32:08:06]
Mallory
 Okay, so it sounds like the important takeaway here is, where you are running the model is probably more important than where it was created, if that's not oversimplifying.

[00:32:08:06 - 00:32:59:12]
Amith
 Yeah, and I think that's the right general advice and if you are willing to use, I think people think a lot about this, which is good. I think people think a lot less about stuff they're already comfortable with, right? So once you're comfortable with something, you tend not to pay attention to it. And a good example that still to this day is the allowance of letting AI note takers into Zoom and Teams calls. People routinely allow any random AI note taker that one of their attendees wants to bring into their meetings to just come on in, record the conversation, record the transcript and you have no idea where that AI note taker is coming from. And so I would argue that you should probably prohibit that and not allow AI note takers from outside parties to enter any of your meetings because you have no control over that. And oftentimes what you talk about in your meetings is some of your most sensitive content there is.

[00:32:59:12 - 00:33:14:22]
Mallory
 Amit, you said that we are using Chinese models for some of the work that we are doing across the Blue Cypress family of companies. Are you taking, are you concerned about the government regulations and restrictions? Are you okay just waiting to see what happens? What's your take?

[00:33:16:02 - 00:33:38:12]
Amith
 I'm definitely fine with the wait and see kind of mindset for now, I don't think there's gonna be any major shutdown. And the other thing is exactly how do you do that? You can say, hey, all US companies you're hereby prohibited from offering inference services or even using Chinese models yourself. I mean, that sounds like an enormous mistake because that would just slow down everyone in the United States.

[00:33:40:03 - 00:34:08:10]
Amith
 So I don't know that that's even close to being possible, but you never know. So I think a wait and see approach is reasonable in terms of the government regulation side. I'm not waiting to see what the government does to use these models. I'm gonna use the best engine for the job. And as long as I can run it in a verifiable safe way, which to me that specifically means servers in the US from an inference provider I trust, I'm gonna be willing to run a pretty wide array of models.

[00:34:10:11 - 00:35:19:16]
Mallory
 Well, everybody you can think of this episode thus far as the World Cup of Artificial Intelligence, but we've got good news. The industry is proposing its own referee. So on July 14th of this year, Demis Hassabis, CEO of Google's AI Lab DeepMind and a Nobel Laureate, proposed an independent group to vet powerful models before release, checking for serious risks like cyber attacks or dangerous biology. The model is FINRA, which polices Wall Street, funded by the industry, but operating independently under government oversight. So not a government agency, but not the companies grading their own homework either. He wants it running before year's end. Now I drew unusual agreement. Leaders at Microsoft, OpenAI and elsewhere signaled support with Hassabis citing this summer's messy improvised government interventions as the wake up call. But not everyone sold. One analyst called industry self policing, foxes guarding the hen house. And the deeper worry is that rules written by the biggest players lock in their advantage. So I mean, you sent me this link. You said, we've got to cover this on the podcast. What do you think about Demis's proposal?

[00:35:21:04 - 00:36:13:14]
Amith
 I'm out of the various different ideas that are out there. I do think it's the one that probably has the most merit that I've heard of anyway. And so the concept there, when we say do something like FINRA, it means that essentially the industry pays for it, but doesn't control it. And so there has to be essentially an entity that's formed and managed and governed that is not controlled by the industry. And the reason it has to be funded quite heavily is to get the technical expertise you need into that entity, you're gonna have to pay people a lot of money. That expertise is right now concentrated principally in AI labs. There are some people in the world of consulting and things like that that are quite good at AI, but you need people that are extremely deep at AI to have any chance at being useful in performing the duties. You can't have like a typical government institution that's bounded by all the rules with respect to compensation. So it's gonna be an expensive thing to do. So it has to be funded by the industry.

[00:36:14:16 - 00:36:16:09]
Amith
 Nobody else is gonna be able to fund this essentially.

[00:36:17:12 - 00:36:48:04]
Amith
 But at the same time, it has to be totally neutral, right? So will it be something that potentially could advantage one of the companies or another company? I would suspect probably not. There's always opportunities for corruption and these kinds of things, just like with all government institutions where different issues can arise that can lead to improper behavior. But outside of that, I think the structure he proposes would be unlikely to have that or as unlikely to have that type of problem as any other government institution.

[00:36:49:14 - 00:38:37:04]
Amith
 I think the concept though, the Foxguard and the henhouse, I don't buy into that. What I do think is true is that it could favor bigger players. So the bigger players locking in their advantage comment, it's not so much the Foxes regarding the henhouse and therefore, they're kind of self-dealing in terms of allowing models through that shouldn't be allowed through that kind of thing. But it's more of if we are pushing for more regulation that makes it harder for startups that don't have the kind of resources as the big players to be able to release models. And he does talk about this in his paper where he describes models of a certain size, which typically the earliest startups aren't gonna be playing yet in that space. But what if a startup that has very limited funding comes up with some kind of a novel breakthrough in model architecture that makes it possible for them to build a model that's more powerful than Fable today or whatever the Fable equivalent is, right? And at some future point, do they have to be subjected to the same regulatory process or not? So I think there's a lot of interesting questions still to be asked. I do think something has to happen. This is too powerful of a technology to just let it be the wild west and people do whatever they want. The only thing I'd say is you have to have global alignment and that's tough, right? So you have to get everyone in China aligned, everyone in Europe aligned because there is model development going on over there as well, and lots of other parts of the world. And then there might be certain countries that refuse to play balls. So there might be a leader out there that has a desire to say, you know what? We're gonna be the open and free AI place where you can do whatever you want. And model developers may choose to incorporate there and build models there. So what do you do in that case, right? So is this a UN thing? Is it something else? I don't know the answers to any of these questions, but I do think that in order for this to be useful, it has to be global.

[00:38:37:04 - 00:39:04:10]
Mallory
 Right, I agree with that. I think what's tough is that the people who are the deepest in AI and have the deepest understanding of this technology are probably employed at the great AI companies in the US. So I do kind of understand the fox guarding the henhouse sentiment, but then I agree with you, like something has to be done. And having the government pull a model off the market three days after it's released to check, like that was really messy. So that does sound at least like a more certain process.

[00:39:05:10 - 00:40:57:21]
Amith
 Yeah, and I think, you know, I don't think that the Chinese government wants bad models getting out there because it hurts them as much as anyone else, right? So their power system is not benefited from having rogue models in the wild in China or anywhere else. It's not helpful for them or anyone else that's in a position of power to have anybody out there have access to, you know, nuclear weapons grade power in this technology, right? It's probably much more powerful than the nuclear weapons in a lot of respects. So I think that there should be an opportunity to gain alignment. I don't know how you do that, but I think if you don't get China specifically involved, this goes nowhere. Because China is arguably right there with us. I don't think there's six months behind. I don't even think they're necessarily six days behind. The other thing we have to remember about China is they're operating with at least one hand tied behind their back from a hardware perspective. They have access to a generation behind or actually two generations behind in terms of the Nvidia chips they've been using. They've built their own, but reportedly those chips at the moment are not nearly as good as the current stuff that Nvidia has in production that the US model companies are training on. So, you know, that's pretty remarkable. Just as a side note that they're able to do that. And what it's actually teaching the Chinese model companies is how to do a really good job with constrained resources and constraints oftentimes are the mother of invention. We say that a lot in technology and a lot of other circles. So I think we're disadvantaging ourselves in a lot of ways here in the US with the way we're behaving about a lot of this stuff. I don't think any of that stuff ultimately has worked out in terms of export bans and stuff, but I'm not saying that things don't need to be done, that advantage companies in your own country, whatever your country is, but ultimately what we've been doing obviously hasn't slowed anything down in terms of the long-term lens. And so we slow ourselves down and China does not,

[00:40:58:24 - 00:41:03:10]
Amith
 the story is pretty much written. So I don't think anyone's gonna go for a policy that has that outcome.

[00:41:05:10 - 00:41:15:17]
Mallory
 Well, Amith, with the free models pouring out, governments improvising, the industry is begging for referees, if an association leader takes one thing from all of this episode, what do you think it should be?

[00:41:17:03 - 00:43:42:04]
Amith
 My number one piece of advice is always educate yourself. And so being here with us on the Sidecar Sync is one piece of that puzzle. I think it's really important that you learn about these topics. It's not so much that you need to be the technical expert, but you need to know how this stuff fits together. So when you talk about AI, it's not a monolithic single decision thing. It's a nuanced and complex thing. And it's worth learning, both because the absence of that knowledge could likely lead you to make poor choices with your strategy and your execution of the strategy. But also it's possible that if you don't educate yourself, that you're gonna really miss a window. You're gonna miss the opportunity to create dramatically different types of member value. Being able to realize that this curve that I'm talking about in terms of abundance of intelligence, whether it's the kind of intelligence that invents, new math and new physics or something like that, which we don't yet have, or if it's intelligence that can deliver dramatically better member service or new events or different kinds of products that would benefit your members, that intelligence is already here. But those things are things you have to know about. So my comment coming back to these models, the progression of these models, the speed, the intelligence, and ultimately what this means is cost is coming down is, you can afford this stuff. Maybe not this minute, but probably actually you can. But you certainly will be able to, probably faster than you can move, probably faster than you can plan and execute something. So if you were to look at something and say, hey, if we could only use a fable class model and process all of our content with it, we'd be able to do this really amazing thing that we hadn't even thought about until today. But if we did that, it would cost us a million bucks and we can't afford that or $10 million. Well, that's great. Think through the idea and just imagine if it wasn't $10 million, if it was $10 or free, right? And build the plan based on the assumption that it's going to become free or close to it. And then by the time you get done building that plan, it'll probably be true. Or a version of the technology that's close enough will be there. So that's happened literally in the last few years, over and over and over and over again, where people still don't seem to see that pattern. They still keep making choices bounded by the constraint of the current cost structure that they're evaluating. So that to me is the biggest takeaway in terms of strategic opportunity.

[00:43:44:04 - 00:44:02:12]
Mallory
 Powerful AI is getting cheaper, more open, and no longer the property of a handful of US companies and everybody from governments to the labs themselves are trying to scramble and figure out rules while the ground keeps moving. So for you, another takeaway isn't necessarily which model is on top this week, because that'll change again by next month,

[00:44:02:12 - 00:44:08:01]
 (Music Playing)

[00:44:18:19 - 00:44:35:18]
Mallory
 Thanks for tuning into the Sidecar Sync podcast. If you want to dive deeper into anything mentioned in this episode, please check out the links in our show notes. And if you're looking for more in-depth AI education for you, your entire team, or your members, head to sidecar.ai.

[00:44:35:18 - 00:44:38:24]
 (Music Playing)