Summary:
In this episode of Sidecar Sync, Amith Nagarajan and Mallory Mejias tackle the latest developments in AI with a fast-paced series of updates—from Claude overtaking ChatGPT in the voice mode battle, to Google's shockingly tiny Gemma 3 model, to the rocky rollout of GPT-5 and what it teaches us about innovation at scale. Then, the duo zooms out for a big-picture look at AI’s massive and growing energy demands. Can AI keep advancing without breaking the grid? Enter neuromorphic computing—a brain-inspired breakthrough that could power the next era of AI more efficiently. Hot dogs, ant colonies, and nuclear power all make appearances in this can't-miss conversation.
Timestamps:
🎉 Thank you to our sponsor
📅 Find out more digitalNow 2025 and register now:
https://digitalnow.sidecar.ai/
🤖 Join the AI Mastermind:
https://sidecar.ai/association-ai-mas...
🔎 Check out Sidecar's AI Learning Hub and get your Association AI Professional (AAiP) certification:
📕 Download ‘Ascend 2nd Edition: Unlocking the Power of AI for Associations’ for FREE
🛠 AI Tools and Resources Mentioned in This Episode:
Claude Voice ➡ https://www.anthropic.com/
ChatGPT Voice Mode ➡ https://openai.com/chatgpt
Google Gemma 3 ➡ https://ai.google.dev/gemma
OSS Models (OSS-20B, OSS-120B) ➡ https://huggingface.co/mistralai/Mixtral-8x7B
Mark Hersam’s TED Talk on Neuromorphic Computing ➡ https://www.youtube.com/watch?v=66b49ZrJzdw&t=3s
https://www.linkedin.com/company/sidecar-global
https://twitter.com/sidecarglobal
https://www.youtube.com/@SidecarSync
⚙️ Other Resources from Sidecar:
More about Your Hosts:
Amith Nagarajan is the Chairman of Blue Cypress 🔗 https://BlueCypress.io, a family of purpose-driven companies and proud practitioners of Conscious Capitalism. The Blue Cypress companies focus on helping associations, non-profits, and other purpose-driven organizations achieve long-term success. Amith is also an active early-stage investor in B2B SaaS companies. He’s had the good fortune of nearly three decades of success as an entrepreneur and enjoys helping others in their journey.
📣 Follow Amith on LinkedIn:
https://linkedin.com/amithnagarajan
Mallory Mejias is passionate about creating opportunities for association professionals to learn, grow, and better serve their members using artificial intelligence. She enjoys blending creativity and innovation to produce fresh, meaningful content for the association space.
📣 Follow Mallory on Linkedin:
https://linkedin.com/mallorymejias
🤖 Please note this transcript was generated using (you guessed it) AI, so please excuse any errors 🤖
[00:00:00] Amith: Welcome to the Sidecar Sync Podcast, your home for all things innovation, artificial intelligence, and associations.
[00:00:14] Greetings, everybody, and welcome to the Sidecar Sync Your Home for all of the content you can possibly imagine at the intersection of artificial intelligence and the world of associations. My name is Amith Nagarajan.
[00:00:27] Mallory: My name is Mallory Mejias.
[00:00:29] Amith: and we are your hosts. And, uh, we're up to no good once again over here at sidebar.
[00:00:35] Right, Mallory, we're working on the next edition of Ascend and you are taking a deep dive this week and kind of reviewing what's changed in the last 12 months since we last updated the book. Uh, what's going on so far with that?
[00:00:47] Mallory: You know, I am just working away on a Send third edition. Super exciting and a really great exercise to be able to look back on the past year or so, and also look ahead for kind of what's coming or what we think might be [00:01:00] coming in the next one to two years.
[00:01:02] I will say, going in, you and I both talked about how, oh, we don't know if that much of the book will have changed from second edition to third edition. I can tell you, Amme, I think a lot is going to have changed. We're gonna be talking about. Advanced reasoning models. We're gonna be talking about MCP, which we've talked about on the podcast.
[00:01:20] Right now. I'm kind of outlining our section on AI generated video, which we did mention briefly in the last edition. But there's just been so much progress since then, even with something as simple quote unquote as image generation, right? We had chat GPTs native image ability that can get words correctly.
[00:01:38] So I, I wanna stuff as much as I can into the book, but I'm trying to keep it. Simple, digestible. Um, so it's been a really fun exercise to look back at the past year and, and see what's changed for associations specifically.
[00:01:52] Amith: Makes a lot of sense. And you know, the retrospective that we go through and making a major update to Ascend, and for those of you not familiar with it, [00:02:00] is our totally free digital download and print book available that is specifically written for you in the association market.
[00:02:09] And it is all about how to think about ai. Obviously teaches you a lot about AI and also tells you a lot about how to think about adopting AI in your organization. And uh, you know, certain aspects of that book I think will probably stand the test of time, but we wrote that book in its first incarnation in 2023, um, around March, April is when we first released that.
[00:02:28] And, uh, our first started really working on it in depth and we released it, I think in June. Uh, and then we updated it again, a major update, basically a rewrite around this time in 2024. And here we go again. And on the one hand, uh, some of the concepts we talk about there do stand the test of time in terms of the general thought process around how to think about ai, how to create.
[00:02:48] Flexibility, how to create clarity, how to improve your culture, how to bring everybody along, how to think from a bottom up perspective, not just the top down perspective, meaning, uh, really inculcate, uh, into your team, [00:03:00] uh, a sense of wonder discovery. Curiosity, give them tools and set clear guidelines so that you can have as much.
[00:03:07] Bottom up innovation as you can possibly imagine, uh, well beyond what you can do top down. Those concepts I think will be relevant in five years and maybe in 50 years. Um, but you know, a lot what we talk about in the book is how to think about prompting and how to think about agent design and to Mallory's point video.
[00:03:23] And that's changing so crazy fast. So, you know, every week we do this and we talk about what's up right here, right now. And this is an opportunity to look back 12 months or so and, and make a major update. So really exciting stuff. We can't wait to bring that to you. Um. If you needed another good reason to come to Digital now 2025, which is in Chicago, November 2nd through fifth.
[00:03:44] You have yet another one, which is that we are committed to producing this new edition, the third edition of Ascend. Prior to that show. And if you are in person, you will receive a copy of the book at the event. Plus, it's at a pretty cool venue. The lows, um, downtown, very close to the river, [00:04:00] very close to the lake.
[00:04:01] Really cool brand new hotel. Uh, so can't encourage you enough to be at digital now, November 2nd through fifth. So Mallory, what do we have going on today?
[00:04:11] Mallory: We've got a lot going on today, Amit, aside from the podcast, I've got two doggos behind me playing, so if you guys hear any growling, I promise all is well on on my end.
[00:04:21] But we're gonna do something a little bit different today and I'll be curious to hear what you all think about it. So I wanna start with a rapid fire segment on three quick AI updates, and then we'll dive into a more critical issue that could fundamentally change AI's trajectory. First rapid fire.
[00:04:38] We're gonna be talking about Claude versus chat, GPT Voice modes. We'll be talking about Google's tiny but mighty Gemma three model and some of the backlash and reception around GPT five, which we just covered. Then we're gonna be talking about how AI's massive energy consumption is driving a revolution in computing that mimics the human brain.
[00:04:58] So first we've got [00:05:00] Claude versus chat, GPT Voice. You all know we've been huge fans of chat, GT's, voice mode for a long time, especially Amit, you use it constantly on walks, on drives basically everywhere. When Claude voice launched back in May, it had kind of, you know, this walkie talkie style where you press a button between turns.
[00:05:18] It didn't seem compelling enough to leave chat GT's seamless flow. But Amit, recently you posted on LinkedIn that. You're finally switching to Claude voice after you had some GPT five connection issues. So can you talk a little bit about that?
[00:05:31] Amith: Yeah, I have mixed emotions about it to be honest. Uh, you know, I, I switched to it simply because g PT five has been super buggy for me.
[00:05:39] Uh, just in, in. Pretty much in all aspects API usage, using it in the app, uh, using voice mode is basically not functional. And, uh, I sure hope OpenAI fixes this stuff up. I mean, they probably just have an insane amount of demand and I am empathetic to that. But at the same time. You know, they're the big player in the space.
[00:05:57] They need to do better than this. And I know that they're working on it, but, [00:06:00] uh, my, my viewpoint is, is that, you know, sometimes this happens because you're kind of forced to. So I was all excited. I was going for a walk the other evening and I was all excited. I was gonna talk to my AI buddy chat, JPT, all about some ideas that had, and I do this a lot for brainstorming.
[00:06:15] And, uh, so I get on Chap CPT and I try to connect and I try to connect and I try to connect. That didn't work so long story short, I even tried the four L models to try to downgrade myself and see if that work didn't work as well. In any event, I said, well, you know, I know Claude and I love Claude for using on the desktop.
[00:06:32] Uh, I use it for software development. I use it for writing. I use it for a lot of things. That's my go-to, um, on the computer and, uh, in general. In any event, I went to K club. I knew they had this capability and I had an incredible experience. That's what I wrote about on LinkedIn actually shortly after I got home from my walk because, um, you know, I was just kind of in my mode of using chat GPT because I got used to it just like we all do.
[00:06:55] And using Claude that evening, uh, really made me rethink what I'm [00:07:00] gonna do. And I didn't, I didn't quite mind the walkie talking nature of it, the pressing the button. I kind of got used to that pretty quickly. In fact, in the way it reduced a little bit of anxiety that I sometimes have with chat GPT voice where you kind of feel like.
[00:07:11] You're gonna get interrupted, and I'm a little bit long-winded as our listeners know. So it's, it's one of those things that chat chief tends to interrupt me and therefore, you know, I kind of like Claude only talking when I ask it to.
[00:07:23] Mallory: That is so funny. I can imagine you just going on this long intelligent rant and then voice mode being like, oh and oh, uh uh, you go in the back and forth thing, like on the podcast, uh, I tried out Claude voice as well.
[00:07:36] I thought the walkie talkie having to press the button every time you talk, it's a little bit clunky in my opinion. I kept forgetting, but once you get the hang of it, it's not so bad. I felt like the audio quality was also a little less impressive in Claude voice than than chat GBT. But um, I actually wanna skip around then.
[00:07:53] I was gonna talk about Google Gemma model next, but since we're talking about GPT five. I wanna discuss [00:08:00] briefly how the rollout has been messy. You just mentioned how it's been buggy for you. The same for me. Users are also saying it feels cold and clinical compared to GPT-4 oh's warmth. Some people joking, I don't know if it's completely a joke that they, they lost their friend overnight when OpenAI yank the older models without warning.
[00:08:18] We've also discussed the model router system that picks up different variants for different queries, which users don't like, or some of them don't like because they don't know what they're getting. Sam Altman has also admitted that they kind of botched the launch and brought GPT-4 O back as a legacy option for paid users after that backlash.
[00:08:38] So, Amit, talking about GPT five here, what, I guess, what is the lesson, what is the takeaway for associations with this perhaps botched launch?
[00:08:47] Amith: I think one of the lessons is you need to move pretty quickly in order to innovate and try new things, and sometimes you're gonna get it wrong. So I think being open-minded about that as an organization and being willing to take some risks.
[00:08:58] You know, you, you don't get a home [00:09:00] run if you never swing hard. So you have to go forward sometimes. And clearly that's what they've done. GPT five is a big release. It has lots of different things to it. I think it's an exciting step forward for the industry. I think they're gonna figure it out over the next 30 to 60 days probably.
[00:09:14] And then it'll be quickly, I don't know about quickly forgotten is what I was gonna say, but, uh, it'll be an an element of their learning as an organization and probably the industry's collective learning. They're at a scale that nobody else is at by, by an order of magnitude, right? They have 700 million weekly active users.
[00:09:29] They'll probably at a billion weekly active users by the end of the year by most projections. Uh, that's a different problem domain than Claude or, uh, anybody else in this space has right now. So again, that's why I'm empathetic towards it. But I think one of the things to learn is. Solving multiple different problems in parallel.
[00:09:45] So what they're trying to do is make the model smarter with GPT five's thinking mode, uh, with all the capabilities they touted in their live stream. A couple weeks ago at the same time, they introduced GGPT five nano, and I believe one of the reasons [00:10:00] users are having the problem they're having is many things users ask for are not very complex.
[00:10:05] So it's not necessarily a a sophisticated, complex reasoning problem. But when you get routed to the smallest possible model, which is what's happening, I think a lot of the time the response is the quality isn't quite as good. It's probably actually a step back from GPT-4 0.0 in terms of personality, tone, style, warmth, et cetera.
[00:10:23] Um, whereas the full GPT model is, is probably an order of magnitude better than 4.0. I really don't know, honestly, 'cause I haven't been able to test it that much due to the issues I mentioned.
[00:10:32] Mallory: And that I will point out to our audiences a lesson in Friction. You've got Amit here, avid, GPT, voice Mode user who is experiencing some bugs, a lot of friction using the product, and is now using Claude Voice.
[00:10:45] Do you think you'll go back to it or do you feel like you're gonna say,
[00:10:48] Amith: I'm not a, I'm not like a done forever kind of person. I'll kind of let myself get burned quite a few times by something before I learn. So it's partly because I wanna, you know, give people multiple shots. I appreciate that being on the other side of the fence as a [00:11:00] vendor.
[00:11:00] You know, when you, when you mess something up, people giving you some grace to quickly fix it, as long as you're super transparent about it and you're on top of it. I think that makes sense. We're all, we're all human. Even the AI is kind of human at times, right? So we all, we all mess up. So I, I don't think this is in any way going to be a long-term negative for open ai.
[00:11:20] I just think that it's something that, uh, they need to get past, they need to quickly address this, probably need to go out and like suck. The entirety of the compute supply that's coming online and available. And of course everyone's competing for that as well. So it's an interesting time. I think the only thing I'd say is, uh, expectation settings.
[00:11:37] So, you know, when they rolled out GPT five, obviously they had a bunch of test groups, you know, small numbers of people, um, but they, they might have considered like a larger beta, uh, with maybe, you know. I don't know, a fifth of, uh, or sorry, not a fifth, five times as many people, or 10 times as many people as they had in their early testing because I think they would've learned a lot of these things if they open it up.
[00:11:57] Um, part of the problem with the early testing group they had [00:12:00] is it's people like Professor Ethan Molik, who we talk about from time to time. A variety of other, obviously very knowledgeable power user type people and their patterns of use are gonna be different than the average end user consumer. Um, so, you know, I think that they could, could have done well by opening up testing a little bit more broadly.
[00:12:17] That's an old school playbook from the software industry. I mean, software companies oftentimes would have beta periods that were even public betas where people could just opt in or close betas where. You have to be invited, but either way, a lot of people would know about the beta for a period of time.
[00:12:30] Part of, of course, the pressure in this world is things are moving so incredibly fast and so competitive that I think a company like OpenAI or Anthropic or others is probably hesitant to do something that open. But I, I kind of sense that some of that will start happening in the future because of this.
[00:12:45] Mallory: I also doing some prep for this topic. I read that Sam Altman said they have better models. They just truly can't release them because they don't have the infrastructure, the compute to support it, which I think is really interesting and kind of ties into our main topic [00:13:00] of the day, but. Before we jump into that, I wanna talk about how Google just released Gemma three, an open model with a 270 million parameter variant.
[00:13:09] That's million with an M, making it one of the smallest language models available. Most small models these days that we cover particularly have a B in their name to represent billions of parameters. The latest open AI models we covered, OS, S one, 20 B, and 20 B are an example, but this tiny thing can run directly on phones and browsers.
[00:13:29] Supports 140 languages. Amit, what is your takeaway on the latest Google Open model? It's, it's a trend line. We're understanding, we're seeing all the time. What's your takeaway?
[00:13:40] Amith: I think what you just said is the key part. Uh, the trend line. And so what we're seeing is model compression. We've been talking about this for the entire time.
[00:13:47] We've been doing the sidecar sync almost two years now, and the essence of this is that we have the compression. Of knowledge and ultimately intelligence into smaller and smaller form [00:14:00] factors. And as you make something smaller, it's cheaper, it's more portable, it's um, faster as well. So Gemma three, 270 million parameter variant will run.
[00:14:11] Roughly an order of magnitude faster than a 2 billion parameter model, and roughly two orders of magnitude faster or a hundred x faster than a, uh, a 20 billion parameter model like OSS, uh, one 20. And that's all else. Equal architectures may, may differ. Many of the larger models that say, Hey, we're, you know, 500 billion parameters are MOE models or mixture of expert models, which is a, a fancy way of saying they actually had multiple different models built in and they automatically switch, which by the way is different than GPT five's router.
[00:14:38] Those are actually. Totally separate models, and there's a piece of software on top making a choice at first. Uh, whereas the, the mixture of experts have a router concept in them, but it's, it's, it's on a per token basis, essentially in its newly instant, very, very different idea. Uh, in any event, the, getting back to the question, um, if you can get 140 languages and a bit of the knowledge of humanity [00:15:00] into a little model like that, that you can put on, you know, the smallest possible phone drive, you could probably run it on an Apple watch.
[00:15:06] That's unbelievable. It just tells you how far we've come because this model is, I don't know what the benchmarks are saying because I haven't studied this one, but I'm guessing it's probably something like a GPT-3 0.5 class model, maybe even a GPT-4 turbo, the original, you know, scale GTT four, um, that model.
[00:15:24] It's probably comparable because that model's a year and a half old. So if this little tiny model is about as good as state-of-the-art AI from 18, 24 months ago, that's a pretty amazing feat. And when you put together lots and lots of inquiries to small models, you can do some pretty amazing things. So I.
[00:15:41] I'll give you an example from the world that I've been spending a lot of my time on for for months now, which is this Skip AI agent that we've talked about on the pod a number of times. Skip is essentially an expert at data analysis, programming and uh, talking to users. It's a conversational agent that can go throughout your enterprise [00:16:00] wide data catalog, extract data.
[00:16:02] Transform it and ultimately write code to present it to you in, in the form of interactive charts, graphs, dashboards, things like that. Essentially, it's an analytics solution. Uh, it's very, uh, conversational like chat, PT or Claude, but it lives in a secure platform. Now, this agent, um, what it does is it actually breaks down a problem into dozens, sometimes hundreds of small problems.
[00:16:24] So if you need to, for example, build a membership dashboard. There might be 20 or 30 different components in the dashboard. So Skip doesn't just do it all at once, skip will break it down, analyze the problem like a human would, or like a team of people would, and then ultimately breaks it down into lots of different component parts and then is able to spin up.
[00:16:43] Lots of instances of small models, which will generate first drafts of all of the different bits of code that are needed. Then use slightly larger models to test that. Slightly larger, larger models from there to compose it ultimately resulting in potentially hundreds of prompts being run. You can't do that if you need to use the [00:17:00] biggest possible model.
[00:17:01] Um, if you have these small models that are at a, they're not as smart, but they're really smart models. You can do amazing things. You just have to think creatively about it. So the trend line, getting back to your point. Is the key message. It means that AI is getting cheaper and cheaper approaching free, and the intelligence level keeps going up.
[00:17:19] Um, another way to put it is if you can get 90% of the power or 80% of the power of something truly revolutionary or pretty close to free, and it's very fast versus a hundred percent of the power for very, very high prices, and it's very slow or hard to get access to, most people are gonna pick the 80, 90% solution because it's amazing.
[00:17:38] Um, it's not like an airplane ride where you wanna say, Hey, hey, do I want 80 or 90% of the airplane ride? No, no, no, no. I really want a hundred percent of the airplane ride. I wanna, I wanna land, I want a frontier Airplane ride. Yeah, exactly. So I want, I want the best airplane. Um, in this case, it's, it's a, it's a totally different type of thing to, to analyze this on.
[00:17:57] These small models have unbelievable utility. [00:18:00]
[00:18:00] Mallory: All right. Let's move to our main topic for today, AI's energy crisis and the neuromorphic solution. So by 2027, AI is projected to consume a hundred trillion watt hours of electricity annually. That is as much as the entire country of Argentina. We're also talking about 200 billion gallons of water per year just for cooling data centers.
[00:18:22] Tech companies are literally considering nuclear reactors to power future AI infrastructure. The root cause is something called the Von Neumann bottleneck. The problem isn't actually AI itself, it's how computers have been built since the forties. The Von Neumann architecture separates memory and processing units and moving massive amounts of data between these two consumes enormous power, especially with today's data heavy AI workloads.
[00:18:49] Claude came up with this analogy for you all. Think of it like having your membership database in one building and your engagement tools in another city. All that shuttling back and forth [00:19:00] waste massive energy. Here's something that really puts this in perspective. Back in 1997, IBM's Deep Blue became the first supercomputer to beat a world chess champion.
[00:19:10] Gary Kasparov, one of the greatest chess players in history, but here's the energy comparison that will blow your mind. Kasparov could play that championship level chess using the energy equivalent of a single hot dog. Deep blue. The supercomputer needed the equivalent of 250,000 hot dogs worth of energy for the same game.
[00:19:31] The human brain operates on about 20 watts less than a light bulb while AI supercomputers need megawatts. The brain secret, not so secret is that it co-locate memory and processing. No shuttling the data back and forth. Everything happens in the same place dynamically and efficiently. So scientists are now developing neuromorphic computing chips that mimic brain architecture where memory and processing occur in the same place using [00:20:00] nanotechnology.
[00:20:00] These devices can be reconfigured with electric fields just like neurons and synapses adapt in our brains. Is the chair of Material Science and Engineering at Northwestern, and his lab has already demonstrated nano electronic devices capable of machine learning with a hundred times less power than traditional computers.
[00:20:20] The topic today too, I should mention was inspired by Mark her's Ted talk that we will link in the show notes. Amif, kind of just an observation on this point before we get into questions. I think it's so fascinating that technology is mirroring. Biology and that when we don't have a solution to a problem, we can kind of look at the human body or look at nature, for example.
[00:20:42] Uh, and I did a little research on this with Claude just because I was going into kind of a nerdy deep dive, but obviously we have neural networks. So basically the entire AI field copies how brain neurons connect and fire, I didn't know this, but Microsoft, uh, is looking at storing data in. Synthetic DNA, which [00:21:00] is nuts.
[00:21:01] Uh, the Amazon warehouse routing mimics how ant spine food. Anyway, those are just a couple examples I wanted to share with you all. I think it's really neat to think about how a lot of the answers are around us sometimes, but what's your takeaway on this? It's a lot to unpack.
[00:21:16] Amith: I mean, it makes sense to learn from every source of learning you can get.
[00:21:19] And so in the natural world, I mean the, the natural world has had a little bit of a head start over our species, specifically just a little bit part of it. But we're, you know, we're discovering a little bit of, a little bit more every day, which is part of what's exciting is I think AI's gonna help us find more of these discoveries.
[00:21:34] So, you know what you mentioned about the co-location of memory and processing and the bottleneck and the energy and efficiency. That's a big, big part of the problem. Um, there's a lot we can learn from the brain, and in fact, actually as another call out to digital now, twenty twenty five, one of the keynotes I'm excited about this year is a talk by a medical doctor and an AI expert teaming up to talk about the similarities and the differences between the way the human brain and the way [00:22:00] AI brains work.
[00:22:00] So you gotta check that out. That's gonna be an amazing session. Um. I'm sure we get into a lot of these concepts. So, you know, my general viewpoint is that there's a lot of opportunity. We're insanely inefficient in the way we do ai. Part of it's what you're describing at the compute level where you have compute and memory separated essentially by the Grand Canyon or by the Pacific Ocean is probably a better analog in terms of distance to cover.
[00:22:25] Um, you know, so it's, it. Very, very, very difficult, uh, to address that with the classical computing architecture we have, lots of people are working on lots of different innovations. Um, that's actually part of what makes the Grok Language Processing unit, uh, and the Cereus wafer scale, uh, uh, technology so compelling is because they're able to reduce that bottleneck dramatically by.
[00:22:47] Not co-locating memory in the exact way you're describing with neuromorphic designs, but somewhat similar to that in some ways, they, they're able to reduce a lot of those memory bottlenecks. And so, um, there's a lot of innovation happening in this area and that's really, [00:23:00] really exciting. So anything we can do to become more efficient, that's better.
[00:23:03] The other thing that's really, really inefficient about current AI is that we have, uh, what's called a quadratic problem in the current transformer architecture, which we've talked about a little bit on this pod in the past. But as a quick refresher, um, the bigger the context window, meaning the amount of.
[00:23:17] Of information that's being processed, whether it's, uh, text tokens or, or pixels or, uh, segments of video. Um, essentially what's happening right now through the attention mechanism in this transformer architecture is, is you're comparing every token against every other token in order to predict the next token.
[00:23:32] And so that mechanism, there's lots and lots of optimizations that have been happening since the original transformer in 2017. But ultimately we still have some flavor of the quadratic problem, which is really ridiculously inefficient. Um, and so we are solving that with massive amounts of compute, massive amounts of energy.
[00:23:50] There's going to be algorithmic progress. There already has been a lot of algorithmic progress that is. Advancing the field in a way that's gonna make it more and more efficient. Part of what made Deep Seek [00:24:00] back from January such a big deal is that they were able to do, you know, at the time oh one and then almost oh three level processing performance with radically less training and in a more efficient way.
[00:24:10] And then the same thing was true for the QEK two model that came out about a month and a half ago. It had GPT-4 one level coding ability with far less, uh, requirements. So. We are innovating really fast, and that's exciting. I think coming back to the topic for today in terms of hardware, the more general concept of neuromorphic designs or.
[00:24:30] Designs essentially that mimic, uh, you know, the, the biology in this case, the, the, the neuro biology of the brain are super interesting and, you know, the, the, the one hotdog versus a quarter million hot dogs. Yeah. It's kind of what it is. You know, we've got a pretty efficient power plant, you know, running, uh, for, for us in terms of powering our brain and our brains can do some pretty amazing things.
[00:24:51] Mallory: And as a note too, uh, the hotdog reference I got from Mark Ham's Ted Talk that I mentioned, and I believe he did the TED Talk in Chicago. So he was talking about Wrigley [00:25:00] Stadium. You know, what else is in Chicago is digital now. So maybe we'll have to do some, some hotdog analogies there.
[00:25:06] Amith: For sure.
[00:25:08] Mallory: Amme, what do you think about, I guess where I'm stuck here, I will say in the past two years, I know we've talked about infrastructure and energy demands.
[00:25:17] I think even for me. More recently, I'm starting to understand how serious this is. Like we've talked about it kind of abstract, but the past probably six months to a year, I've realized, wow, this is truly a bottleneck and something will have to change. What would you say to people that, I don't know, feel bad is the right phrase, but feel like they should wait before implementing AI organizationally, or for their members until we have a more efficient way to use it?
[00:25:48] Amith: The question is, is what's, what's the threshold that you need to be at in order to consider it reasonable enough, and how does it compare to your existing energy use in other aspects of your life? You know, when I talk about, for example, the use of [00:26:00] Google, every single Google search pretty much comes back with a AI search results at this point.
[00:26:04] So, and Google is by far the biggest inference provider in the world through Google search, because they're running, I think like 10 x as much inference as any anybody else in the world, including open ai. Because they run Google search on top of this AI layer. And that was done because quite frankly, the demand is speaking for itself.
[00:26:21] Most people are going to AI to look things up than, than they are going to classical Google search. So Google had to change their game. And so you are basically using AI already now. How heavy do you go into, how big do you go into it? Are you concerned about the, the energy consumption, the water consumption?
[00:26:37] Uh, I am. Um, yet I also feel that, um, first of all. Disadvantaging yourself, first of all, like if you're doing good work to advance some cause in your field, and if you're disadvantaging yourself in your field because you're not using ai, that's not good. The other thing is, uh, the way we're going to solve a lot of our scientific problems and climate is a branch of science that needs as much [00:27:00] energy as we can put towards it is through.
[00:27:02] That's really the chance that we have to resolve this. So I think that, um, it's a double-edged sword. No doubt. Most new technologies are, this one is one that I think has this potential of giving us new solutions. Um, I also think that the additional demand is going to force us to be a lot smarter about the way we think about regulation around energy, thinking about nuclear, you mentioned that earlier, that uh, you know, AI companies are not just thinking about it.
[00:27:27] I think they're pretty far along and, you know, bringing back. Decommission nuclear plants as well as building new ones, and then getting into alternate forms of more modern nuclear technology like SMRs, which stand for small modular reactors, which are essentially like 18 wheeler sized. Containers that can power a thousand homes.
[00:27:48] Um, and these things are designed in, you know, with modern physics and modern understanding of material science and are done in a way where they're kind of inherently safe or dramatically safer than any other [00:28:00] preceding nuclear technology, both because of the physics in terms of how they cool. Um, they have passive cooling essentially, which doesn't require, uh, generators to pump water through them.
[00:28:09] That's part of it. And the other part of it is, is the materials that are used while still radioactive are not capable of being reprocessed into weapons grade material. So you solve two of the biggest issues with nuclear technology with the more modern approach. But the United States specifically has been pretty backwards about our mindset about nuclear for some time.
[00:28:27] Of course, that's my opinion, but. The reason I say that is, you know, many other countries, including some very climate oriented cul uh, countries like France and others in Europe have gone big on nuclear in with, in some, with classical nuclear technology and some, with some of the, the newer technologies I'm describing.
[00:28:43] Uh, so. I think we, and, and Nuclear, the reason it's such a big conversation is because it is not carbon emitting. It's basically something that, uh, has much more, uh, much more of the profile that you'd want in order to be, uh, climate oriented and, and environmentally [00:29:00] sensitive. So we can solve some of the bigger nuclear questions that, and that's not even talking about fusion, right?
[00:29:05] Fusion would be the one if we can figure that out, um, for any kind of fixed generation requirement. Pretty much solve energy if you can solve fusion. And that's, you know, that's on the radar. But I don't think there's anything around the corner in terms of commercialization of that. Um, but with SMRs and other forms of advanced nuclear technology over the next five, 10 years, we can definitely solve a big part of this problem if we're willing to do that.
[00:29:25] Mallory: So do you think it's safe to say, based on what I referenced from Sam Altman earlier, that we have way more advanced AI out there, but it's just not released to the public because we can't support it with the energy demands.
[00:29:38] Amith: First of all, I, I think there's a chance that he's got more advanced ai. I also think there's a chance he doesn't.
[00:29:44] Ah, okay. I dunno that he, I mean, I, I, I suspect he probably does. Um, but both because of alignment and safety and also because of the practicality of deploying what's presumably larger models and even more compute intense models, there are some constraints that aren't about, um. [00:30:00] You know, the, the most immediate demand.
[00:30:02] Um, I also think that in the world we live in today with the competition very, very strong from lots of major players, uh, not just the ones that are heavily funded to the tunes of tens of billions of dollars, but these startups in China that are doing unbelievable work. There's lots of people out there that are releasing great models.
[00:30:19] And so OpenAI has to have kind of the next thing out there. But my bottom line on it coming back to energy is, um, I think there's the energy constraint. I also think there's the manufacturing supply chain constraints that are equally problematic, um, because there's just not enough compute capacity. And to have the compute online, you have to have construction.
[00:30:35] To have construction, you have to have, you know, cement, you have to have a lot of labor. You have to have an environment where you can bring in materials from anywhere in the world in a cost efficient way. On and on and on. Of course, energy powers, all of that. But I think there's a lot of complexity to this, uh, to make, to just say, Hey, we wanna support 2 billion people in chat, GPT.
[00:30:54] This is gonna take a lot of work from a lot of different industries.
[00:30:57] Mallory: Mm-hmm. Do you think AI [00:31:00] will solve its own supply chain issues and, and energy problems? I
[00:31:05] Amith: think AI plus a lot of human labor will solve a lot of these problems. Um, but I think AI is gonna be a big part of it because the, the, the scale of these problems is, I think it's not necessarily beyond any of our capacities.
[00:31:18] Just the number of genius level people that are out there who can solve these novel problems is very limited. And there's more of them today than there were a hundred years ago. Because our population is larger and education's more broadly available and you know, people have healthier life lifespans.
[00:31:31] So really smart people who do this kind of crazy work into X and stuff like that, they live longer so they can contribute more. All those are exciting things, but those are all linear scale solutions to what is an exponential problem. You know, it takes us 25 plus years to grow a human from zero to the point where they're.
[00:31:48] Kind of useful in the workforce, right? And, and contributing. And that's a long lead time. That's a really long lead time. That's 50 doublings in AI power. If we stay on the same pace that we're on now, which is basically inconceivable in [00:32:00] terms of how big of a number that is, uh, both in terms of power, but also in terms of, of what the requirements are to to fuel it.
[00:32:06] So, you know, I think that AI coming online strong where we have the equivalent of hundreds of billions of PhD level people working on this problem, yeah, you're gonna solve a lot of issues that we cannot solve as a society. We just don't have enough of enough kind of intellectual horsepower available to solve these problems.
[00:32:25] Mallory: One takeaway a myth from this episode. I know we covered a few things, rapid fire, and then we talked about the nanotech, which is quite interesting. What do you think is the takeaway here?
[00:32:36] Amith: To me, uh, the thing that you have to remember is there's always a kind of order of magnitude shift waiting around the corner.
[00:32:45] So this idea of neuromorphic computing in general, and then specifically the neurobiology of the brain and the structure, we're only talking about one aspect of it that this particular design is trying to solve for, which, by the way, this is far from commercialization. It's in the lab right now in a, in an academic [00:33:00] environment.
[00:33:00] But, um, we're moving things through those pipelines faster. If that were to happen, you have like a hundred x improvement in efficiency, which also likely means approximately the same improvement in compute power, which is unbelievable. Um, and this is just one particular dimension of the advantages of our biology.
[00:33:18] It doesn't talk about things like neuroplasticity, which is the ability for our neurons to be able to kind of strengthen, weaken, sever, and regrow new synapses between neurons, which is not something that AI models can do right now. There's lots of algorithmic research happening in terms of. You know, uh, different, there's different approaches to this, but the basic idea of mimicking that in the model where the model can continue to grow and kind of adjust its weight, which is theoretically very powerful, but also potentially very dangerous because, you know, all the things we do for AI safety and evals are based upon the idea of a model being fixed in time once it's trained.
[00:33:51] Um, so there's, there's a lot more. Um, and so I would say to people. Don't be, don't be rooted in the idea of what you can do [00:34:00] now. So obviously have to start somewhere. So learn what's going on now because that's how you prime your brain to understand this, this new world we're entering into. But don't think about what you can and can't do in your business and with your strategy or with your career even based on what the capabilities are today.
[00:34:19] You know, like we started this episode, we've updated Ascend. This is the third edition coming out this fall. It's radically different than it was even two years ago. And we can, we do this stuff for a living, but we could not have predicted what's happening right now, two years ago. And we say that very openly to people.
[00:34:36] So the thing you have to do is get really good at recognizing what the worthy. But currently unsolvable problems are, what do you mean by worthy Is that they're, they're important. They're things that if you could solve problem X, it would mean, uh, a phase change, a step change in your business, your ability to deliver value, your ability to solve problems in your field or in your profession.
[00:34:58] And there's a lot of those. There's [00:35:00] lots of these so-called unsolvable problems. Um. Those are the ones that I get excited about. So any problem that someone says we can't solve, that, that's not solvable, that defies the laws of physics is what they'll say, which is generally never true. But, um, you know, these unsolvable problems are super exciting because with what's happened with ai, actually they're likely to be solvable and maybe even solved in 18, 24, 36 months.
[00:35:23] And if your strategic planning horizon is two years, three years, maybe even longer in some cases, if you're really ambitious. Well, you should be thinking really hard about what's going to like and change and plan for the AI that will be here in a year to the greatest extent you can. I don't know if you can do it for two years, but say six to 12 months from now, there's certain things that I'm pretty sure are gonna be in our hands, and you should build your business model around that, not around today's capabilities.
[00:35:51] That's my takeaway.
[00:35:52] Mallory: Exponential leap is just around the corner, and I feel like digital now is a great place to go to start looking ahead. [00:36:00] I hope you all will join us. We'll have some hot dogs. We'll see how much energy we need to get through those few days. Everybody. Thanks for tuning into today's episode and we will see you all next week.
[00:36:11] Amith: Thanks for tuning into the Sidecar Sync Podcast. If you want to dive deeper into anything mentioned in this episode, please check out the links in our show notes. And if you're looking for more in depth AI education for you, your entire team, or your members, head to sidecar.ai.