Interview w/ Alice Locatelli | Senior Director of Digital Strategy, Society of Actuaries [Sidecar Sync Podcast Episode 09]

Show Notes

In this episode, Alice Locatelli from the Society of Actuaries shares insights on integrating AI into digital user experiences. She discusses the importance of user research and its role in enhancing AI applications, highlighting a 20% increase in user engagement from personalizing website experiences. The conversation also explores AI's future in professional development and its strategic implementation in organizations, providing valuable perspectives from an association actively leveraging AI.

• Let us know what you think about the podcast. Drop your questions or comments in the Sidecar community: https://community.sidecarglobal.com/c/sidecar-sync/ 
• Join the AI Learning Hub for Associations: https://sidecarglobal.com/bootcamp 
• Download Ascend: Unlocking the Power of AI for Associations: https://sidecarglobal.com/AI 
• Join the CEO AI Mastermind Group: https://sidecarglobal.com/association-ceo-mastermind-2024/ 

Thanks to this episode’s sponsors! 

• AI Learning Hub for Associations: https://sidecarglobal.com/bootcamp 

Tools/Experiments mentioned:  

• Hotjar: https://www.hotjar.com/
• Hugging Face: https://huggingface.co/
• rasa.io: https://rasa.io/
• Society of Actuaries website: https://www.soa.org/

Social:  

• Follow Sidecar on LinkedIn: https://www.linkedin.com/company/sidecar-global 
• Alice: https://www.linkedin.com/in/alicelocatelli/
• Amith: https://www.linkedin.com/in/amithnagarajan/ 
• Mallory: https://www.linkedin.com/in/mallorymejias/ 

Welcome to Sidecar Sync, your weekly dose of innovation. If you're looking for the latest news, insights, and developments in the association world, especially those driven by artificial intelligence, you're in the right place. We cut through the noise to bring you the most relevant updates, with a keen focus on how AI and other emerging technologies are shaping the future.

No fluff, just facts and informed discussions. I'm Amith Nagarajan, chairman of Blue Cypress, and I'm your host.[00:01:00]

Hello everyone. And welcome to episode nine of the Sidecar Sync. My name is Mallory Mejias. I'm the manager over here at Sidecar. And we're excited to bring you a special edition episode today. But before we get into that, I want to do a quick sponsor shoutout.

Today our sponsor is the AI Learning Hub for associations and non-profits, if you aren't familiar, the AI Learning Hub offers flexible on demand lessons. So you can learn AI at any time, whenever it fits into your busy schedule. Not only that, but we consistently add new lessons to the Hub based on the latest AI advancement. So you can be sure that you're keeping up-to-date with what's going on in the AI space. You also get access to weekly live office hours with AI experts.

So you can ask them all your questions. And finally, maybe the best part you get access to a vibrant community of fellow AI enthusiasts so you can connect on your AI journey, share your challenges and learn and grow together.

And one quick note, I don't think we've mentioned it on this podcast before, but we also have group pricing available.

So if you want to get your [00:02:00] whole staff on board for the AI Learning Hub, you can do that for one flat rate, you can get more information on this and the Learning Hub itself at sidecarglobal.com/bootcamp.

All right. So what's so special about today's episode? Well, today's episode is special because we have our first special guest ever. Today, Amith and I are interviewing Alice Locatelli, who is the Senior Director of Digital Strategy at Society of Actuaries. We saw her and her colleague, Ryan give a session at digital now, 2023, this year about the innovative work that they're doing at SOA.

And we knew in that moment that we wanted to bring her as a guest on the podcast. We have a great conversation about the AI projects that they have going on at SOA Alice's tips, tricks and advice for infusing AI into your own organization, chats about some of the challenges that they faced. It's a really good one. So get ready.

I'm going to read Alice's bio right now, and then we'll go straight into the interview.

 Alice Locatelli is an executive champion of new technologies. Drawing from [00:03:00] experience leading digital, IT, operations and product. Her passion matches up early adoption with organizational need creating strategic advantage. Currently the executive driver for digital strategy at the society of actuaries, she was previously head of technology at an international association in the legal space. Before the association world, she was COO of a blockchain analysis, SaaS company. Led product at a cutting edge social sector, startup and helped organizations to drive strategy through data at a social impact consulting firm. And alumni of Stanford, U Chicago and Emory. Alice is a beyond barriers coach, a fast track mentor and advises organizations developing cutting edge AI solutions. Without further ado, here's the interview.

Mallory Mejias: Alice, thank you so much for joining us today. I was hoping you could give listeners a little bit of information on your background and how you came to work for Society of Actuaries.

Alice Locatelli: Yeah, absolutely. I'd love to. So I think of myself as an executive champion of new technologies. What I really love [00:04:00] to do is match early adoption with organizational need to create strategic advantage for organizations. So, I previously worked at the Museum of Science and Industry, where we worked with a startup to build a cutting edge ticketing system that is really taking the cultural arts space by storm. And there I met a Society of Actuaries, current CFO Rose Faley, who brought me into society of actuaries to do similar types of work, really tackling the hardest problem that we have at society of actuaries, which is our digital user experience.

So that's what I've been working towards doing over the course of the last few years.

Mallory Mejias: Absolutely. We brought you on this podcast because we were really inspired by your session at digitalNow. I was hoping you could give listeners kind of an overview of that session and the neat work that you all are doing with AI at SOA.

Alice Locatelli: Yeah, yeah, I'd love to. What we really talked a lot about there was our digital user journey and how we're making changes in that journey for our users [00:05:00] and infusing AI along the way. I start out by calling it a digital user journey because the user part is much more important to me than the AI part.

Really focusing on our users and understanding who they are, what they need and how we can give that to them in an easy way that makes sense to them. Our real goal is that their user experience feels intuitive and doesn't get in the way of the really important work that they're doing either going through our exams process or as members and professional actuaries getting access to the kinds of information that they need. So we've spent the last couple of years really focused on that journey a lot of it within our website. And I'm sure we'll talk about details on that, but a lot of it within our website and figuring out the best ways to get them the kinds of experiences that they need and in many cases, AI is part of that answer.

Mallory Mejias: You talked a bit in your session as well about user research methods. how did those insights shape [00:06:00] your AI journey?

Alice Locatelli: Great question. Our user research is absolutely core to what we are building. It is really hard to build a user journey without knowing what users want and need and who they are. So we did a really big research effort, starting with qualitative feedback. We ask users a lot of questions about how they use the website, why they come to the website in the first place, how they use all of our other digital assets, apps and emails and all of the other things that they engage with us in. To really understand, in the words of users, what they're doing, what they want to be doing, how it's working. One of my favorite parts of the qualitative research was to give people a task, like, sign up for an exam and watch them try to do that on our website. It is incredibly insightful to see. What things users click on that you wouldn't necessarily expect, but that can make sense to them.

So [00:07:00] especially for anybody out there who might be listening and doesn't have a big research budget or a big AI budget, finding a couple of users and asking them to perform some tasks that you think seem fairly easy on your website can be a really insightful way to understand some low hanging fruit that you can tackle to make things easier for them.

We followed on to the qualitative research with quantitative research fielding a survey to really understand what priorities users put on the different types of changes we thought we needed to make to understand not just from a few users in their very detailed words, understand more and mass what, what matters to a lot of people across the organization or across our engagement.

And in addition to the qual and quant research, we also implemented a few tools. My favorite one is Hotjar, which records sessions on the website and gives you a lot of statistics about what people are clicking. Within Hotjar, it also shows rage clicks and things like that. U turns, some [00:08:00] fun terms that can be really, really insightful to see which pages you're, you have that are really frustrating to users.

One of my favorite ways of thinking about this is progress over perfection. You don't have to roll out the absolute best website on day one, but you can make it better tomorrow than it was today. So looking for that low hanging fruit or those experiences that you can impact today through that research is an incredibly helpful way to approach it and more palatable, especially if you don't have a big budget or a big team to do this.

Amith Nagarajan: Alice, I can see why we get along so well because progress over perfection is actually one of our core values at Blue Cypress.

Mallory Mejias: Yep.

Amith Nagarajan: And you know, we live by that. It's so important to have that mindset and also fairly uncommon, I think, in the association market because there is such a desire to not mess up essentially.

So culturally, that's awesome that you guys have that perspective. I'm glad you mentioned Hotjar. I was going to suggest that as well because it's a tool that you actually can [00:09:00] use for free initially, and it has some fairly inexpensive tier. So even if you're not an organization that has the experience or the resources of an org the size of S.O.A. you can put hot jar on your website literally a few clicks, and you can, like, literally watch recordings of what people do on your website, and you can visualize that painful friction that we talk about. It is not a theoretical thing. You will see it right in your face.

Alice, I have a question for you. So when you think about user research, you know, one of the things that you tend to see is a couple different schools of thought or camps, one which is very driven by this pragmatic type of approach to say, hey, let's figure out what the user thinks they want and go build that. And then there's more of the Steve Jobs camp of like, well, I need to have the vision for what they should want, even if they don't yet know it.

How do you guys think about those two camps, and do you strike a balance between the two? Do you find yourself leaning in one direction or the other? Talk to us a little bit about that, please.

Alice Locatelli: Yeah, we definitely strike a balance between the two. For me, it starts with understanding what users [00:10:00] think they want and what they're asking for. And in many cases, that aligns with what we would want to give them. But there's the, the. Proverb that Henry Ford, if he had done what users asked for, he would have given them a faster horse instead of building a car.

And we do think about that as well. So, I think if, if our users had asked us for what they wanted, and we didn't apply that type of thinking to it, we would not have necessarily ended up with a personalized website the way we have it. Users didn't come to us and say, I know I need to see something different than the guy sitting at the desk next to me.

They just told us what they really needed and what they were looking for. And it was up to us to apply the ingenuity to say, wow, at least in our organization, and I think this is not true in every association, in our organization we have some very different types of visitors coming to our website. The example that I gave at digitalNow is we have candidates who are going through their sometimes 8 years worth of exams. And during that time, the [00:11:00] main thing they really care about is registering for exams, studying for exams, understanding exams, showing up to exams. Their world is all about exams. Once they've finished that 8 year process, one of our actual quotes from our digital user research was we never want to see anything about exams ever again.

So, we immediately identified that there are these really two very different user groups within our website customer base and recognize that we needed to apply that differentiation. No user came to us and said, I'm in the exam process, but later I don't want to see that. We had to recognize across the, the feedback that they were giving us, that this was something that was actually important to our users is to really see what they need to see.

Similarly, for people who are new to the actuarial profession, they need really, really different content than someone who's been an FSA for a lot of years. So really understanding the nuances within the user feedback and not just [00:12:00] taking exactly what they're asking for but rooting what you're doing in what they're asking for and then understanding what is the best way to give that to them is how we think about it.

Amith Nagarajan: I just want to drill a little bit deeper into that and the progress of a perfection comment, because I think what you're describing right now is often where people sometimes get stuck because they go too far down the rabbit hole of the different personas and try to go multidimensional and they, they basically end up getting stuck there because they try to create 8 or 15 or you know, 2000 different profiles, whereas, you know, what you described at the most fundamental level is there's two different, you know, major profiles. There's people who are candidates and people that are in the profession post exam process, and they have a fundamentally different outlook on what they're looking for.

And so the comment I would make there is like for associations thinking about how do you get started and just make a little bit of progress. Most associations, in my experience, have a similar dynamic where there might not be student to professional member or something like that, but it might be some other similar concept where there's just a fundamental divide somewhere in the profession, [00:13:00] and it might be time based.

It could be something else. It could be geography, international versus domestic, for example, and just focus on one thing and focus on that one simple thing to provide a slightly personalized experience. It's not truly one to one, but it's better than not doing it at all. And that's where a lot of people in my experience get stuck because they go for that ultimate solution rather than going for something that's a significant improvement over where they are typically.

Alice Locatelli: Yeah, I think that's great. And we are in the middle of that. We have a lot of different user groups that come to our website. Some examples of really critical user groups for us are employers. Employers often are providing and sometimes paying for exams and or professional development for our members and candidates.

They are an incredibly important group to us. And we are now doing work for them to make sure that they have the personalized experience that they need as well. We also have not just precandidate students, but also their parents often in Asian cultures. The parents are the ones looking at our website to [00:14:00] understand what an actuary is.

And we need to tailor content differently for them as well. So there are a lot of user groups that we could have started with staff is another one that use the website differently than our than our members. There are a lot that we really could have put into the forefront of this. So we really aimed for that 80/20 rule and said 80 percent of the people visiting our website probably are either candidates members or pre candidates people who want to learn about the profession.

If they're not logged in, we're going to assume that they are a precandidate for now, and we'll get more nuanced about those other audiences as we go, but you're right, it is progress over perfection. The website is not perfect for everyone who lands there today, but it is much better for the 80 percent of people and we can continue to iterate and work towards a better solution for the 20 percent of people.

Mallory Mejias: I have kind of a controversial question, Alice. I'm interested to hear your take on it. In a previous podcast episode, we talked about the idea of AI providing like the [00:15:00] first round of feedback. So maybe providing it with a screenshot of a webpage or a login process, and then asking AI to provide that first round of feedback.

I think this could be especially useful for an organization with limited budget or limited staff to conduct user research.

What do you think about that?

Alice Locatelli: Yeah, great question. I first and foremost love user feedback. So if you can get user feedback to me, that is worth a lot more than any AI feedback that you're going to get on your website if you're not far down the path yet, but we definitely look at some of these AI tools and really try to find either user feedback or AI feedback anywhere.

There's a cheeky website roaster GPT out there that somebody created where you can put in your website and it'll give you a scathing review of it, no matter how good your website is. That can actually be a little bit helpful in some ways. There are also some other tools out there. We haven't used them for SOA.org, but things like UX sniff and those kinds of things. I'd say the most useful thing [00:16:00] might be using some of the accessibility tools that are out there. Not necessarily AI specific, but I think a lot of websites that haven't spent a ton of time or energy on user feedback often are missing some of those accessibility traits.

So if you're looking for a tool to use, I'd probably start more in that direction. But that said, I love any feedback that I can get on the website and I really try to apply a good filter to it and say, you know, this is maybe at 10 percent of what I might wait a user actually using this at, but I think any feedback that you can get as good because really, what you want to do is look for those blind spots.

You're in your website day in and day out. Your team knows your website knows what's intended by it and you see it differently than others do so any lens that you can get to that, I think is worth applying.

Mallory Mejias: So it sounds like so far based on our conversation that most of your AI projects, I don't know if that's fair to say, have been focused on the website. So it seems like that [00:17:00] first step was the user feedback, user research period. And then what were your next steps? What were the actions that you took after that?

Alice Locatelli: Yes, so I speak mostly to the website side of things because that's been my side of the world, but I don't want to discount that across the organization there are a lot of other initiatives going on with AI in our education department and our research department internationally, engagement, really across the board.

We are looking at what are the right problems to solve for us. And can AI can do that better than not AI so I'm speaking to the website side of it because that's what I've been implementing. But we really do look across the organization. So, in terms of what steps we used to implement it on the website, we really started with this user research and then leaned into figuring out that 80/20.

We, technically speaking, we went into visitor groups and designated visitor groups to determine [00:18:00] those groups that we talked about candidates, members and give them that different experience. And then from there, we started adding small touches in things like, when you're going to register for an exam, seeing register if everybody sees the register button, it doesn't feel as personalized. But if you're already registered for that exam showing you're registered, do you want to check your schedule? Or do you need day of information or that kind of thing? So some of those small touches, I think, make a really big difference with AI and they're not very risky to do because it's pretty clear where the data is coming from and what's happening, but it can help you gather data on how people are interacting with the website.

So, for example, for us, as we change that button to register, or you've already registered, we can really see, are people clicking that button more now that it's personalized for them? And is that helping them giving recommended content? Those kinds of things are other pieces that can really help us move forward and see, do [00:19:00] people actually click on the recommended content or do they go and click on other navigation items to find what they're looking for?

Are we giving them good content?

Amith Nagarajan: A follow up question that I have, Alice is regarding personalization. So I know your focus has been on the website, at least in this conversation, we're heavily focused there on that project, but other modalities of content delivery, be it through, you know, verbal means through podcasts, perhaps I know you guys run an AI driven newsletter, like through these other modalities.

Do you tie personalization in, do you try to unify it in some ways so that if you learn, for example, the that I am in the exam taking phase of my career. Do you then tie that to your other modalities is what I'm trying to ask.

Alice Locatelli: Yeah, great question. The, the AI driven newsletter is extremely helpful for that. So we use rasa.io for our for our newsletter and it is really, really great. We've had great results for our candidate connect newsletter. We get the right kind of content for our users showing up and they really love it.

We have not done as much on the [00:20:00] podcast side yet, but I, I do have in mind in the future as we continue our journey of if I think about my crawl walk run as we continue our journey, I think running could look like when you pull up a podcast. The summary of the podcast is generated for you and for who you are.

So if you, if you're going to look at a predictive analytics podcast, we might be able to say you're early in your candidacy. You're probably studying for a test on this or you're later and you've been using predictive analytics for a while. And so let's give you a more technical summary. So what I would love to eventually get to is where it makes sense across our website to have some core content that we use, and then for individuals that we have the right information for generate better summaries of content or more relevant information across the website.

Amith Nagarajan: You know, I think in 2024, we're going to start seeing some real innovation around audio as a modality, because it's actually fairly easy to do [00:21:00] generative AI with audio at scale, because it's much, much simpler than video, where we'll also see some interesting advancements continue to happen with video, but it's just such, such higher bandwidth computationally.

But with audio and podcasting world, I imagine a world to where it's almost like a choose your own adventure podcast where, you know, I'm listening to this podcast and I heard Alice talk about personalization and I'm listening to you on my iPhone, you know, going for a run or something. Oh, Alice, tell me more.

And then you just start telling me more about personalization in the podcast. That's, that's like a real thing. Like we have all the pieces and parts of that. There's no fundamental scientific discovery required to power that scenario. It's just an engineering challenge at this point.

Alice Locatelli: Yeah, that's right. We've been looking at it more for our professional development as well. We spend our actuaries really love to hear from other actuaries about the different topics that they want to hear about. So we spend a lot of money in our organization flying in famous actuaries to speak on these topics and deliver these professional development [00:22:00] topics.

Wouldn't it be nice if we could have them write up a script or what they want to say and have an AI version of them record it. So we aren't there yet, but it is something that we've been looking at and experimenting and we'll, we'll keep a close eye on that space, not just for the money saving, but just the idea that we could provide better content faster, more often try different things, have some short form videos along with some longer modules, really kind of play with it a little bit more and get more data on what our users like and are responsive to.

Mallory Mejias: I'm fascinated by this idea of website personalization. I definitely think we could do some work on that on our end with Sidecar. I'm wondering what tools what platforms do you use for that? What are the steps that you actually take to implement that personalization on your website?

Alice Locatelli: Yeah, great question. We have Optimizely as our base for our website. And I try not to delve into tools too much because I think it can be really different for everybody. There are a few different tools that [00:23:00] have been useful in different ways on the AI side. Optimizely has their own their own platform and I hope I pronounce this right, Algolia, I think is another one that does similar types of things.

So, there are a few tools out there that can give useful personalization but a lot of it depends on your base technology stack. So if you have Wix for your website, or if you have WordPress, or if you have some of the other tools, these things might not be the right things for you.

So, for us, we really focus on solving the problem and what’s the best way to solve the problem and then identify the leaders in the space for solving that particular problem and narrow in on the right tools to fit our technical stack.

Mallory Mejias: Have you seen greater levels of user engagement and satisfaction because of that.

Alice Locatelli: Yeah, yeah, we have. So we rolled out our personalized homepages a few months ago, and we have seen a 20 percent increase in engagement rate. So that's Google's [00:24:00] definition of engagement. It's a kind of a combination of whether people stay on the page for a long time or they click through to something else.

So we've had about a 20 percent increase in engagement rate so far, which is terrific.

Amith Nagarajan: That's, that's fantastic. And that's a really quick ROI. I mean, that 20%, you know, if you multiply that against the number of visitors you have over the course of a year, you can actually translate that to a direct financial ROI. And that's exciting. And I'm sure the qualitative feedback you'll start to get over time with people having like lower stress levels, knowing that, hey, the button said I'm already registered something as simple as that and not having to, like, click again, go, oh wait a second. Did I not register yet for that and be worried about it? That that's a form of friction, right? It basically creates this uneasy feeling for the user. And you guys have eliminated portions of that. So that's super exciting.

Alice Locatelli: Yeah, that's definitely true. And while we don't have a ton of qualitative feedback, we have gotten some and people have liked what they've seen. What is kind of an interesting thing about personalization is that if you do it well, people [00:25:00] kind of don't notice it. So, for us, looking at our website, we can see all of the different views that people have, but when users come to our website, they only see the view that they see.

So they think, oh, wow, this website has gotten a lot more efficient, but they don't think, oh, this must be AI powered and personalized for me. They're just sort of picturing what they see and finding what they see. So we definitely hear that people are liking some of the content recommendations more and seeing more of what they want to see on the website and that it's easier to navigate to find what they are looking for.

But it is kind of this funny dynamic that if we do personalization, well, it's a little bit invisible.

Amith Nagarajan: That makes a lot of sense. I mean, it's certainly the case what we've seen with, you know, the rasa.io power newsletters over time, where the idea of having to ask people what they're interested in completely goes away because you just learn it over time and people actually you find in the data trends very clearly that their engagement increases over time.

Unbeknownst to them, it just becomes the [00:26:00] thing that a lot of people say, hey, this becomes one of the most valuable resources I get because it’s actually personalized for me. So I think that's one of the things that associations really need to focus on is some basic, simple personalization tactics for 2024 that they can go implement pretty easily.

Alice, I had a follow up question for you. It's a slightly different tangent on the professional development stuff you were mentioning earlier, but I think it would be really interesting to a lot of our listeners. So, especially since SOA has deeply technical people in their audience, do you feel it is part of your responsibility to educate them on artificial intelligence itself and, you know, how it's going to affect actuaries over time?

Is that something that you guys are doing or planning on doing?

Alice Locatelli: Yes, we have been doing that for a while. Our engagement team runs a whole community around. We call it the emerging topics community and they do webinars and topic calls and all sorts of engagement around for actuaries and really understanding the space [00:27:00] of actuaries.

We’re also engaged with a lot of other organizations in the space. So internationally connecting with, international actuarial associations to understand what do we think the future of the profession will be? How will the profession change over time? And correspondingly, how can we better educate our actuaries today and in the future?

So an example is, I think it's been acknowledged that a lot of the lower level wrote tasks associated with fields like accounting and actuarial could shift to be completed by machines in the future.

So, if people are not doing those first and second year tasks for 1 to 2 years, how do they gain the expertise to be able to critique the work that they typically would do later on? I think it's a question that we're going to be asking broadly across a lot of different industries, not only mathematical industries, but really, if you don't need to do some of the grunt work early on, how does [00:28:00] that translate to your more senior roles?

I can say that personally, I have a couple of little kids and one of them is seven years old and already coding some AI bots, very into the space. And so as we see these shifts in the profession, we will be supporting our current professionals, but we also have a generation of upcoming professionals who are learning a lot of this earlier and getting used to a lot of it earlier.

AI school and a lot more personalized learning in school happening. So I think we'll, we'll start to see a, a C change in how people actually learn and what some of those really effective learning techniques are that I think can, can really help not just the younger generation, but also those of us who are already in the workforce, help us learn faster and learn about learning really.

Amith Nagarajan: That makes a lot of sense. That's super interesting. And I think there's no time that's soon enough to start with kids, you know, to learn technology, to learn AI, to [00:29:00] learn computer programming as well. Some people say that well, programming is going to become, something of a lost art at some point because I will take care of the lion's share of it.

And I'm still encouraging my kids to learn how to program, even if they have no interest in it, just because I think it's a good way to learn how to do problem solving. If anything, if nothing else, it's certainly valuable to know how these things actually work under the hood, especially if doomsday scenarios ever occur, and we have to build new forms of robots to compete against the AI that’s taken over the world. So we'll see what happens. But the reason I asked you that, Alice, is that a lot of people I talk to that deal with very technical disciplines make an assumption about their audience. They say, well, you know, they're doctors, they're engineers or they're computer scientists or whatever the case may be.

They already know about AI or they already know about stuff related to this because they're so technical. They don't need us as their association telling them about that. And, you know, you have a very technical audience, right? So that's why I was curious if you guys are taking that, taking that on and going after it.

And I applaud that because I think there's a false [00:30:00] assumption in many of these organizations that have engineers or architects or doctors as members that those people are somehow deeply immersed in it just because they're intellectually capable of understanding this stuff at a very deep level doesn't mean that they are actually doing it.

Plus, like all of us, these professions all have inherent biases resident within them. And so, you know, if I'm an actuary, I probably am thinking differently about what the impact on me could be from AI. And that's different than someone who's looking at it and more of a macro level. So I think that's wonderful that you're doing that.

Have you seen, you know, you mentioned like the lower levels of the profession potentially being fully automated, the impact of that in terms of interest in the field and how that might affect people coming into the exam pipeline for you guys, have you seen anything happen there?

Alice Locatelli: We haven't seen any changes to folks joining the exam pipeline. But I think we are thinking about what is the profession and how do we think about the profession? And how does it change over time? It's a big effort for [00:31:00] our board and our senior executive team to really understand. What that looks like a lot of our focus is on determining what that, what that future role looks like for actuaries.

Really helping them to use the tools that exist because remember actuaries have been using AI for a lot of years. Predictive analytics has been around for a long time. We didn't get to this single insured era that we're in now without having all of these tools. So we don't think of it so much as a big sudden cliff that we're all jumping off of.

But just this progressive additional tool that's getting rolled out, whether we like it or not, and that we really want to help folks to understand it and come up to speed with it. So things like adding our predictive analytics certificate, those kinds of things and engaging in topic calls and engaging the community in these conversations has been really helpful to helping prepare actuaries for the future.

Amith Nagarajan: You know, it's an interesting kind of side topic, which we probably don't have a lot of time for, but I'll just mention, you know, some of the companies that I work with both [00:32:00] within our family of companies in the association space and outside have a heavy emphasis on data and thinking about how they can accumulate a data set that has unique and proprietary value to then drive better predictions.

And in many respects, like a large scale insurance company is the house of data and their ability to make profit is based upon clearly the algorithmic side of it. But the data set critically informing those algorithms and the ability for those actuaries and others to make good predictions and you know, ensure that they're not underwriting the wrong policies with undue risk relative to the premiums and all those things that insurance companies think about how they drive their their profitability.

But one of things I'm curious about at a macro level is if the broader availability of, you know, super powered AI independent of the proprietary data, obviously, but just the general ability of the broader algorithmic capability potentially could democratize the field in a way in terms of the insurance space and other you know, prediction oriented markets like that.

What are your thoughts on that? Because, you know, [00:33:00] there's a lot of opportunity for innovation and a lot of times the incumbents that are the largest behemoths in terms of data and also financially have lower motivation perhaps than disruptor coming into a space that might have a very different business model for something like insurance.

Alice Locatelli: Yeah, I think that's true. And you mentioned the medical field earlier, and I think this is a really great example. My closest friend is a chief medical officer of a physician's group. So we talk a lot about AI in the medical space, and it's both these large data sets and predictions, but also how does AI change medicine at its core.

So how do we diagnose, how do we care for, how do we connect to internet of things devices? I mean, a present day example is our Apple watches have changed how we behave as humans or, you know, whatever your watch of choice is I guess it's changed and more people are getting their 10, 000 steps and more people are doing this preventative care because they have a device that connects to them to help them do that.

And thinking [00:34:00] about those impacts happening across the medical field, where we can tell what patients are doing and why and how we can help them is making big differences in the medical field. Similarly, even within hospitals, so I know University of Kansas medical system did a large scale look at all of their data to understand why they had a particular problem relating to diabetes patients.

So, diabetes patients were being readmitted at 3 times the rate they, they should have been and they were able to find that the key link there was after analyzing lots of data, including things like what bus routes patients took to get to the hospital, where they were coming from, where their pharmacies were, what the hours of their pharmacies were all of this kind of data at the end of the day there, they were able to reduce their rate of readmission for diabetes patients by changing the education that patients got on their way out of the hospital.

So, some nurses were really good at explaining the instructions and we're good at getting [00:35:00] compliance and others were not delivering the information in the same way. So, really all of these kinds of data sets are coming together to give us a lot of information in ways that we haven't necessarily had access to it before.

So, I do think we'll see some big changes in in the space and how we think about it, how we perform. And I think for us, at Society of Actuaries, we're really focused on staying close to regulators and understanding what the goals of the regulators are and what they're trying to achieve as well as the large insurance companies, as well as the different groups that are either building a software or bringing together these data sets to really try to stay at the nexus of what's happening and, and keep an eye out so that we're delivering the right kinds of information to people to help make this all work better for all of us.

Mallory Mejias: Sounds like you guys are approaching this from a great angle, and I'm sure we have a lot of listeners that are feeling motivated by the work that SOA is currently doing. I want to ask for those listeners who are feeling [00:36:00] motivated and may not know what the best next steps to take would be. You mentioned earlier this crawl, walk, run methodology, and I was hoping you could dive into that a little bit more and explain how you all approach innovation at SOA.

Alice Locatelli: Yeah. So for us, it really starts mostly with user research and understanding what the users need. So I think problem identification can often be a missed step in, in the innovation process. It can be hard to figure out what problem you're solving, especially if you or your CEO or somebody higher than you has seen a demo of a cool AI tool that seems like it would be great for you, it's really tempting to buy that tool and implement that tool without necessarily thinking about what problem you're solving.

So starting with understanding the problems that you're solving or the issues that users have is really that first step. And then from there, I think it's figuring out what is that progress over perfection? What can you build tomorrow or maybe not tomorrow, but in three months [00:37:00] that's better than what users have today?

That can be different for everybody what that exactly is, but think about that 80/20 rule and don't try to solve for everybody. Just try to solve for that core group of users, make their experience just a little bit better, and then keep making it a little bit better every day. It's a little bit like weight loss and I hate to draw that parallel, but it's a little bit like weight loss that the consistency of making these small changes is much more important than having the right large change that you work on for five years and pour millions of dollars into and then roll out at the end of it.

Amith Nagarajan: I think that's such a key point that it's about making small progress every day, sticking with it and, you know, spinning that flywheel, building that momentum over time. Breaking the inertia of because you haven't done it or because you haven't necessarily been the most innovative organization that it grounds you and, you know, the state of paralysis essentially, and also that ability to kind of experiment as you go. Uou know, the crawl, walk, run [00:38:00] methodology you're describing is really similar to what we talk about in the ascend book, which is the AI book for associations we wrote earlier this year where we talk about this framework called learn experiment build, which isn't exactly the same thing, but it's kind of a similar idea in a sense of just trying to get awareness of what you can do, play around with it and kind of a non-mission critical way. And then only then really kind of deploy things into bigger and bigger pieces of work.

I think, you know, you guys probably have experienced this, that, you know, as you continue to add a little bit of knowledge, a little bit of knowledge, a little bit of knowledge, you know, collectively, your culture shifts and you're able to, as an organization, do things. You can move mountains, essentially, that whereas it might have been impossible even 12 months ago.

Does that resonate with you,  the last thing I just said.

Alice Locatelli: Yeah, absolutely. And I have an example from my last association. So I previously worked at a, an association for legal technologists. And 1 of the big shifts that we made while I was there was to move off of our old AMS onto a Salesforce based AMS.[00:39:00] What we did was started with the users that were excited about it and got them involved in it. And whether they understood it or not, or if they just had a piece they were excited about, we tried to engage our staff in any way that they were interested in being involved. And then as we rolled it out, more and more people opted in to saying, yes, I want a dashboard that I saw somebody else had.

So I think there can be this tendency to want to get everyone on board from the very beginning and have full agreement on something before you move, but there's also a progress over perfection in change management as well.

And that if you can use those super users or the people who are really excited about it to engage in new tools, new software, and then other people see what they're doing. I love when people opt in to want it, especially people who've been reticent. Opt in to wanting access to a new tool that they might have been previously reticent to use.[00:40:00]

I think that's the most powerful kind of change management and the technology space that you can have.

Mallory Mejias: What are some of the biggest challenges you faced with your AI projects at SOA?

Alice Locatelli: Great question. I think determining where the 80/20 is is something that is going to be a challenge for everybody. So, everybody in an organization has different users that they care about or different, you know, different motivations or pathways. So, for us I can frame pre candidates, candidates and members as the 3 audiences to you, but there was a lot of conversation that got us to that point. Employers are another really big audience for us and really important audience for us. Even if they're numerically smaller and part of the question is, do we hold up the personalization of the website to make sure that we are taking care of that very, very important audience to us?

So I think some of where you draw those lines and what you do first [00:41:00] can be challenging because there's often a sense that a feeling that, once you roll something out, it's done and I'm never going to get anything more than that. So we've had to really work hard to break that preconception in our organization. We really use a very iterative approach to building the website and changing the website.

Any suggestion that we get from people that they have any strong feelings about, we put into an A/B test list, and then we review that list regularly every quarter to say, what do we think are the key changes that might drive indicators for our users? And let's A/B test some of those things to know whether they were good ideas or not.

So really trying to build in those processes so that people in the organization feel heard. Users can see what feedback they've given us that we're considering for later. And that you really build transparency for that iterative process and make sure that everybody can really [00:42:00] see their ideas somewhere, whether it's in version 1 or version 10.

Amith Nagarajan: Alice, I have a question to follow up on that a bit, but it's again a bit of a different direction on it. So, I know a fair bit about your organization. I know your CEO is very supportive of change and you have a culture of very innovative professionals thinking about how to take small risks and, you know, see how the process works, everything you're describing, which is awesome.

And I think there's a lot to be learned from that. But imagine if one of our listeners, let's say is an earlier career professional. So, you know, you're part of the senior management team. You have a CEO who’s super supportive, but let's say I'm an earlier career professional. Maybe I'm three or four years out of school.

I'm really interested in this stuff, but I don't have any formal power. And let's say I also work at a very tradition bound organization, aka inertia bound, right? So like a group that isn't doing what you guys are doing. What advice do you have for that person who's early in their career and they want to do something, but they don't necessarily have any formal authority?

Alice Locatelli: It is a [00:43:00] great opportunity when you are younger in an organization to learn a lot about managing up, remember that the people that you are managed by are not always smarter or better than you. I also say this to my own staff. I am not necessarily smarter or better than you. I've just been doing it longer.

So really taking that opportunity to figure out what is it that your senior management team wants or is looking for, or even your direct manager, what can help them look smarter? What can help them be better? What can help them achieve their goals and really asking yourself what's in it for them. So understanding, you know, if your manager is charged with increasing sales or improving engagement, look for opportunities that fulfill the need that your manager is already charged with accomplishing and tie in your efforts into what they're already doing, build a case for what metrics you think you can move with whatever [00:44:00] this, you know, tool or change or process is and give that information to your manager and then listen to their feedback, really understand what they're saying and learn from them, learn from their experience and then take your idea back and either change it or shifted or, you know, so I think it's just a when you're young in your career and you have really great ideas and you're really smart, it can be frustrating because you feel like the more senior levels aren't moving fast enough. But remember that they also often know something that you don't know. So try to figure out what that thing is by tying into what they're trying to do and pushing through your initiatives that way, which I will say is a skill, no matter what level of seniority you are, even when you're at the CEO level, you're doing this with the board. So it's a skill that never gets old and is incredibly important to earn while you have lower stakes earlier in your career.

Amith Nagarajan: That's wonderful advice. I think it is applicable to [00:45:00] everyone at every level. And I'll say, you know, for those of you listening in that, that particular clip right there is worth the cost of admission to the podcast by itself in the time you invested today. I'll add one thing to that is and we talked about this in the book as well.

It's talking about like bottom up driven change management as opposed to top down. By the way, we mentioned the book a couple of times. For those that don't know, it's sidecar global.com/AI. You can download the book Ascend: Unlocking the Power of AI for Associations for free. But in that book we talk about bottom up change as well. One thing I like to tell people is that it's a lot easier to beg for forgiveness than it is to ask for permission. Now you have to be very careful who you tell that to because they have to use some judgment there. But I think people at every level of an organization have a lot more capacity for judgment and common sense.

Certainly more so than AI does. And so giving people a little bit more autonomy to test things out in small stakes ways, and then show the results. That's one of the best ways to get senior level people to pay attention is if you show, hey, I got this thing done in one [00:46:00] day that you gave me three weeks to do.

You didn't say, hey, can I use this tool? You just went and used it, right? Don't upload your AMS's data into Chat GPT. And there's other things you have to be thoughtful about. But I think it's an opportune time for leaders of organizations to give their teams a little bit more autonomy so that they have the opportunity to experiment in small stakes ways.

Alice Locatelli: Yeah, I think that's right. And the other thing is your hobbies can become your job. So experiment with this in your own life. If you have a personal website, play with these tools and see what's out there. There's a lot of free and open-source tools out there, whether you're a developer or not. So really engage in it, engage in the conversation on LinkedIn.

There are some great influencers sharing a lot about really neat things that are happening in the space and the more that you know about it and can bring to the table, the more your senior leaders are likely to listen to you because you have the answers. So I think focusing on it, even as a hobby is a really valuable thing to do.

Amith Nagarajan: Alice, you mentioned open source briefly. So can [00:47:00] you give us the quick thumbnail of your viewpoint on open source versus proprietary closed source AIs? What are your thoughts on that in terms of, you know, if you were speaking for SOA and what you know of the organization, how open would you guys be to open source models being used to power your AI strategy versus closed source or a mix of the two?

Alice Locatelli: Great question, and you mentioned our very supportive CEO, Greg, who has been phenomenal in supporting everything we're doing. And I think we would decide based on what the problem is and what the best tool is to use it. I know right now that I'm playing around with a lot of the models on hugging face to really understand what AI can do and using auto gen to pull those models together into some fun conversations.

I like to choose funny topics and have them all talk about these funny topics. Um, but using those tools. Maybe they are the end game, or maybe they just point you in the right direction of a proprietary tool, [00:48:00] but all of them help educate you on the right questions to, to ask. Personally, I prefer in an organization that is not a software organization, I prefer not to build custom software if we can avoid it. So I would lean towards using a proprietary tool that does 80 percent of what we want over building our own custom solution that then we have to host and upkeep and all of those kinds of things. But it, it could be the right solution depending on what the problem is.

And I think there's a lot of education to be had in open source. The other thing I really think about open source is that it is the startup of the AI space, so often we know in industry startups do really innovative things. They are the R & D arm of some of these big organizations. And then eventually they get acquired by one of the big organizations and pulled in.

So, I think of open source as a way as a way to tap into. Understanding what is new and happening in the space that [00:49:00] might later end up in your. Salesforce or your other big, big tool that you're using.

Mallory Mejias: Well, Alice, it has been so awesome to have you on the podcast. You're our first special guest ever. And sometimes when it's just Amith and I chatting, it can feel like a little bit of an echo chamber because we don't see our listeners real time, but it's really exciting to have someone from an association come on and talk about the work that you're doing.

Do you have any final thoughts, anything you want to leave with listeners?

Alice Locatelli: Well, I'm so honored to be your first special guest on the podcast and really appreciate all the support that Sidecar has given Society of Actuaries over time and will continue to give us. So thank you for all of that. I think if I can give one piece of advice, if I haven't said it enough times today, please listen to your users, find out what they need, find out what they want, and then decide what, what the best delivery mechanism is for them.

Mallory Mejias: Thank you so much to Alice for a fantastic conversation. If you find yourself listening to this episode, wondering how you can do work like [00:50:00] SOA at your organization. I of course recommend that you check out that AI Learning Hub that we mentioned in the beginning of this episode, not only for the educational content, but also, so you can connect with other association professionals, nonprofit professionals, to see what they're doing at their own organizations.

And hopefully be inspired by that.

Believe it or not, we have another special edition episode lined up for next week. We will be diving into the top 10 predictions that we have for AI in 2024. So stay tuned for that.

Thanks for tuning into Sidecar Sync this week. Looking to dive deeper? Download your free copy of our new book, Ascend, Unlocking the Power of AI for Associations at ascendbook.org. It's packed with insights to power your association's journey with AI. And remember, Sidecar is here with more resources for webinars to boot camps to help you stay ahead in the association world.

We'll catch you in the next episode. Until then, keep learning, keep growing, and keep disrupting.[00:51:00]