The industry is kind of going crazy at the moment.
There's also an integration mania that's happening with these companies.
So you shouldn't be thinking about chat GPT during this talk.
You should be thinking about a capability that's trained with your company or community's materials.
Know's what you know, and that is integrated into all of your channels, slack, discourse,
Twitter, Zen desk, etc.
And that's speaking through those channels as a support agent or even possibly the brand.
How many people have seen the Gartner hype cycle before?
Okay, so I'm getting like pretty old now.
And I feel like I've been through this like four or five times.
And we're at a point in history where people are talking about things like AI alignment research.
And there's a subset of the population that is seriously concerned that AI could actually extinguish the human race.
That suggests to me we're coming up here.
That's a pretty inflated expectation in my experience.
Now, I could be wrong. Maybe AI will kill us all.
But honestly, it looks like another tool to me.
You guys can all tell me I'm wrong in 10 years.
But basically, there's a lot of hype.
The company space is exploding.
And so there's a lot of froth out there.
Okay.
That makes it difficult to say exactly what the world is going to hold because you usually have to get really burned out on any given technology
before people figure out how to apply it in a smart way.
But I'm going to attempt to look into the future on the basis of scenes things that we've seen before.
And we're always going to hue back to like basic good community principles.
We're not going to look at vendor marketing material, but we're going to look at community principles.
And so what are those? Well, unsurprisingly, community is about people talking with one another.
And so the core two principles are basically like to the extent that you can use a technology to enrich or simplify people's interactions.
It's probably a good thing.
To the extent that the technology replaces, complicates, disincentivizes or gets in the middle of human to human contact, then it's probably a bad idea.
Okay. A couple of principles below that.
And, you know, as community professionals, a lot of us talk about trust.
Okay. So if you think that trust among human beings is important and I kind of do, then basically disclosure is a key principle.
It's a very bad thing if people are confused whether they're talking to a robot or whether they're actually talking to a human.
How many people have done tech support with a company in any context where it wasn't clear whether they were talking to an automated robot or a human?
Yeah, okay, that's most people thought so.
They didn't give you disclosure. The second is opt-in.
So basically like ideally you want to let people choose whether they want to talk to a robot.
Now there's one possible exception here, which is many people are going to feel like robotic responses are okay provided that they come with a human fast follow.
So if the robot is the only thing that you're talking to, that's bad.
But if the robot does something useful, like for example, a GitHub automation bot, and then you're presuming that you're going to talk to a human next, that might be okay.
We're going to talk about examples here.
So let's get into the options.
I'm going to try to structure this as a goofy menu.
And we're going to fuzz them fiddle around and find out because I think that that's like the core method of how human beings learn things in the world as you mess around and you try to figure out what's going to happen.
Sometimes you get burned and then you learn from that and then you rinse leather repeat.
So the first option, let's call it the intern.
Okay. In my community a while ago, we launched an LLM implementation that on GitHub automatically triages a GitHub issue.
And so you type what your issue is or what your pull request is, it reads it, it compares it to other things that we've seen in the past and then it applies instead of GitHub labels.
Automatically to the issue. Now for disclosure purposes, we actually told the community that we did this and posted it.
So far, this is going really well. I suspect one of the reasons why this is going really well is that while it is driven by LLMs, nobody knows or really cares, it's just another GitHub bot, right?
So you can make the bot smarter with the use of LLM, but this is not really functionally much different than what most people have seen before.
If you have, if you use the QR code to the slides, you can see where we announced it to the community for transparency purposes and you can actually access the code or steal our code if you want because it's open source.
A branch off of the intern pattern is there's a lot of systems that already auto detect spam for us. How many people have got a discourse instance where they're dealing with spam posts?
Fewer, but they're out there and I was just triaging spam posts earlier today on hours and yeah, we're not going to read this anyways.
So it's possible to set up filters where posts get sent through an LLM and where you're asking questions about is is person A being abusive to person B is person A using sexist, racist, otherwise violating of the code of conduct.
Language in their post, that might require special admin or mod intervention. We're not doing this yet to the best of my knowledge. This isn't being done, but it's super easy to build.
And I'm thinking about doing this for FOSDM 2026, it's just one of one of my ideas.
Shout out to the Devroom organizers, they're the only Devroom at FOSDM that I've seen that has the code of conduct points of contact on the whiteboard to the right.
So that got noticed in a good way.
So our second pattern is called the researcher. I'm planning on releasing some open source code that does this. We are already doing this internally at Grafana.
So the idea here is that you might want a bot to read everything that's ever been said on a given topic and then summarize it.
Either for internal stakeholders like product management or for your community people so that they know how to go about supporting a particular case.
In the researcher case, you're going to create a mark down document that might be 400 pages long and then you're going to ask it questions.
Okay, this is fairly unabtrusive for the external community because they're not going to ever talk to a robot.
It's simply an automation way of being faster about reading everything that they've said and being aware of their feedback.
We use common room internally. A many community professionals may be familiar with either a common room or what's the other one. I'm forgetting it.
Somebody shouted out.
Or but thank you.
So we're doing this with common room.
Our third pattern, I like to call the robot in jail.
And the reason we call it an robot in jail is that we do put it on the forums, but we segregate it into a given category.
We describe that category. We do not ever make it an auto responder to anything else and we let those interact with it who want to.
So we're doing this right now on our slack and for example on our discourse.
Again with full disclosure following the principles that we describe.
Open source code to help you do this is available for discourse today.
Lots of vendor integrations exist for this.
I've talked to a couple of vendors and I've said how do you recommend people do this and they recommend this pattern.
They say create and ask AI channel.
So it's obvious and clear to everybody what they're getting involved with.
And if you hate the idea, don't go in.
Okay, this one is what I would like to do next.
This is called the egg drop or the reddit sidebar.
So if you've been on IRC for a long time, you often have mobs in the channel.
People come in and they ask the same questions over and over and over again.
You want a person to talk to a person, but you want the person to be able to use an LOM.
So we want to be able to extend to moderators or people that we trust in the community.
The ability to tag the bot and say, hey, explain fact number seven.
We're explained the common instructions about this thing that we get asked every single day.
So interaction at the behest of a person, but not directly the bot answering to the person.
And so in this sense, it's like making an LOM like an IRC bot a help utility that people can call,
but that never speaks without being spoken to.
That would be the reddit sidebar.
This is one that I'm telling you is coming, although I haven't seen it quite yet.
How many folks know about donut AI?
There's Slack matching bots.
Okay, so most people don't.
So there's a company called donut and basically you invite this bot into a Slack or into another chat forum.
And it periodically pairs people.
And it says, hey, Joe, and hey, Susan, you should meet and get coffee.
And that's configurable a hundred different ways and so on.
I happen to love the movie, everything everywhere all at once.
So I couldn't resist and everything bagel reference.
But essentially, this would be the use of LOMs for the purpose of knitting people together.
So the LOM wouldn't speak when not spoken to, but it would basically understand what you're interested in on the basis of what you've talked.
And then it would arrange something like speed dating.
It would say, hey, community member one.
You should talk to community member two because you share these interests.
This cool thing that this other person said.
So the robot is just trying to make an introduction and then get out of the way and let people talk.
I think this is an intriguing thing that's at least we're thinking about, but we haven't done it yet.
There is always this option.
You could do all the things in all of the places.
And I got two other case studies I want to share with you.
The first, I tried to obtain permission to tell you who the companies were.
I was not able to obtain it.
So I'm going to describe them as best I can.
The first is a Gen AI company.
They basically automated everything full stack using LOMs.
And since the very inception of the company and their argument was that they were dog fooding their own product,
which they were.
So you can kind of understand why they felt that way.
At any rate, they had not invested in a community.
And this seems to be preventing them from building a community.
And that they're training their user base to just go talk to the support bot because that's like what the whole product is about.
And so it seems like an active thing that's preventing them from building a community to be frank.
The second is there's a company in the data space that does has both open source and also a commercial enterprise offering.
They got a lot of internal pressure from one of their VPs to help them reduce support costs within the company.
So they went pedal to the metal and basically automated all of the things.
Whenever you come to their support forum, the support bot immediately answers you.
And some of the folks internally that I talked to there said that, well, is this a success?
I mean, it kind of depends, right?
On paper with the financial metrics, they did a really killer job reducing support costs by something like 70%.
Because most supports requests were deflected and never got seen or dealt with by a human being.
On the other side, the community folks are seeing overall engagement decline.
Who's surprised by that?
Right?
I mean, so this is the tradeoff, right?
If you really max these things out, you're kind of getting in the way of humans interacting.
And you save money at the cost of potentially compromising some of the point of what community was about in the first place.
Okay, so basically how it goes wrong?
I mean, there's a lot of concern online about AI slot right now.
Put simply, how it goes wrong is that you prioritize the wrong thing.
If you prioritize efficiency, cost, and scale, it tends to go wrong because you've made community about something other than people, pretty straightforwardly.
The other way that it goes wrong is that you violate the principles.
So you don't disclose that it's an LLM and you kind of fool people into thinking that they're talking to a person and then you bait and switch them or it becomes obvious when the robot can't answer the question correctly.
Or you don't let people opt out.
If anybody knows the movie Vanilla Sky, there's a moment where a character gets trapped.
And he realizes he really needs to escalate the issue and talk to somebody who knows what the hell is going on.
And he's in a sheer panic. That's that moment.
There's about a million articles online about how to reduce customer service costs.
And as a community person, it's always kind of weird because it's like, well, I don't really want to increase customer service costs.
But I also don't entirely want you to make them zero.
Anyways. Okay. So there's this range of two hot and two cold options.
We can think of them in three categories. You could go with no robots and you could say LLMs are evil and that the training is wrong and that they're not fully open sourced.
You could take the position that they should stay out of your community completely.
You could adopt a couple of these patterns that we're talking about or you could go pedal to the metal.
And it's going to depend on your ecosystem, obviously.
The trade-offs that I've seen so far and in talking to other people who are doing this, they're not talking about it publicly as much.
But basically, if you do some robots, if you do, for example, in turn and robot in jail, you're going to create these special areas that people need to know about.
And that means that in order for people to get value at of them, you're going to have to do advertising and socialization.
And they run the risk of becoming special tricks that only your insiders know about and that kind of to some degree lessons there value.
It also leaves common questions on the table.
So if you do robot in jail over here on this Slack channel, people are still showing up to your general channels and they're asking the same frequently asked questions that they always did,
which is why I was getting interested in the egg drop model. If you remember that, we talked about.
So you go pedal to the metal. I mean, it's pretty stark and it's pretty simple.
You deflect a lot of support tickets. You save a lot of time and money.
And it's really, really bad for networking and connection because you're diffusing all of the random connections people would make,
which could develop into deeper conversations or even relationships.
You're shortcutting that and chopping it up, right?
So quick recommendations for 2025. I said at the top of this talk that everything is changing really fast.
And so I'm going to reserve the right to change my mind. If I come back in like falls down in 2029, I want to be able to like,
basically disavow everything that I said on this stage here today because that's how fast it's moving.
But I would like to recommend to you robot and jail researcher and intern.
And the reason why I'm recommending those things is because they match all of the principles that we described up top.
And they give choice and agency to the people who are in your community at the cost that you might have to do some,
some marketing inside of your community to let people know that these capabilities exist.
I recommend that you avoid no LLMs at all because just like any other piece of software, it's automation that can save you time.
And I also recommend that you avoid pedal to the metal because I think your bosses will be super happy with the financial returns and you will be super sad about the community returns if you do that.
I am personally interested in doing future work in this area and I'm going to be thinking about and possibly implementing if I can get the time,
I drop and read it sidebar and thinking about with some community folks that I trust.
What the super intelligent donut would look like, man, that's a weird sentence, isn't it?
What would a super intelligent donut look like?
Anyways, the final warning that you get before you rush off to implement is that people aren't technical problems.
People don't need to be solved like technical problems.
And please try to avoid automating people.
I think sometimes our community can be prone to an engineering mindset where we think about how to get efficiency and automation.
And in the community world, we are simply not technical problems to automate away.
Last, resources and some open questions.
Really, really smart discussion on LinkedIn a while ago.
Emily frames the question and then the threat underneath of it is worth reading.
It's basically like if people show up to your ecosystem and all the code that they're putting in via PR is mostly written by LOMs.
How do you feel about that?
Unsurprisingly, the answer is mixed.
Duck alignment is working on policy resources for open source projects.
So if you're worried about and or thinking about this, you should read about this.
And also what RINMAN QSO is doing with the community contributor covenant.
And this is about, I mean, unsurprisingly, you're going to find principles of disclosure in there as well.
I work with Richie Hartman.
And he did the keynote for Faustem.
And he ended the keynote with B. Excellent to each other.
And I couldn't think of a better way to end this talk.
So that's where we're going to leave it.
And I wish.
Thank you.