All right, hello everyone, oh thank you, thank you, my name is Shireen Bellamy, I'm a
senior developer advocate for AI security and quantum over at Cisco, today I'm really
excited to talk to you about beyond MCP servers because last year when I was doing a lot
of my developer advocacy a lot of talks that I went to or spoke about had to do with the
emerging agentic protocols and building MCP servers. So I'm excited to kind of expand beyond
that this year by looking at why network automation's need knowledge graphs. And I'm going
to do that based on an open source multi agent system that I'm a part of that's there's a group
that I'm a part of at Cisco and I'll explain more about that later on, but basically there's an
open source project called coffee agency and that's where I'm going to share my lessons that I learned.
So a little bit about me, I'm not from Belgium, I'm from Boston, Massachusetts,
I went there for grad school and so yeah my background is actually an AI and human computer
interaction, but ever since joining Cisco last year I've learned a lot about network automation
and where AI leaks into the networking world. So the way I was able to do that is through this
agency project which I'll talk about in a minute, but other open source activities that I do,
I do talks and attend meetups a lot for the Pi data community in Boston and New York City.
So if you're over there, say hello. So what we're going to talk about, we're going to start off
by just reflecting on the year of MCP and agency last year, then we're going to talk about an
issue that I encountered while working on the project and how I saw that by learning about the
power of graphs. So after that, I'll talk about how to try out coffee agency yourself and build on
what I already did. Starting off with the year of MCP 2025, so MCP really started in like
the end of 2024, but I think it's fair to say that it clearly was widely adopted
at a very large scale throughout last year. I don't think many people are arguing about whether
or not MCP is a standardized protocol for agents to communicate and that it solved a problem.
That's very clear from the messaging last year, my opinion, and in terms of things that
were built with it, there are so many MCP servers that have been built that I didn't even feel
comfortable putting up a number on the slide because I don't know how accurate the numbers are
that I found online. There's so many out there's one, like I feel like every day, there's a couple
of hundred or something or a couple of tens or I don't know. So yeah, MCP, large growing.
But I also want to talk about the year of agency, right? I spoke about how we all know
what MCP is. A show of hands is anyone know what agency is in here? In the front, and that's because
there are part of Team Cisco, so clearly I have some advocacy work to do here. The way that I learned
about agency is through a reference application that I worked on. Now, what is agency? It's not just
a Cisco project. It's actually a collaboration of different companies that came together and said,
hey, with all of this agency AI talk that's happening, what does the internet of agents really look like?
When everyone's talking about these protocols, what do these systems look like as they scale out?
What can we mix and match and change in the in between layers? That's what the internet of
agents is about. So agency team, we have a reference application, we call it coffee agency, which is
based on like a coffee trade system, and there's two demos in there. There's a Gordo demo,
which is a simple tuition system communicating, and what that's showing is a very simple example,
not only of a protocol working for these agents to discover each other, but it's showing this
concept, like I mentioned, of mix and match, so it's showing that if you have, if you look at the
messaging layer and the transport layer that's involved in the protocol, you can use an A2A server
and have a layer of slim, and then have these agents talk to each other, so that goes into
that and shows that at a very micro level. Then what I'll be focusing on today is the Lungo application,
which is a full multi-agent simulation with different patterns of communication. So it's extended a lot
after this presentation, so I don't think we'll have time to answer all the questions about
like updates on it, but I do have a QR code from my link to it at the end if you want to like
message me through there. But for today, I'm going to be talking about the very base layer of Lungo,
and how I got started, how I learned about what was the problem with how I learned what was the
problem with protocols. Well, not about protocols, but about like MCP. So before I get into that,
again, this is what the diagram of the structure of it looks like. So the way that the coffee
multi-agent system works is that you start off with the auction supervisor agent as the buyer,
and all these agents are designed by Lungo. By the way, there's structure through Lungo.
But the supervisor agent is able to communicate with all the farm agents from different companies
and request information about their supply of beans and the resources that they have.
So it also includes, like on top of the A2A over a slim transport, which allows for the
pubs of broadcasts and it allows for point to point communication as well. There's also two MCP
servers already existing in the repo, which are the weather service, Python file, and the payment
service Python file. There is a walkthrough available. I uploaded this online, so there should
be publicly available after the talk, like all the QR codes and links that I have in here.
Of how this works more in depth, this is just a quick presentation, so I'm going to fire through there.
Here's just a quick screenshot from very far away. What it looks like, the base, I will zoom in
later and show you what I did in the same UI. So the problem that I encounter while working with
coffee agency is that there is an issue of infrastructure awareness among the agents. If I asked
it a simple question, like how many beans do you have, then yeah, it'll answer, but sometimes
if I answered something that required a little bit more depth, why did this order fail?
It wouldn't be able to tell me specifically why or it would give me a very long paragraph
in response. What I decided the issues were, based on what I was going through, is that the context
window was filling up really fast because every query that I was sending in was basically giving
the whole graph and in turn returning way more than I needed. And that's because when in the typical
structure, the LLLM was doing all of the reasoning and because of that, it was talk, it was using
way more tokens than needed. So in my research of how to fix this issue, I learned a lot about
folks adding graphs to MCP servers and reference applications that exist in that round.
Anthropic has reference application for knowledge graph memories and an MCP server.
There's an MCP server for a Neo4j going to what I said earlier. There's an MCP server for a lot of
different things. But specifically when it comes to IT infrastructure issues, LinkedIn was able to
resolve their amount of tickets that were solved due to integrating knowledge graph with their
MCP server. And Cisco also has a couple of case studies to where they were able to do some
deep network troubleshooting in that manner and they were able to implement Jarvis. And that's
sorry. So given that, I'm going to call attempt one that issue that I spoke about, where the LLM
is dumping all of this JSON and in return, I'm getting all of these tokens used up because
look how many characters are there, right? It's a lot to block. So what if instead of this situation
here where you have the supervisor asking a question to the MCP server and the MCP server
clearing directly to the LLM to parse that JSON, what if the MCP server answered it directly?
So I know I said that the talk is beyond MCP servers, but the key is what can we do to enhance it?
Not, you know, we did MCP last year. Let's get over it this year. I'm not on that boat.
So attempt two, which is my actual first attempt, is that looking at knowledge graph extraction,
because that's all of those case studies from different companies. I look more into knowledge graphs
and I learned about retrieval mechanisms that were out there, like CAG and GraphRag. So
the reason why these both interested me was because CAG was a logical form of guided reasoning
with multi-hop Q&A. So I figured network topology has a lot of hops and you get usually to solve
queries, so sounds useful, and then for GraphRag, because it's known for extracting entities and
relationships. Again, I just felt like because of that natural graph traversal kind of property
that network topology is naturally have, these two would be a good place to start in terms of looking
out what it would look like if I could extract a knowledge graph. So what that procedure looked like was
I collected the Lungo deployment logs and network diagrams as text, all the information
fed it into both approaches, and then I generated a knowledge graph. And with that knowledge graph,
I would just query different things like what services depend on the slim gateway, which is the
transport that I mentioned. What I found was that CAG was still missing some implicit connections,
even though they claim to be better at the multi-hopbing and stuff. So that wasn't a good idea.
And then for GraphRag, it was better at summarizing, but it was very, very slow and it kept
rebuilding every topology, which is the same issue that I came across in attempt one. And take
it one step further, I learned that both of these are actually two extract graphs from documents,
which are, you know, just deterministic static modes of information. So since Lungo's network
topology is already structured data technically because it's already structured in that
JSON, why extract what we already have, which led me to my final attempt, which is having a graph
backed MCP server. And what that looked like was I ended up creating my own MCP server specifically
compatible with that knowledge graph, but then I also attached a graph database to it.
So I'll get more into that, but the goal for this was to have the tokens more aligned with the
output to reduce the token below, have a native traversal instead of generating the same thing twice
and having more direct queries. So I figure if we're in a situation where the supervisor has a
question for the server, the server is running directly a query, a cipher query, which is what you
can use to traverse the database, then the supervisor would receive exactly the right amount of tokens.
This is the proposed stack of what I used to do that. The orchestration was with land graph,
the interface through the MCP server, the retrievals through the cipher, and the data is done with
a knowledge graph in Neo4j. This is what a knowledge graph in Neo4j looks like, and it's the visual
for the topology demo. So again, the two big things that I generated, it's the knowledge graph
and the MCP server was several different features that align with network topology.
And ultimately with that, the capability that I pitched to the team is this idea of impact analysis.
Identifying which visual node in the knowledge graph would be impacted if other servers go down,
and that was done with the cipher query, like something like this, which would be something that
you can solve a question like, what is your blast radius? So here's an example of the approach
that I did in the same UI as the beginning, and inches to time, I'm just going to skip around a
little bit, but basically what I'm asking is for the first question is to show the approach that
I implemented, and you'll see there's a very precise answer, and how many characters it is,
which would the token amount differs depending on what tokenizer is being used. So I just measured
it in characters, and then there's also an example, if you say, can you use a approach one,
which is the bad approach, you'll see the problem that I ran into, where it was generating
just a large amount of text. So you can check that out on your own time, but what the UI is doing
is recreating the story that I went through and giving you the opportunity to go through the code
and expand upon it yourself. What about other use cases like routing dependencies? Well for that,
it would be an example of a data change, maybe adding obviously routing information, and that
physical topology we discussed, the questions would be something like, what prefixes are affected
if router x fails. Policy dependencies, that would look like something like the data would be security
groups, or ACLs, as graphs, or firewall rules. And the question would be like, what breaks if we
remove this one firewall rule? So there are a lot of growth opportunities. Again, this is available
online, so I'm not going to go to too much into depth, but I truly have a feeling that in the
realm of context engineering, all of these are greater approaches to play around with the reference
application. And today, what we spoke about, this is the full realm of agency and that
Internet of Agents. We spoke about specifically the syntactic layer and the messaging layer.
Syntactic being A2A, MCP and messaging layer, we didn't speak about it, but it was it's in the demo
for like the mix and match concept we spoke about swim. So getting started, when you get to the QR code
for a long ago, you're going to see again, there's two options. Remember, long ago, there's
one we spoke about today. And then my demo is on my repo. So with that, I spent through the end.
I'm sorry if it seemed a little rush towards the end, but thank you very much. If you're interested
in further questions about agency, the work that we do, or how to extend this, that's the QR
code for my LinkedIn. And then the QR code for the demo that I showed the quick UI and about this
presentation is on the other side. Great. So we do have a few minutes for questions. Some of the issues
which you touched upon are expected to be solved. Also for example, with this A2A protocol,
do you have any take on this kind of how matured this and so on from your side?
Yes. So if I understand your question correctly, you're asking if there's room to mature beyond
the components that I showed you today, like the A2A.
Yes. So A2A, MCP, they both are doing essentially the same thing in terms of being
two different communication patterns. However, what I'm showing today is it's more of a reference
application for looking in between that. So us as network people interested right and how things
are communicating under, that's what this is. Yeah. So if you were looking to like mod out A2A with
other components, I've seen here, like maybe you want to add like age and identity, age and
directory, other things. Yeah. It's more aligned with this. And one more question.
Hi, thanks for the talk. How much maintenance would it be to keep this effectively in knowledge
graph, but like a customized one, with an MCP server as the topology updates, as the
like consider that if we talk about something more dynamic than just the network of physical devices,
what we're talking about, the policy changes that you mentioned, or the cloud native deployment
that may change every minute. How much high maintenance would it be to implement this approach
in this case? Well, in that case, it depends on the architecture that you built. So, for example,
if the policy changes, right policies are changing everyday, especially like, you know, you're up.
So, when you deal with that, maybe if you alter any of these other areas,
you can automate a certain section of that, and that way you're not working to keep up with the
zones of suicide.
We have time for maybe one more question, anyone else? Yeah.
Well, thank you for the presentation. First of all, it was very interesting, but my main
question is, what do you think it's the most useful strategy to fill up the graph that the
base of the knowledge graph, whatever it is? What do you mean by fill up? But, I mean, in this case,
you mentioned that we already have a structured data for the network topologics, etc, but let's
imagine that we are doing this, let's say in real life, so we're starting from an existing project,
or an existing architecture, what do you think will be the best way to start building this
knowledge graph, or what do you use or suggest what techniques we can use to build the graph
before you can be able to use it? Um, would this existing architecture already have a jented properties
within it, to any capacity? Yes. Yes, like what? I mean, we may have
lots of documents that may define parts of the architecture, and I want to build the graph,
including, let's say, all this knowledge that it's shot the red and in different sources. Okay,
so I would say for that, again, this is a very small scale, but you're talking about something
that's existing, that for that, keep in mind that chart that I just had on the screen,
but then your focus would be more so on the messaging layer, because that's in charge of
like transporting the message that you would need to, like, for these agents to come in and share
the data and join the broadcast through either pub sub or request response or whatever it is.
So yeah, in that sense, if you want to automate it, I would say that all of the enhancement would
be done in that layer. Thank you. Mm-hmm.
Great, well, thank you very much for this insightful talk.
Okay.