Okay, we can start hello, my name is Genia and today we're going to talk about mobility
of virtual machines in Kubernetes clusters and especially about storage live migration and
cross cluster live migration for your virtual machines we will cover how and why in features and
requirements and we will see a short demo first of all how many of you familiar with cube
verted self like do you know what cube that does yeah most of you okay for those who don't
know cube it like we run virtual machines in Kubernetes and how many of you familiar with live
migrations and we talk about cube version okay so let me explain usually when we talk about live migration
in Kubernetes we mean compute live migration it's been there like since the beginning of
Kubernetes and it means that your virtual machine remains running while your virtual machine
instance is moving from one node to another and one of the requirements for this type of live migration
is that your VM must use a shared storage means your VMs PVC needs to have read right many
access mode because at some point at some point you need to have two virtual launcher pods that will
be connected to the same PVC so and in this case we have one VMI virtual machine instance
that whose job is to synchronize between the source and the target and this VMI serves as
the point of truth for the for this synchronization remember this you will need this information
in five minutes but I'm from the storage team and we were like everyone talks about live migration
but we were not invited to the party and so we invited ourselves and we heard feedback from
customer from abusers who say I want to live migrate my VM but it uses some old storage
with read rate once file system PVC but my VMs are so great I want to keep them running and they say
okay we might have a solution for you so we said we can do storage live migration so when we do storage
live migration your VMI move still moves from one node to another but also your disk data needs to
be copied from source to the target so it means we removed and it means that there is no point in
time when you have two pods who need to use the same PVC so it means that even if your VM has
simple storage with read rate once access mode it still can be live migrated to another node
and this will work with for any combination of volume mode and access mode so you may have a
local storage with read rate once file system and then you want to upgrade this storage like
you buy a new storage you install it and it has like read rate money block you can migrate your VMs
to use this new storage so like this is helpful like when you upgrade your infrastructure
when you want to rebalance your storage and when you want to retire your legacy storage solutions
and adopt new new ones. Now let me show you how would you do it like if you want to storage life
migrate one VM so we have a user guide you you can find instructions there but in general you would
need to update your keyword CR you may also need to add the feature gate for disk expansion if
you're using cubic most likely already have it because there's very few things you can do without it
so but the things are related to storage life migration like throughout strategy and the
workload update methods so you update your cubic you create your destination data volume with the
blank source and your target storage class name you can not specify the storage class name it
then we will use just your diff whatever you have default in the cluster and then you need to update
your VM spec just add the update volume strategy and new new PVCs or data volumes names once you
updated your VM spec you will see that virtual machine instance migration resource had created
and your VM is started migrating so and in the end you will see your VMI moved to another node
and you have your new PVC and your VM is using this new PVC and we still keep in the old PVC
you can you can go and clean it up manually and so this was about migrating one VM like if you want
to do it in bulk we already have orchestrator for that it's available in cube width it's super fresh
from the oven you just need to install the cube width migration operator it will install the
controller and all the CRDs you need and you would just need to create migration plan in this case
I'm showing you multi namespace migration plan so you can list the namespaces and the virtual
machines inside those namespaces and the disks that you want to migrate in the part of where destination
PVC you can specify the storage class name volume mode access mode whatever you want
and just create the migration resource once you create the migration resource the migration
will start and migration resource will report the progress and the plan also will show you how how
is it going okay so we migrated our VM to VMs to the new storage but then it looked too simple
and you got feedback from the users who say yeah how great in the storage is nice but
what if I want to move my VM to another cluster and they're like okay moving to another cluster
sounds like a challenge but sounds like a very nice one an interesting one to solve so
generally can help you with your upgrades with again with your load balancing
infrastructure consolidation if you have multi-cluster environment you can just easily move your VMs
between the clusters and one of use cases like the most painful use cases is if you have a single
load cluster and you want to keep your VMs running without cluster of migration you wouldn't be able
to do any maintenance on this single load cluster without stopping the VM so today this cross
cluster live migration you can move your VM to another cluster do whatever maintenance you want
to do in your source cluster and move it back or keep it in the new cluster like so
live becomes easier and you have more options of how to keep your VMs running
so this was divide and now let's talk about the essence of the cross cluster live migration
so it appears that cross cluster live migration is just a variation of storage live migration
let me explain why you still need to move your VMI from one node to another you still need to
move your storage from the source to the target and in this case it just appears that your target
node resides in another cluster so we say okay we're a simple storage guys this is storage
migration everywhere let's let's do a cross the clusters so how we did it
leave it and give you give you solutions to live migrate your compute and storage and this is
not new and this is already used in different virtual in different virtualization solutions like
open stack and it's used for compute live migration but in our case we had a problem we didn't
we need to somehow synchronize between the source and the target and the source and the
target are not even in one cluster because you remember when we talk about compute live migration
you have VMI which is one VMI and one VMI can do it all it can access the the source
the target and do this synchronization so what we did like the essentially the innovation bits that we
added to the Kubernetes is the synchronization controller which is now responsible for synchronizing
between the source and the target so we invented the centralized live migration that's how we
called it so with the centralized live migration you know how to VMI you have two virtual machine
instance migrations and you have a simple controller that serves as a source of truth between the
source and destination VMI and so generally you need to and very important part of this is
that your weird handlers need to be connected through migration network and your sync controllers
need to be able to talk to each other through migration network so now let's see how would we do it
if you want to migrate our VM again if you have a user guide you you can find the little steps there
but then in what we need to do in cubordial we need to activate the centralized live migration
feature gate once you activate the feature gate you see that your virtual synchronization
controller pods appear in your cluster and like they you need to wait for them to start training
and another thing you need to specify the migration network like network deserves a talk on
itself and this talk happened here and now we go and we get it here he explained it he explained how to
how you can do it so once you update the migration network you will see that your virtual handler pods
will restart and now they will they will know that they need to use this network for communication
with other virtual controllers pods for like migration itself okay and then
we would go to the target cluster and create a target virtual machine resource
the best way would be to copy the source VM spec to replace the network and disks and
config maps and secrets whatever you need so that so that it will match what you have in your target
cluster then the most important part is to set the run strategies wait as receiver wait as receiver
is a special run strategy which means that once you create your VM it does not start training
it just it creates VM it creates VM i but it's waiting for the migration to start and waiting
to receive the data and you also need to add the annotation for restore run strategy so once your migration
will be finished you will get the desired run strategy like a normal run strategy of VM it's like
it can be always run a fader or manual next step we are still in the target cluster and in the
target cluster we would need to create virtual machine instance migration resource this one looks
pretty simple like name namespace migration ideas whatever idea you want VM i name your target VM i
name super easy but now let's go to the source your source virtual machine instance migration needs to
get like say the same migration ID the VM i name and it needs to get the connect URL so this is the address
that this is the synchronization address for your virtual machine instance migration to communicate
with the other virtual handler pods in the in the target cluster so you can you can get
this synchronization address from two places one is cubir tiar you have this synchronization address
in cubir tiar status or another place if you don't have access to cubir tiar you can do you can
look at it at the target VM i am which we created one step back so once we created it will get
the synchronization address in its in its YAML so you can take it from the take it from there so
it will be just IP address and and the port and you just need to copy it here
okay now requirements of course your two clusters need to be somehow connected it can be L2
L3 network because your synchronization controllers need to communicate with each other somehow
and your handlers need to communicate with each other also your clusters cluster no like source
and target nodes need to have compatible node architectures and VM CPU models it generally
it requires for every live migration but for crossplaster it's like especially worth mentioning
and also you need to create or make sure that your target cluster has all the resources
that your VM requires so if your VM uses like vgpu make sure that you have it in the target
or you can do an orchestrator who can do it for you actually have an orchestrator I will talk
a bit later about it and another thing that you need to do is you need to
is to make your clusters trust each other so each of your clusters has a ubered instance
and each ubered instance has its own certificate authority you need to exchange the certificate
between the source and the target cluster so they will exchange the data you can find
the info in the user guide how to do it like it's super easy yeah and every time
you talk about security it's super easy no limitations we would not live my great storage or crosscluster
this that are shareable and shareable I'm not talking about read rate minorities I'm talking
about read rate 90 plus specially configured that this disk has several writers like you can do it
in pub weer you need to like a special set up for it but we cannot guarantee that we will keep your data
consistent if you have multiple writers then we also do not support for systems devices
do not be confused we confused with the file system volume mode it's not the same it's different
it's like for system devices it devices that you list under like the main devices for system
different things and also landings because we can not guarantee that you have the same lands in the target
and now let's see how little demo to short demo it will not be as impressive as we get
demo I'm not like sending pinfields just network guys they send the packages versus the package
yes we cannot we cannot like lose the packages we are chilling storage we just don't lose the data
no okay I'm not so chill but okay so you can see on the left is my source cluster on the target
on the left is my source cluster on the right is my target cluster we created virtual machine
it has a data volume it's now it now pulls image from registry it also has a network
I've already done it for this
I've already done it for this to import
I just connect to the console and we write some super simple file
a little special
now we got to the target cluster I already have VM YAMO prepared for this
so I create my target VM you see that VMI target VMI also created it still has waiting for
sync status so we're just sitting there doing nothing waiting
and you can see that I have the sanitation for the restaurant restore and strategy
and my run strategy is very to the receiver
and you see that the data volume also got created but it's not populated it's still waiting for
for consumer and we create the VMI
VMI sorry
and now we want to get the the connect URL from the from the cube with CR
and our source VMI
has this connect URL so we created and you see the migration started
this is the target VMI I am started scheduling
I'm the source VM is still running
and now we connect to the console in the target cluster
now we connect to the console in the target cluster
and you see it didn't even ask us for password
so generally you can watch your YouTube on the same VM and it will be migrated across the clusters
and you wouldn't even notice
so let's eat with demo
now let's see and I talked about orchestrator
the risk orchestrator for bulk cross cross cluster migrations
it's available and open source it's called the forklift
and it also has nice UI you can you can try it
and then the created for developing the feature go to a leecher, the Xander Viles, Alex Kelenuk,
the CD and the stems to Miguel for demo setup
and stems to community, keyboard community for reviews and acceptance
and thank you for watching it