Hello everyone, thank you everyone for joining my name is Andrea Andreshan and today I will
try to guide you through what I've done but with the cloth on the hard work testing
first structure. So first this is the outline of my presentation. I will guide you through
the steps where I used cloth. Everything started actually from scripts that I implemented
and was like let's see what I can do with the eye to reproduce what I've already done.
So the purpose was to automate and to end harder validation for a specific analogue device
is hard to achieve. There is a set of development boards that use
yokto images on base. I think you can run different images but I put it yokto on them
and for the testing pipeline where I used cloth first you I didn't find the flash project
by checking the TFTPE directory for work configuration then I detect the work connection
where the board is connected to can run the test on it and after that run the hardware test
and also generate test rubric that will be uploaded as artifacts in GitHub or on our specific
artifact storage site. The key benefit of this hardware infrastructure is that you have a
fully automated board discovery and validation with no manual configuration.
So these are the steps is like a chain because for step two you need the output from the previous step
for step three from the second and that's going on. As input we need the TFTPE boot directory
to check what was the setup that we flashed and booted then we need the available serial ports
and also a board configuration JSON but you will see that I have two cases. One where I detect the
you weren't a connection of the board and one where I give it a config file to know exactly which one is
and as outputs the output from the previous step and then as a whole a mark down test rubric
the first step so the check of TFTP boot is done by cloud so everything is prompting
you here and here. I will show you just what cloud is actually does and I told him hey
check this you will see in the next slides that I have there actually prompts that task that I
told him to do but the first step as a pipeline is to check and detect the board that I've connected.
The thing is that you can have multiple boards because if you want to have an entire system you can have different
ADSP boards connected and this identification is made through the U-boot file description where you have the
boards name in it on it is this one and identified this sum as a kit setup. So this are the files that you found in the
TFTP boot folder and you just detect the board. This are some examples of the boards that we
support with this specific cloud context. Then after we connected what board was flashed we
want to check the what connection to the machine. You may ask why I don't use I don't know the SSH
connection to the board it's because in your network you may have multiple boards and if you
want to connect with SSH maybe you run tests on the harder connected on your other teammate
or something but if you run it on your UART connection from your machine then you will be sure that
is the board that you flashed before implemented. This is also like cloud will just list the
UART available boards. Here I had two boards connected to my machine so it found USB zero and USB one.
And then with a serial import is a best script that also generated it checks based on the
host name of the thing of the board connected on the UART connection it will discover which one
is the board because I can have multiple like a 594 and a 598 board but if I flashed the 594 then I want to test that one
because maybe on the 598 is another e-mage word version that was previously theft.
Here are some results from Iran. I told you I detected two boards and on the USB zero connection was found
the 594 setup. For board identification from the previous step this is the host name and will basically
match the detected board with the UART connection. Then it will try to run some harder tests on it.
So this is a cloud context created specific for this setup. So previously I talked with him I
gave it some best scripts that I had with test and told him hey this is the generic one and for this
setup add some extra some extra test that were not implemented and it created a Python framework testing infrastructure.
So what I had previously was some best scripts. So it will try to connect the board, authentificate
and then run with the UART connection, some generic tests that include like serial, E2C,
crypto or ashram. Sorry. This are like some generic test executed on the board. I don't know exactly to
told you more about it because I'm just the person that just included them in the pipeline. So it's not something that I know.
This are some results of the test generated, of the test run with cloud and also this is the output that it prompts in your CI or on your session with cloud
internal. But oh, this is sorry. This is the detailed output. What's nice is for failed tests for failed tests.
It also can give you some possible causes of why the test failed.
Maybe for persons that interact with the hardware and they know more about it. It's not that much, but for an engineer that all we want to do is to boot the board and he doesn't know
the steps exactly and why it probably failed. It's actually nice to have a hint at first to know what to start to debug or something.
I told you it generates a test rubric that you can export and keep somewhere with the results.
It's not much, it's just the test. This are the supported boards that the context that I've created and worked with it supports.
And as you can see for specific setups has tests that run just on those ones.
You can connect to the board with a board rate credentials.
You can find this on our GitHub, is there a documentation and that's what I've used to create the context.
And use it to support this.
So after I tried this in a terminal session with Claude, I was like, how would sound to you add this in a workflow on GitHub on the workflow that I've worked with?
It's like build and flash and then testing.
I also kept the tests that I've had in Bash and I just added the Claude ones.
So for the testing part is just the checkout of the repo and the context because I want to have the context that I created that I know it's working.
And the one that it's not necessarily modified by previous runs.
And all I need to do is check out the repo's context.
We boot the board to be sure it's on a neutral stage.
Download the artifacts, then flash and boot the board.
Then I will run some tests with Claude.
This one is that also does the hardware identification and this stage is just running some tests where I give it to him the configuration file.
So this will be the first stage and it's really simple because all you need to do is just source.
I actually think we boot the board first to be sure it's on the boot mode that needs to be.
And then just give it like Claude.
Hey, identify board type from the TFTP boot files.
And it will identify what I've booted and flashed before.
Then you say, hi, find board on the specific USB connection.
This way you don't need to do anything just pre-bron, push and Claude will do that for you.
What's nice is if you have the context locally, you just open a terminal and say,
You can do all this process from, I don't know, building the octo image, flashing, booting and also testing with one terminal session of cloud.
But for all of these, you just need the proper context to be created and to be sure it's working it.
And this is the second stage where I told you, this was the first one that I created because first I just give it him by default the USB and board connection.
And then I was like, okay, let's see how it's working to find itself, what board and what connection, where is connected to, to know where to run the result, to run the test.
But basically, from a prompt engineering, it's pretty easy to just open a terminal and say, hi, just boot the specific setup that I have here connected to mine.
To my machine and not open like GDB, open OCD and copy all this in the TFDB boot.
This everything can be done with, with Claude.
So for this, I actually use two different contexts. I have the first one, which actually is context two, sorry for that.
Which I created with where I give him, what I give him specific the connection.
And then upon that, I tried to make him detect the board and work to run the test.
This is an example of the workflow run, so it works fine.
You can, I mean, I can do parallel because I use NFS boot for the full image yokto.
So it's shared and I can use the 594 with root of FAS4598 because one work.
So it's sequential pipeline.
But in the end, Claude successfully identified the board, executed test, VI, UART connection, and the results were captured in the workflow blog and also in MDF.
These are the steps. Actually, it's like everything done.
I mean, you can done this without a very good knowledge of how to run the test.
Compared with what I've, with my experience when I try to add this in the pipeline, it's pretty easy to just open a session and then prompt him, tell him, hey, just do that.
And this is the CICD integration summary.
It identified the hardware and automated testing the boards.
This are like the main, the main script that Claude wrote.
Thanks for watching.