Okay, I take it to 11 o'clock, so I'm going to make a start if I'm okay with people.
Keep it down, because I don't have a microphone for you guys.
Okay, so I'm on.
Thank you.
So, I trust we'll stop by for a moment and attempt to change some stuff in the world.
We'll see how we get on.
First of all, I'd like to say a great talk by Immonol.
We're working in exactly the same area.
He's way more technical than me, so there won't be any statistics in this.
But we do fully support the concept of statistics for assessing software.
So, just a very brief thing.
I hope is known to quite a few of you, we're specialising in open source.
We do a lot of software engineering for international scale clients.
And we tend to recommend open source for everything.
That means that we have lots of friends, but also we make some enemies, particularly in the proprietary world.
So, here's a question that we've ended up crystallising, particularly in last few years.
And as we ask it, people sort of assume that this is a stupid question, but it's not.
I would like you to all take away from this talk.
Why are we trusting software?
And at the bottom there, this is the actual official way to install Rust.
Now, I'm not calling out the Rust community, Rust is one of the most sensible communities in terms of protecting supply chain.
But to install Rust, you are surrendering control of your machine and you're pulling some stuff from the internet and you're saying, here you go.
Okay, so you are trusting the Rust community and you're trusting that command line when you do it.
Right? Many software programs get installed exactly that way.
It's a risk every time you do that. Right? Who are the people behind that thing that you're trusting?
Now, our journey on this specific topic got started in 2016.
We were asked by Bosch, how open source could help with safety.
Now, Bosch, for those who don't know, has made billions of breaking systems for cars.
So they're pretty good at safety.
The question came up because people in Rust were starting to realize that the systems are getting much more complex.
Historically, they could put all of the stuff together themselves.
They could design silicon, design hardware, write software all the way up, turnkey system, bang, Bosch certified.
Okay? But then customers are demanding, oh, can you use Q and X? Can you use this third party silicon? Can you work with this board?
Can you integrate all these other suppliers? That's a very different risk profile and it's much harder.
They started thinking, well, how do people achieve that kind of interaction between all these different things?
Oh, open source is pretty good at that. Let's have a go.
So we started research project and the answer very quickly, we stopped the research project because, first of all, there was no overlap between safety and open source.
That may not be obvious to you. Please think about it. This is the one bastion of human activity where high tech is concerned, where it's still not open source.
It's mainly proprietary solutions, secret behind closed doors. Very, very strange. Don't you think?
Right? But the answer that came back was you can't use open source for safety. This was from other guys in Bosch.
And the reason is, it doesn't comply with standards. It doesn't have requirements, so you can't say where it's supposed to be doing.
It doesn't have an architecture, so you can't say that you've designed something to satisfy their requirements, and it has no traceability to say what the software is doing.
Okay? So that is enough to stop open source in its track versus safety.
Now, the standards. That's some of the standards that we're aware of, the deal with safety.
Some of them are all linked to this 61, 508, which I've mentioned. That's a well-established standard.
Over 20 years of history and it worked well for a whole lot of industries, but it spawned these child standards in different industries.
Coating for better or worse has been dragged kicking and screaming into the automotive industry, and then into autonomous vehicles.
Right? And years ago, in discussion with my colleagues, some of whom are here, the thought was, do we really think this is safe? Probably not.
But we'd rather be in there finding out what people are thinking of doing than just standing on the side and watching the proprietary guys make this stuff. So we got involved.
Now, you've all seen that I hope. The underlying tech tenet here is we're trying to get to a change in how this standardization process works, so the people can actually see what's going on, particularly with regard to safety.
Right? So that's the XKCD view, and we are in danger of just creating yet another idea, which in this context is the trustable software framework, to compete with all of the other stuff that people will be working on for decades.
Okay? Very little chance of success, but hopefully I might at least engage some people to think about the issues and why we're trying to do it.
Okay, so here's one thing. Are you aware that if you want to be a safety expert, you're going to have to spend $5,000 to read this document, and if you buy it, you can't share it with your colleagues.
We paid that money, and accidentally our finance director bought the document, and then we were told that the rest of us couldn't read it.
So in public on what was Twitter, which I no longer have anything to do with.
I called them out and said, you know, this is a Ponzi scheme. You're asking us to pay for work, which is done by experts for free, but you're charging $5,000 for it. So we got our money back.
Then we had to buy the 262 standard, which is, so we have, in the end, paid for a copy of the standard, but you know, this is ridiculous.
The licensing of it and the cost of it, this is about public safety. Why is this also secret?
So I just want to dig a bit deeper into one of these standard, the automotive one, which we've been wrestling with now for quite a long time.
They have this fundamental assumption in 262, that if you are where you were software, if you do it right, if you have your architect, you call it a properly, you test it properly, then it will be perfect, it will never fail. Fantastic.
That was true back then when those guys were working on their software before get existed and before Linux existed, but that's not the world we are in now.
We do not have deterministic software as in an old argued very coherently.
The trouble with that is that assumption means that everything in that standard is about process.
It's not about how good the software is, really, it's about how you follow a process.
Literally, argue that you can't use Linux and open source rather you should write the whole thing from scratch.
Lood across.
And to be clear, the processes they are advocating aren't used in the world anymore, in most of the world.
So you're either working at child or open source or both in most organizations that innovate software.
Whereas, these guys are talking about the V-Model. I'm not going to bore you with that, but basically it's the 90s came back and demanded that we still do it their way.
So no open source. I'm going to keep skipping forward because I have lots of slides.
So we're not in Kansas anymore, that's the Wizard of Oz quote.
This is ISO 262, I'm sorry, I'm off camera now.
We're down with a microcontroller up to 10,000 lines of code. That was the kind of systems these guys were wrestling with.
And I don't want to criticize the work that went into this.
For that kind of software where you're doing from scratch, great makes perfect sense.
That's not the world we're in now. We're in non-deterministic hardware because anyone that's spent any time with microcontroll processes will know that you have caching.
You have all kinds of optimized firmware, and the result is, it's not deterministic at the hardware level.
So how do you think your software is going to be deterministic on top of it?
And then we're in the situation with some of our customers, where it's not just a micro-process, it's a multi-core micro-process.
But then there's multiple boxes with micro-processes in them, so we're dealing with all of the complexity of that.
Again, not deterministic. Get it through your heads.
So here's our assumption. We have to accept that, yes, we can process why screw it up.
But we can also just experience random behavior that we weren't expecting.
100% code coverage, fantastic in concept, broken in practice. It's never worth doing.
We are dealing with all of this caching stuff, and then all of the millions of lines of code that are in our supply chain,
the compiler stuff, all of the dependencies, everything.
It is ridiculous to think that you're going to insist on specs, architecture, and guaranteed process for all of that,
and as a result, you will have something you know what it does.
Now, this 615 white standard makes these demands in terms of failure rates.
And it's good. We've got some numbers. The trouble is, as I understand it, and maybe there are people in this audience who could correct me,
nobody actually achieves those standards.
This, 10 to the minus 8 to 10 to the minus 7 failures per hour.
In effect, is one dangerous failure in 140 years. Who is going to be here testing for that long?
It's not going to happen. It's a very hard target.
And as I understand it, what really happens in practice is people get scared of those numbers, and they say,
oh, well, I'll just follow the process then.
If you follow the process, you're compliant, you get your search without actually achieving the target.
So, this is an old topic. I've been banging on about this in private and in public, to some extent.
Since 2016, so I created a public mailing list, which still existed, it's been very quiet for a few years,
as we hung it down and tried to work out what we were doing.
But that's the email I wrote, and I basically spelled out as a call to action, to older people like me, from different communities.
Can we have a thought about what we actually need to do to get to promises for software at the kind of skill we're dealing with?
And no, we're not going to rewrite it all in rust.
Much as I would love to believe that we could do something like that.
So, how do we make promises for the software that exists, not the software that is in our mind's eye, kind of thing?
Right? So, that was the original call to arms that we would know where it comes from, how to build it, how to know what it does, how to know what it is supposed to do, and we can compare those two.
And also, we can make updates because as you know pointed out, we're in this process of rapid exchange in software.
So, the key thing we learned, and we did learn quite a lot more, there was lots of discussion.
I think my name is still there, please browse it your leisure if you're interested in history.
But the main thing was, if you want to trust software, then you will need some evidence ideally.
Right? You would want to see evidence, and then ideally, you'd rely on other people reviewing the evidence as well as you, and there would be some consensus.
Right? Now, here we are at the topic of the talk.
With approximately 20 minutes to go, almost, it's good. Sorry, I'll go back for that. So, trustable software framework. Let's be clear about why this word was chosen.
Trustable is different from trusted and trustworthy. Imagine if you're in China, and you're offered software from America at the moment.
Or vice versa? Imagine if you're in Europe, you know, offered software from some Middle Eastern country, whatever it is, right? Providence does matter.
I would love to say that we're all human beings, why can't we all just get along, but there are preexisting biases, okay?
And most of us would think differently about software depending on its provenance.
You know, if it's been created by people I know and trust and love, I'm going to treat that with a different view.
So, provenance is a crucial thing, but also, I want the evidence to exist, rather than you telling me, I mean, I'm a bugger for this.
I go around the world saying, I've not lied in business, right? I haven't ripped people off, and I halt to that, right?
But does that mean I'm trustworthy? You've got to decide. And you might want to check the evidence of that, talk to my former colleagues or whatever, right?
So, trustable means give the evidence and then let the recipient decide whether to trust it, as opposed to a certain mind trustworthy, okay?
So, we recently announced that we're moving a lot of our stuff to the Eclipse Foundation, because, frankly, the Eclipse Foundation has a big network of trusted parties,
contributing and trying to advance the state of the out of open software.
And in particular, they're working a lot with the automotive companies around this software-defined vehicle topic, whatever that means.
So, we're migrating this work, the trustable software framework and some other stuff into Eclipse, and we'll be backing from that base.
Now, crucial thing about the trustable software framework, first of all, it's conceptual, but then there is some software, okay? So, these are the concepts.
So, you remember that call to action I wrote in 2016, we've treated slightly, but still the same things apply, and I reckon these will still apply in 10 or 20 years,
because they're fundamental stuff, it's not specifically about software.
We will need to know where it comes from, because we are going to be having to maintain it, we will need to be able to build it, reliably, and know what we built.
We will need to deal with changes, and as I said, you know, a ridiculous way to change in the kernel, but it's not just a kernel.
From our point of view, we're talking about software that's going into a vehicle, that's all of the kernel, the bootloader, and all of the system libraries and dependencies,
and all of the user and stuff, and then all the supply chain tool chain that was used to build all of that is what we would need to trust.
And all of that's going to be evolving rapidly. So, whatever we do, we'll need to be automated, we'll need a test strategy that is applied automatically on every single release of the software.
And we need to know what it does, and we need to know what it's supposed to do, and then we need to know what it must not do, as it was all of what it's supposed to do.
Okay? So, simplistically, you might start with a kernel and say, well, it must drive, and it must be a crash.
And then you can build down from that, you can start breaking down the individual things that would affect whether it'll crash or not.
Okay? Then whatever we've established at that basis for, this is what it's supposed to do, and what it must not do, we will need to test the hell out of it.
We will need to figure out how to get to a confidence level, because it won't be deterministic.
So, in an old, it's got a very advanced argument around some specific statistics.
We're working on complementary, on different statistics, but in the end, it's going to be a whole load of maths combined with test results to figure out how confident are we, that this thing really does what we think it does, and won't misbehave.
And in the end, all of that needs to be measured. So, all of those factors there, we're arguing, you want to trust something, you need to be able to measure all of that content, and iteratively, repeatedly, get to a confidence view.
Okay? Now, this is our open attempt to describe how you should get to confidence, thank you.
So, trust was offered for me, you can't see that, but you can see it on the website now.
The link is at the bottom, that's temporary, because it will go to a clip at some point, but the website is open.
The top row here, that's those six tenets, and then the next two rows are breaking down into some more detail of what will be required.
It's obviously focused, it's on the internet, you get better pictures that way.
Okay, so I'm just going to run through briefly. So, this left side is literally just about the supply chain stuff.
It's demonstrating, you know where your tools came from, where all the dependencies are, all of that stuff.
This is demonstrating, you can build it, and I saw recently the 3BSD project came with this phrase, zero trust builds, please, let's raise that topic, that's the thing.
You should be able to build all the stuff without roots, you should be able to build it without the internet, and you should be reproducible.
If you build the same thing three times and you get three different results, then you're doing it wrong.
This CICD for everything, so you will need to be able to react and take fixes from upstream, whether it's fixed releases or emergencies security updates.
So, whatever promises you're making beyond this, if you can't do that, while consuming all of those updates, again, you're not doing it right.
This is the expectations, and this is our equivalent to requirements. In effect, there's a whole lot of history around how you define requirements.
And our advisors in Exider have said, you know, after a full career, Jonathan more said, he's still not seen a great set of requirements from anybody, and that's over 40 years, right.
So, we're not talking, calling it requirements, we're calling it expectations, and the basic principle is, we should only define what really matters.
It creates thousands and thousands of requirements. Let's just focus on what really matters and try to get to the key promises that this software is supposed to be making.
That's the results, and I'm going to digress here, because it's a pick up on him and all the stuff, so he was asked.
Are you dealing with changes to the kernel, right? So, I'm not going to explain all of those graphs, but you can broadly see that we have some kind of distribution, we have a target here.
This is what we're trying to shed you on, and depending on the hardware, so we're starting with virtual and only left an Intel box in the middle, our box in the right.
We have different distributions of testing, but that data is being collected as we rev the kernel, and everything else, so the whole destroy is being reved, and we're building up a model of test data over time for all of the changes that we're having to adopt.
So, we're getting into a situation where we can understand how change is affecting the results and not just some one version of the software.
Confidence is about getting to a measurement, and that begins.
I'm still good for time, I think. I'm doing better than I thought.
I'd like to give a shout out to the doorstop project. It's a relatively small project, but this is an initiative start by a chapter who did a PhD around this topic, and he's one of the people who realized,
oh, proprietary nonsense. So doorstop isn't attempt to replace doors.
Dors is IBM's historic requirements management system, very complex, very proprietary, and I don't know what it costs now, but certainly historically very expensive.
This is just some Python code and some markdown and some Yamal, and it works really sensibly provided, you're thinking simple command line type,
an automated type, requirements management. We've based the resource software framework on that, all the way are actually improving for our use case with some specific tweaks.
The net result of the framework is getting to an automated report of all the evidence assessed numerically in effect.
So it's a measure of how confident we are in each of these topics, and the aim is that someone could drill into this and say, well, am I confident in the supply chain?
Have these folks really thought about what matters in the use case? Are they able to defend what they're doing and what could go wrong with that?
So you can drill into the numbers in each individual component.
And the idea is to get to trustable software equivalent to or like a credit score for a personal or normalization.
Now, we're not assuming that all software projects are going to adopt this anytime soon.
But if you think about the CRA, there's a requirement on a subset of the community, particularly the stewards and the manufacturers, to make an argument about how they're dealing with security best practice.
Okay, so some organizations that are supplying open source really should in our view get to grips with how they're going to make promises about the supply chain they're dealing with.
So this doesn't necessarily apply to every single individual project, but it could be applied and we are aiming to apply it at the aggregation, digital level kind of thing or at the system level for a vehicle in our case.
So as a side note, we're open sourcing, it's super, thank you. We're open sourcing a small open source project developed in Rust, shout out for Rust, which does safety monitoring.
So this, in effect, is a small user land program that will check various parameters, it's configurable and it will report on whether those values are being satisfied at the right frequency in effect.
There are various proprietary safety monitors where hoping that this can become a popular open source one. So it's a way of monitoring things happening in an alumni space system.
And we've literally only just plugged in the Rustable Software Framework to it, the work of the next few months will be to demonstrate how people get to grips with the Rustable Software Framework and get to some numbers that will then be maintained with the project.
Take away, it's fully open source, it's going to be a clips, it's mostly automatable. I would love to say it's entirely automatable, but it's not because at some point people have to review the stuff and decide whether they think that anything's been missed.
It's designed to complement existing processes, so we're not trying to change everyone's process and get them to switch to this instead. It should fit alongside what people are already doing.
And it's particularly applicable for existing software including open source. It's extensible, so one of the things we're doing in the background, we're explicitly mapping one of the standard 61, 508 and how I get use case two possible software frameworks.
So we're trying to get to a certification approach that allows us to over time gradually say, well, we don't need the standard because we're using Trustable. So that's a strategy.
And there's loads of work to do. So if anyone's interested in this, please do get in touch. Either via the list, which I'll give you the URL for a minute or email directly to me or get in touch with your thinking general or via issues on the project. So.
Thank you.
Okay. Yes, Philip.
I would see that if you say right down the expectations and check how the expectations are matched.
Yeah.
If this is one of the most review intensive tasks, correct?
Correct.
A small project or whatever is still when you need to have a lot of understanding in order to give such a credit score.
I call you a series here to manage these kinds of skills that you can enjoy and give a score.
Okay. So Philip's asking, given that this could be a lot of work, how would this be made to work for a small project?
And the answer is, I don't believe small projects should be going anywhere near this anytime soon.
The onuses on people who are trying to make promises around use of software to figure out how to make the promises for their whole solution.
So, in effect, we're mainly thinking about this holistically and not trying to drill into every single component, which in a way as a corollary of what one eminors saying,
trying to get to 100% code coverage is a full zero and trying to do this level of work on every single library.
I don't know what we're up to in our solution at the moment, but it must be of the order of a thousand different components.
Trying to do detailed assessment of what all of those components are doing is not going to work.
And I don't believe that the credit scoring does all of that level of data either.
I think in the end, you need acturaries, mathematicians, statistics, folks, figuring out how to get an overall sense of risk without having to drill into every single data.
I think, yes, one loss of quicker.
Okay.
So, what's up with you?
Yes.
Okay.
Yes.
So, the question is, do you know what you should have done?
I would like to find it's my average rate well.
Yes.
But I couldn't figure out, you know, from this floor.
Okay.
So, the question is specifically with aerospace, but it's more generally, how does this lead to certification?
And, yeah, there's going to be a body of people in any industry saying, well, okay.
Why should we trust this?
Okay.
As I understand it in most of the current certification approaches, that question is always used.
You know, how do you know you've done enough?
And the typical answer tends to be, as I've heard it is still down, we put the right people in the room.
They do the work they say they've done enough.
Right.
And then you check with the standards.
Okay.
So, this is not a solved problem.
We may still fail, but it's a genuine attempt to establish an open source way of doing this.
So, I would say it's a very good question.
Our aim with code thing is to achieve actual certifications.
We have one ACLD toolset for part of what we're doing in this regard.
The aim is more certifications that show that we can achieve this kind of approach and meet the standards.
And hopefully, deprecate the standards, frankly.
Okay.
I think that's enough.
Yep.
Thank you very much.