Hello everyone, my name is Marin Parich and I'm a staff engineer.
Laura?
All right.
Hi everyone.
Nice to see such a nice turnout.
Yeah, my name is Marin Parich and I'm a staff engineer at I do.
Social enterprise.
We make a learning platform for children and low and middle income countries.
But I'm not here to talk about that.
I'm here to talk about property based testing.
I'd like to ask you to raise your hand if you've used property based testing before.
All right, a fair few.
And if you use property based testing regularly in your work.
All right.
So I can see that this talk might be useful for at least some of you, which is great.
But I'm also happy to see that some of you have familiarity with it.
So property based testing.
I'll contrast which what I'll call example based testing.
Which I think is the conventional way that we write unit tests.
If we write unit tests, I mean, I hope we all write unit tests, you know.
But yeah, so conventionally you set up specific scenarios, specific expected outputs and for specific inputs.
And you kind of confirm every case.
So here we have a test written in Python, which tests a function that reverses a string.
And we have two test cases here.
We cover the edge case, but including the empty string there, good job us.
It can be challenging with example based testing to come up with all the edge cases.
And this is one thing that property based testing helps with.
Property based testing is also more fun.
It's more fun to add these tests.
So in property based testing, what you do is you specify properties rather than.
Rather than specific scenarios that you specify general truth or invariance that you told for all inputs or that you told under certain conditions.
And then the inputs for your tests are automatically generated and your tests will run hundreds or thousands of times with these inputs.
And there's a little example here, again, Python with library called hypothesis.
Where we are also testing reversing a string, but testing it hundreds and thousands of times.
Where we're just testing that if we reverse a string twice, we get the original string back.
So this is kind of a property of a reverse function that we're testing that we're exploiting here in the test that it's reversible if we invoke a twice.
And that's what we can do with a property based test here.
So when the property based test runs, what we see is that firstly input is generated by the testing framework.
And then the property that we've set up in this case that the function is reversible is checked.
And if the test passes, we just go back, we generate more inputs, we do this hundreds of times, we do this thousands of times before we satisfy and say that test is green, the test is passed.
But if the test fails, there's an additional clever step that property based testing.
And that makes it even more useful, which is shrinking the input.
So the input will be simplified, for example, if it's a string, but for a shorter string.
And an attempt to still find examples that are simpler or shorter, that still failed the test, where the property still doesn't hold.
And in the end, after shrinking a number of times, the failed test you get actually will show you the last value that it was shrunk with.
So that you instead of like a really long, horrible string that initially failed your test to get the shortest possible one that the system was able to find that failed your test.
So it's not a little bit about input generation and property based testing.
So there is some randomness involved in generating these tests at these inputs, but it's not uniform.
Special edge cases are always considered, which means that we as a programmer don't have to remember them for strings that might be the empty string, it might be non-asky characters.
For real numbers, that might be between tricky values, like the infinities, not a number.
For a list, that might be including duplicate items in your list, because if you may say make a list of 32-bit integers at random, then at random there's value going to be any duplicates in it.
Because there's so many integers, but in a property based testing framework, lists will also contain duplicates, so you can test your functions with that.
And you can combine these generators to produce more complex data.
Some testing frameworks call them generators, all of them arbitraries, but it's the same thing.
We need to specify which inputs we want, and then the testing framework generates these.
And when it fails, we also get the random seed that was used to run the tests, so that we can be sure that we can reproduce these failing results.
So let's talk a little bit about properties.
So we can consider properties as invariants.
So what holds two across all inputs, or across the class of inputs, when we invoke a function, what doesn't change after we apply the function on a test.
But anything can be a property, we can test in theory anything that we can test with an example based test, also with a property based test.
We just have to come up with a particular property of our system under test.
So generally, for some inputs, maybe satisfying some precondition to narrow it down to a class of inputs, we specify a predicate.
And for example, a predicate might be if we're testing a function that concatenates two strings, that when we take the length of the concatenation, that should be the sum of the length of the two strings.
And I'll go over some more handy properties that we can identify, because part of the trick to property based testing is finding ways to apply it.
Finding properties of your systems or functions that you can test in this way.
And you can take some inspiration from maths there, so a lot of these properties come from mathematics, but they apply to all sorts of things, not just mathematical constructs, but various more practical situations.
So we have this two activity, which I just mentioned, this we'll be doing over the length of strings, we can also do this with any function that say transforms the strings.
So if we take the upper case of two strings and we combine that, that should be the same as combining them, then taking the upper case.
We have commutativity, I mean it's important if you say building a big integral library or something, that you have to check that your addition is actually commutative.
And it's nice to do that with property based testing as opposed to just writing out three examples and that being satisfied.
But it also comes up in other contexts, like when we have an image and we flip it once horizontally and then vertically, that should be the same as flipping it vertically and then horizontally.
Or if we have some data pipeline, be that in a collection or a database or something else.
If we filter and then sort or if we've sort and then filter, I would say that should again be the same.
Yeah, so another property from maths again, associativity, kind of applied to concatenating strings, kind of applied to concatenating lists as well.
And then some functions are invertible, like we saw, we can reverse a string, reverse a twice, we get the same thing, we flip an image twice, we get the same thing.
And then some functions that have a natural counterpart. So if you're writing a serialization engine, then of course you can serialize and deserialize and confirm that you get the same data on the round trip.
You can do this with encryption, decryption, you can do it with compression, decompression, if your compression is lossless.
If you have some kind of editing functionality and you have an undo redo functionality, you can also apply it there, you can just perform some random actions and undo them and you should get the same state back.
So all this you can state in a property based test, these are all like possible properties.
Item currency of course also is a super common one in computing, the classic example would be sorting, sorting a list twice should give you the same output as sorting it just once.
And maybe end of my time. I have compiled also for some popular languages list of property based testing libraries that go with them.
So if you want to try it out and you see a language of choice here, then yeah, give it a try. It's fun.
And I think at this point I'll have time for a couple of questions, maybe.
Thank you. Thank you so much for listening. Yes.
You're least sure with a lot of programming languages. We have also all the tools for generators when I'm using all the languages which are not here.
So your question is, there's a lot of programming languages in the list and you're asking for a programming languages that aren't in the list.
Yeah, but I have all the languages like we are here and I need to generate those.
To do the in the loop, but using some all all testing in the background, you know who is doing that.
I'm not sure I get the question. I'm sorry. Let's talk after. Sorry.
One more question. Yes, please.
Yes, in conjunction with example based testing. Yes, in conjunction with. It's always good to write out a few example based tests.
There can be nice food documentation. Like you can see how exactly you want your API to work.
And there are certain things that aren't handling the cover property based tests, but there's some things that property based tests do much better.
And they can also use to document these properties that your system might have. So they pair nicely with each other.
All right then. Thank you.