Ready to go. Hello, everyone. So I have come to talk to you today about Kavesveg with a GIS
sort of vibe. I'm assuming that I hardly anybody here knows anything about this subject.
So this is very much interested. Apart from a few potholes with the audience, of course.
So yes, I'm Wookie. I was a cave her before I was a free software nerd. So I started in 1987 when I went to university.
And for about the last 25 years I've been running the UK Kavesveg group. So at least in theory I know something about the subject.
So why do people survey Kavesveg? These cheerful looking people on the right hand side have just a skill of the whole load of passage and are looking very pleased of themselves.
And that's all very nice, but if they don't survey it we don't know where it is because of course it's all hidden underground.
So when we send in grant applications we call it science, but obviously mostly we all like maps here and you get pretty maps out of a lot of tedious caves of being.
This is kind of 1990's vintage and this is a rather new Ashania colour. So they're just people who are used to this. The top is a side view elevation.
One of them is an elevation and one is a plan. This is a mendic where all the rocks are about 45 degrees so there isn't really much difference between elevations and plans.
So what is Kavesveg? How do you survey Kavesveg? We get on your passage. You basically take a line, pick a point, pick another point at the prominent point on the wall or a big rock and just get down the passage like that.
It's extremely low tech and then you measure the distance to the walls and floor and roof and that's it.
So in comparison to any end up with a center line which you can plot. So alright, how do we actually do that? So basically when I started at least it was done with an actual tape measure, a compass to measure the magnetic angle and a kilometer to measure the vertical angle and you scribble it down a bit of paper.
And do a sketch round your of what you weather walls when and where the rocks are and that's the other thing.
And then you type it up into a program so it makes a little bit of sense so you end up with a vertical point around here.
So station 13 to 14 is 10 meters at 330 degrees minus 20 angle and then here's some notes on what the stations were and how far away the walls are.
And then here's your crappy sketch into something that makes some kind of sense and then later on you produce a nice pretty survey to publish in a journal.
Now you might imagine if you've seen people surveying by roads and the used teardolites and stuff on sticks and so why don't we do that underground that's been around for at least 30 40 50 years.
Well the main reason is the conditions underground so first you haven't been caving I've just got a couple of minutes to kind of give you an idea what the conditions we have to go surveying in are.
Results.
I think I can.
There we are that probably better.
I wouldn't like to have to be embarrassed.
I think I'm coming back.
I've done it enough.
It's a pity it's one of footwiser.
I hope you can hear what they're saying it's quite stunning.
It was just said I hope it's not full of water this is not full of water it could be worse.
That's the same chap.
I should thank this man for all this footage.
It's not asking about that but it's not asking about that but it's going to get out of this.
If you click the wrong thing it's quite a double click I think.
Sometimes it's nice but often it isn't sometimes it's huge sometimes it isn't so we stick to extremely low tech surveying techniques there is of course no GPS underground so you can't do that either.
It's not very accurate either so if you survey around at the same point there's often loops in case or you might survey to an entrance and then back to where you started if it's a through trip.
So this is a bit of a problem we have to kind of do some distribute the air is basically around the loop so pretty early on people started writing software to do this because you don't want to do my hand.
When I started it was programmable calculators out in the field so the earlier software I can find is by one Andy wanting to in 1982 it might be free software back in the days when people didn't put licenses on software they just kind of assumed people would use it.
Unfortunately means technically it's proprietary but he's dead now so we can probably just.
Nobody cares I mean it's probably a mainframe and then it was software for PCs and some max stuff and it's now I got started with service in 1991 I think it's safe to say that none of my code is still in the project.
But that has been surprisingly popular 35 years later quite a lot of people still use service so we must have done something right about decade after that so that's just center line processing all of those tunnel and theory and so about decade later in 2000s we started using software to actually do the drawing part as well as the center line processing part and again both of those projects pretty much run it on now tunnels kind of dead.
And there's loads more software I mean I'm only putting some of the projects on here so in the states as windows people.
It's probably software we don't care about that but compass is still going to be going for years almost all of these projects are one or two dedicated people doing a project.
And the quite region as well so walls was very popular with Americans going to Mexico visual topo also still running is pretty much the franca file caves are being software.
If you run windows there's Jim and stuff as another French project which was in turbo Pascal and somebody has to.
And there's new projects still coming on caveways quite recent QT based quite nice as he and there's even people still write non-free software don't want to wrong with them and hilariously.
You can now rent your survey software by the month the Ariane projects aimed at divers who will have too much money so you know who cares.
So this is all great except that we have like 30 different cave service software which all just invented a format and so none of them very few haven't taught to each other which means it's very difficult to move things between.
Once you pick some software it's really quite difficult to move to something else often some of them will read other projects like service will read compass and walls data pretty well these days.
Which is good because walls is dead so people are migrating and smaps is dead.
There was a massive project where we set a certain grand project to translate everything to everything sort of GIS people you know you get everything feel thing but never really going anywhere.
Cave converter is actually quite useful that does some formats.
Service and theory and so close but not quite the same that a fairly stupid script will do the job.
But eventually we realized like took a long time that there's real standards out there for 3D data and maybe we should use those instead and stop inventing our own crazy shit.
So back to the actual instruments we started off with action compass and plano.
But in about 2006 a Swiss guy called Beatheeb realized that you could take a standard like a distance measuring device.
So all these gobbins insides out and replace them with some sensors for the magnetic and angled stuff but use the box and the screen and the buttons and so on and the laser module which is a really hard part.
So and at the same time you also wrote some software to put on these windows, PDAs you could get in 2003 to draw.
So Bluetooth the data to the other device and then draw the cave on the screen.
And that was kind of kind of revolutionary. It took a while to catch on but it's more or less how most people do survey in these days because it's quicker and easier.
And remove quite a lot of opportunities for errors all the shouting out of numbers when you read the tape measure and shout it down to somebody over the waterfall and they wrote down a bit of paper.
And then you wrote times into a computer. Every one of those is an opportunity to make a mistake.
So this is the devices you bought your like a device through a way that gobbins and put oops don't wrong edges again backwards.
Put your little sticking board. Try this thing there this goes instead of that one.
So that has all the magnetic and accelerometer sensors on and you put a non-magnific battery in.
And this went pretty well for quite a long time cost you a few hundred pounds.
It was just about cheap enough that people or at least groups could afford to do it.
The problem of this game is that every time like a decide to bring out a new device you've got to redesign your gadget.
And after this device they made it more compact and more integrated to make it cheaper which meant there was no way to talk to the display.
It wasn't brought out to a separate bus you could just plug in.
So basically it became impossible to maintain and you couldn't get the parts anymore.
So for a while you couldn't get this dose.
So it just do X's.
Which was a bit of a problem because they're very valuable devices and people do of course drop them down pictures and fill them with water and generally stop them working.
So the man in this room in fact quite a long time ago started going well actually we could make her own.
So this 2008 was the Sheldon Attack Pony 2 which is the Compass and Plano part.
That's the rest of easy bit and then you just use the separate electronic.
Oh wow okay I sure I stand corrected.
And yeah we use those they were good.
And then there was the version 5 you could actually buy a laser module so you could make the whole device.
So this is all open hardware unlike everything you've seen so well which is proprietary software and proprietary hardware.
That was all right but the laser module wasn't great so it was a bit slow and the battery isn't as that long and it wasn't really good as any water anywhere.
So now we're on to this thing which is actually pretty good.
Better laser module and it does its own internal calibration which you have to do external and this is all marvelous.
And because this is an open project people have started taking it and experimenting so this is basically the same hardware and software.
In a better case with a lot more engineering as a result of which it cost nearly twice as much money but you know quite a lot more.
And we're going to experiment with that next month to see if it's any good.
Meanwhile the dystophing eventually so I spent an hour and a bus once with the at trying to persuade him that he should open up this software because this is tiny niche market.
And he sold a thousand and 20 years whatever it is or 2000 and you know maybe if we could help him with a software if we were allowed to.
But he wasn't having any of it but he did eventually publish the bomb after you couldn't build them anymore.
So a couple of people updated them and did do some later builds and some Chinese guys now making basically dystop's in China.
It's not particularly cheap but it's a nice bit of hardware.
I saw one last week they do exist but at the moment you have to go to China to buy one.
A little bit inconvenient.
So yeah that's coming along nicely so that's all very very nice and that's what most people do but of course what we'd really want to do is like walk down the caves.
Getting magic surveying from lighter obviously that's how it should work so we've been sort of desiring this for decades.
But I was around about 2014 I think this thing came out which is not Australian device which is so that's a 270 degree laser planar gadget mounted on top of an emotional IMU on a spring which is kind of weird you go why is it on a spring.
Well actually that's quite important part of the thing because IMU is a shit and forget where they are at every 30 seconds.
So in order to have that track basically as you flop the thing backwards and forwards on its spring the laser keeps coming back to the same point so you can keep recalibrating your IMU every couple of seconds which keeps it more or less in train.
And yeah it's quite good we took it caving it's surveys things nicely the problem is that the nine was it 14,000 pound a buy it was you have to pay for the data to be remotely processed by the cubic meter.
So it was extortion so only only rich archaeologists and professionals used it we tried to make a run by buying the two devices for about five grand turns out the sums are genuinely difficult.
So yes more like that so this is actually a proper expensive LIDAR ridial things like 25,000 dollars or something.
But with that you can go and survey big case so this is there's actually a project to survey all the massive caves in the world with the fancy LIDAR.
So they'd be dragging it around so you get all these laser points which show you which basically make a shape.
So that's radically different data to what we normally use now the bit of look I can show you.
So then you can take your point cloud and then color the points which is not what we want.
With actually I remember I'm a bit of a thing tonight.
Sorry about this.
I was hoping if I clicked on it you would take it there.
Then mine it's right there with me.
Should be a window I can't be prepared earlier.
So if you take photos you can color all your laser points with the right point on the photos.
So now you get this quite funky so this is a showcase of Germany and you can spin around and look at things.
So you get even though it's still just a point cloud you actually get kind of some idea of what the things like.
And this is a technology called Poetries.
So the point you've got millions of millions of millions of points and it's really quite difficult to manage.
You want a lot of points right next to you but you don't want all the billions of points off in the distance because it would just be too slow.
So Poetries is a kind of binary chop spatial thing which lets you view data like that successfully.
So you can then stick your photos around the edge just in the 3D model and make really quite snazzy pictures.
That's quite difficult to make stuff like this as a whole load of work involved but it's very nice.
So back to what my models are doing.
The little PDAs you could get in 2003 eventually became an obtainium and the old batteries died.
But obviously everybody had lots of Android devices.
So this sexy topo on the left is basically a rehabilitation of pocket topo for Android.
And then this is topo droid.
So those are the two free software programs that nearly everybody uses.
This topo droid is basically theory on Android.
The difference is that here you're just drawing lines around a center line.
It's just a picture whereas in this you're putting metadata in saying this line is a wall on this line is a pitch edge and this line is a whatever.
So you're doing more detail underground which takes a bit longer.
So it kind of depends how miserable your caving is which is where you want to go.
There's forever isn't no different any else software yet for cave surveying.
There's a few proprietary things but again we don't care about that.
So cave surveys have been looking like this for a long time.
So in about 100 years these aren't fundamentally very different.
And some people think this is bad.
So Julian thought I know what we wanted to do is draw in 3D.
Stop making it's basically all these surveys are a 2D flattening onto a plane to make a map which is fine.
But you know caves are very 3D things and it's quite difficult to get them onto a plop piece of paper and still be useful.
So using the god-o game engine you can take your point cloud, stick it up and then kind of draw rounds.
Just the same way we do a street map.
You draw down the road and then you draw around the edge of a field and say that's a field similar kind of technology.
And you end up with so you get away from the billions of dots which your brain's quite good at interpreting when you spin it round.
But you can't really print it out in a useful way.
So as this cuts it down to a plausible number of triangles so you can actually get kind of useful maps of it.
You need a VI headset to draw it but it's kind of fun.
So that's quite a cool idea.
So just go back a bit when we started back in 1989 we had this 10-ology which is basically being able to look at your caves.
So here's the Royal Centre lines that you drew and you get a nice spinny cave.
So that's 140 kilometers of cave and I can move it around quite quickly and I can look at well the entrances are.
All these ones down the bottom idiots who can't write down the altitude of their phones.
So zero obviously.
And we can put some terrain on so you can possibly see I get this right.
This cave can turn the entrance off because that's just confusing.
This cave is kind of in a ridge here and there's a lake down the bottom.
There was a little bit there you can see the lake here.
So this is this is cave is writing their own GIS it's kind of primitive GIS.
All you get is terrain data and your centre lines but it works and we've been using that for 20 years.
This is slightly snazzy a version of the same thing so now we can drop a map onto the terrain data model and have the caves underneath.
So this is a mendip in the UK so that's some you can see the caves and you can kind of see here.
There's a valley with a cave underneath and in fact we're going to drive this.
We can look underneath and go yeah there's caves under the under the ground.
Which is of course my point about the whole z axis thing maps are not flat.
Flat map for software that can't cope.
It seems everything's on the same level is a bit of a problem for caveers.
So I don't do that to be fair mostly people this problem appears to be largely fixed.
So we discovered after about 25 years that there was GIS software we could use to be fair.
It's only been available to mere mortals relatively recently.
So rather than us adding more and more GIS features to our crappy caves software which is very good at processing centre lines.
Maybe we should process centre lines then import the cave data into more generally capable software.
Where it becomes trivial to add contours and satellite images and terrain data models and all that stuff.
So that's what we started doing.
And obviously you can do this on your phone as well.
So on MSM and you can have all your cave entrance locations and then GPS tracks so you don't get lost on the mountain.
Well hopefully.
And we just started doing this using Qfield so in field stuff for recording new caves.
Because people are again especially on the expedition I want it's full of students.
People turn up they have no fucking clue what they're doing.
So they produce very poor data.
So as if you give them a little feel to fill in we've got a fighting chance that they will actually take a photo of the entrance and take a GPS point with an altitude.
And so on.
So if anyone's actually tried that I would be interested to hear of your experiences.
So we collect an off-loc cave data and we have to manage it.
So the thing called cavemaps.org really useful projects.
Probably been there for 12 more years now.
Someone took all the surveys.
So they're all flat surveys in the UK drawn in the kind of 50s, 60s, 70s, 80s.
All paper surveys.
So he went through, collected them all, scanned them all.
So you can now find, so it's called cavemaps.org because of it covered the whole world.
It actually covers Yorkshire.
It could cover more.
Anyway, it does a very good job of Yorkshire.
So you find you get how many surveys there are and you can get a list.
And then this is the 1949 survey of Mostell Caverns.
And then we can have the 1989 40 years later, which if you look actually matches at reasonably well.
They found some more cave, but it wasn't terrible to start with.
So that's all very good.
The problem is that those surveys, people did all this cave surveying.
I've tried to explain to you and they drew it up onto a nice paper and they published it in a journal.
And then they threw away the notes because it was finished.
And that's very annoying now when we want nice 3D models of all our caves with the actual central lines.
You go back to these people and go, where's the original data?
They get all I don't know.
Am I throwing it away?
Maybe it's in the loft.
Oh, they've died because it was 40 years ago.
So we can survey them again, but for some of these caves, that's pretty hard work.
So in 2005, the UK case of our group set up the cave registry to try and at least preserve the original data.
So people wouldn't just loot, they had somewhere to send it.
If somebody has died and they found all these useless cave notes, go, yes, please send it to us.
We can deal with that.
So we've been collecting data for 20 years for quite all the UK and various other projects around the world.
Actually, people used to be very proprietary about their data.
And they've gone through a lot of trouble to get it and they've published a map with it.
And they didn't want to give it to anybody because they might.
I don't know, published their own maps or look at it or something.
So we've had definitely, honestly, nearly everybody has understood that this is a very niche thing.
And there's no point keeping it to yourself.
And we should share.
There's still a few holdouts, they're annoying, but things are much improved.
Now, I haven't been very long to cover this, but I just thought also to talk about,
especially exploration projects, long-term ones.
You collect a lot of data over a long time and you get new people who don't know what they're doing every year,
or as we do.
And also, obviously, formats change.
You know, it used to be a compass and clienteau,
and increasingly we come to LiDy data.
And some of it, these days you tend to get a file of an Android device,
as you know, original data rather than some scribbly notes to scan.
So keeping track of all that.
And the problem is that people go back to a cave 25 years later,
and they want to find the end of the survey and carry on.
And they need to be able to find, well, where was it?
What was it?
How do I find the note?
What did people write down?
So people come back to base with some crappy notes like this,
and hopefully they write up a logbook entry, which might have a diagram,
some hard to read handwriting.
Obviously, increasingly getting anyone under 25 to write on paper is hopeless these days.
So you have to give them a web interface to type into.
You know, it's fine, we can do that.
But you know, organizing all this stuff over decades is a really interesting problem.
It's very heterogeneous.
So anyway, so we have a big Django website,
which tries to do things like like have all the people and all the trips they did,
and which surveys they produce.
Is there a survey for this trip?
And is there a logbook right up for this trip?
And then the colours match if you've got people on the same trip on the same day,
so you can tell they're on the same trip.
So that's sort of cross referencing is really useful.
And then you can find the original survey data from where this one is.
Here's a 2018 one.
And we have an index to all the caves people have found,
which should taste a little webpage, which should contain the info that's actually there.
So ideally there will be a survey,
and some sort of description telling you what this is.
There's an awful lot of crappy little caves in between the massive interesting ones,
but you don't want to keep finding the same entrance and exploring.
Have it to get a rope, often go down.
Checkers and go anywhere.
Hopefully someone took a photo with the entrance,
so when you get there you can tell that you're at the right damn cave,
but they often didn't,
especially again 30 years ago before GPS was much easier to lose your caves.
Or at least we have records of,
okay, we just don't know where it is.
And then you find them again, they get a new number.
And eventually 12 years later you work out is the same thing.
It's all good fun.
Anyway, that is basically it.
I will stop talking.
I could obviously go on for hours,
but that's enough for now.
You've got the general idea, I hope.
If anyone wants to talk to me about
the joys of using Q field and merging maps and merging big datasets,
or managing datasets,
I am interested, and there's other people around who are.
Thank you.
He's got any questions.
I really watch too many say-fame always,
but what about drones,
and so things that do it for you?
It's a good point.
I didn't mention that people have tried drones on the ground.
They work surprisingly badly,
so firstly there's no GPS,
and a lot of them rely on that to do a hover.
So we took a very expensive drone underground in Ireland.
The other problem is they suck themselves into the walls,
because of the Venturi effect.
So as soon as you get a bit closer to the wall,
it goes, oh, and smacked into the wall,
and falls down a shaft.
And now your expensive drone is down a big hole.
So we did that.
It bounced off the wall,
and we managed to catch it before the £1000 drone fell down the hole,
but we decided we should stop at that point and give up.
So there's the sort that have like bands around,
and that kind of works a bit better,
and you can, but it's a very expensive, again,
I think, Swiss drone designed for inspecting tunnels,
which has a carbon fiber shell round,
and is designed to work without a GPS signal.
But I've seen them use that on a national geographic glassier thing,
but again, it's 20 grand,
it's not accessible to me in mortals.
It could be done, I think.
Now they've got a lot cheaper.
There's probably software for just not doing the GPS thing,
but it's difficult to fly drone underground.
You can't see it to the other problem.
As soon as it's more than 20 yards away,
or if you're trying to go up there to look to see if I think,
I mean, we're really, really useful for things
when you can see there might be a hole in the ceiling,
40 meters up,
and it's an unbelievable amount of work
to go and get yourself up there.
People do that.
So no, yes, it is interesting.
People haven't had much success so far.
So my question is, how do you assess the quality of data
coming into the K registry?
Because obviously you've got pen and paper,
and you've got later data,
and you've got later data.
Like how do you know what data's good enough to press?
So, what the answer to the question is,
we don't care.
So each project manages,
we give them repository,
and they can do what they like in there.
The object is to not lose the data,
right?
So we don't care what format they keep it in,
how good it is,
what could care they take of it,
or any of that stuff.
We just give people basically a file store.
Now, obviously,
it is important if people are sharing data,
or other people's,
so we do have a grading system for cave surveys,
which is, again, designed 30 years ago,
in the age of obstacle instruments,
and really doesn't need an update actually,
because things are changing different now.
The new digital instruments are definitely better.
The accuracy of the instrument is similar,
it's probably a bit better,
but the main thing is,
there are dramatically fewer mistakes,
just because you're Bluetooth in the data,
and nobody's writing it down three times
on muddy bits of paper.
Do you export the data to open street work?
I'm not doing that,
but people are,
so I did mention the slides,
and there's a lot of cave entrances
in the UK and France are on open street map.
It's really, really useful.
It used to spend hours wandering around mountains,
and looking for entrances,
which you knew were over here somewhere,
and now you just take your phone with you,
without,
okay, I don't know,
okay, venture is marvelous.
So, yeah, I think we could do more of that.
It's not, I don't think it's been very organized so far,
and it's just people putting entrances.
I did remove one last week,
because Breitweight Hole has a cave entrance,
and there is an entrance in Breitweight Hole anymore.
There used to be 30 years ago, but it fell in.
So, yeah.
We could definitely do more, and it's good.
There's a slight problem that Americans hate the idea.
There's this long-standing problem that Americans protect their caves,
by not mentioning where the entrances are, and keeping them secret.
Everyone else in the world has horrible caves,
that don't really have a problem with people going in
and shooting the bats with shotguns and stuff.
So, so we don't have to worry about it.
So, Europeans just published entrance locations by large.
And there was a big fight on Wikipedia about 12 years ago
when Americans tried to insist that entrance locations were protected data.
Nobody could put them up.
And you go, fuck off.
It's all grown in.
You can keep yours, can't we?
They were trying to keep it out of the template.
That was it.
Very annoying.
I was thinking of the actual passage data as well.
Obviously, provided it doesn't get interrooting engines.
Interesting question how useful that is to everybody.
We could have got loads of passage data to put in.
I have done a little one in France where you could just walk in one entrance.
It was easy cave.
You could come out another one.
So, I put a path through.
I see somebody has removed it again.
I check that's night.
So, yeah.
Interesting question.
What people would?
I think we should ask those same people what they think.
We've got loads of data.
Rooting in caves is a thing which is not solved.
Because often there's more than one way to get there.
And it's highly variable how difficult it is and how much equipment you need.
We could actually, for that big complicated system I showed you,
we could really do with some rooting software that we kind of estimate.
If I want to get from it, because people are coming back 40 years later.
We knew how to get there when we found it.
But now, some 20-year-old going, I want to get to here.
How the hell do I do it?
It's actually got a hard problem.
Is there a consensus on formats and standards now?
Or is that still just unsolved?
No, there isn't.
But there's fewer bits.
There's a certain amount of, they used to be like 50 cave programs.
And now there's about 10 that people are still using, I think.
So, it's less of a problem.
But, I mean, in terms of the data formats.
I think, you know, because there's the 3D format that, like,
Therian and Servex share, they both output that.
And there's a plugin, which is in QGIS, which will import it.
So that basically just works.
I don't know if this QGIS plugin is for anybody else's formats.
There might be.
So, yeah.
So, I think there is people understanding that we shouldn't be using our own crazy formats.
Any more than we absolutely have to.
So, I think just the sense of line processing part will probably remain specialist formats.
And feel of them.
But everything else, we're just used to standard things.
You know, GJs and Shakefiles and all that.
Malaki.
Thank you.
I was just wondering, is the data accurate enough that you can tell whether you could
physically squeeze through certain openings and whether you can't throw others?
Yeah, pretty much.
Generally, you can tell from a certain, I mean, the surveys are annotated with things like squeeze and two tight.
And obviously, it's subjective.
How small a person and how enthusiastic are they can you find?
You know, it's an excellent sport for tiny women.
Otherwise often disadvantage in sport.
So, yeah, generally the surveys are, you can kind of tell what's.
Practical.
Thank you.