Let's welcome on next speaker Sophia the floor is yours.
Thank you.
Thank you so much.
I'm so excited to be here.
I've been in this Debra Malat and it's first time presenting.
So thanks to the organizers for taking a chance on this talk.
It's a little bit of a weird one.
Fraising the question of do we need another open source taxonomy.
Why am I talking about this?
I've spent my career in market research consulting
analysis doing research and analysis projects now around open source software
community ecosystems.
And through all of that I have found myself building taxonomy.
Either design comprehensive service questions or to create groupings
and hierarchies in ways to group and analyze data.
And now increasingly data about open source projects and communities.
And mainly I guess if you want to use case,
why am I doing this to begin with?
If we work on a project we gather a bunch of data.
There's always the question about what about this project is influencing the data or the behavior
or the risk or the metrics that we're calculating.
Is there some nuance or context that we can look at in our analysis and our grouping
to compare how different projects may be a language versus something supporting container
run times?
Is that going to look different in our data?
And how do we group that?
So to do that we're going to need a taxonomy.
So where do we find them?
I started really proud just looking everywhere and anywhere on the internet from things like
our government administration use and the standard industry classification.
This was designed in 1937.
So a little bit old, a little bit more technology started.
There was a couple of revisions here.
And it has a nice split between software and hardware.
But if you look at the categories, this is not particularly helpful in terms of grouping open source software
and packages.
So I kept going.
They actually revised this.
And now the technology services are now even more less granular.
So not necessarily helpful.
Note that that original classification is still being used today in the screening stage commission
in the United States.
I kept going.
I went through public administrations.
I went through other types of governments.
I went through the UN data.
And what I didn't do because I don't have a group of researchers that my back in call
is a systematic literature review.
So I realized that's a big gap in this approach.
But you'll see why I didn't necessarily get there.
So then I started with the internet.
What does the internet say?
Anything that was in front of a not behind a paywall.
How are people grouping different types of software and technologies in and around open source?
Clearly, we know hardware, software split.
That's an easy one.
And within software, we saw a split between application and system.
More application is loosely defined as interacting with the end user and system is loosely defined
as everything that sits on top of the machine to interact with the machine from operating systems, drivers,
from where is language processing.
Utilities, I snuck in at the end here because this is where I started to see some conflict,
where some taxonomies had that included in the application layer and some had it included in the system.
And the question I had really is, well, what is a developer?
Are they a user?
Are they interacting with the system?
And it's ambiguous because they could be anywhere in a full-sac development environment.
What was also missing was that there's a whole bucket of other stuff,
especially in open source software communities that aren't software or hardware.
It could be protocols, frameworks, networks, standards.
You notice from my slide deck, I work with the open source project chaos,
which designs metrics around measuring open source health and sustainability.
We also make software, but a lot of our assets are actually content and not necessarily software.
So, where would this fit in this taxonomy?
In our other bucket, which happens a lot if you're familiar with these sorts of exercises.
The thing that actually found the most helpful was a Wikipedia page about ontology engineering.
So, not actually software development at all,
but actually really liked the categories that they were distilling to describe the engineering process.
Starting with an organization, the mission requirements and goals,
what is the function of the thing, what are the individual components that are needed in the function,
how do they communicate with each other, how do you connect them together,
what is needed for information to process and to work through these,
to work through the system, and where does it actually get a run in terms of the physical environment and the attributes of that?
So, again, not designed for software at all, but probably the most applicable generic taxonomy I could find
that could generally describe groups of functional things that could be applied to say open source software, open source software packages.
So, the more I started looking at taxonomies, I kind of went down a little bit of rabbit hole,
and I realized, wait, there's kind of different kinds of taxonomies.
I started making a taxonomy effect taxonomies, and so, again, getting a little bit over over the top here,
but I started noticing certain characteristics about them, whether or not they were functionally designed,
like that ontology engineering page, where we're really just looking at the core function of the thing,
or is there some sort of organizational umbrella in context that's describing the need or the function of the thing,
which kind of goes all the way to the other side of that, which is a completely context-driven taxonomy,
which an example would be, and maybe more often the case for researchers, is you're given a bucket of data.
You might describe a taxonomy in general and say, this would be the perfect theoretical way to group the data,
and you apply it to the data, and you have 90% of the things in one category, and maybe a couple of random others.
That is not an effective way to analyze your set because statistical significance requires you to have more distribution in your bucketing,
and to have enough, or at least an interesting sample or breakdown to look at.
So, often you end up maybe breaking down categories so you can have more interesting groupings, or more granular groupings,
or lumping things together when your sample sizes are too small.
So, examples.
This is a functional taxonomy.
I apologize, the source is internal, this is done for internal exercise, and I have a couple of these,
and I realized it couldn't share all the data, but I could share the metadata because it's not sensitive,
where a colleague of mine attempted to propose a functional taxonomy for open source software packages,
framework, language, library, and database utility operating system, and I tried to apply this to an exercise this year,
and I found, I was still missing things, particularly infrastructure, so that kind of physical environment.
So, again, this person doesn't work on my team anymore, so it couldn't go back and ask them,
why didn't you have infrastructure in here, maybe they were focusing just on developers.
So, possibly something we could apply, but there were still some shortcomings to this approach.
This is the results from an actual project that I completed earlier this last year,
where I looked at a set of packages in our environment, and I was trying to categorize them in a functional taxonomy,
to again evaluate our open source infrastructure portfolio, what we're using, we're re-depending on,
where there's certain characteristics of these projects that change the way that we evaluated the risk,
how critical they are to our environment, what kind of resourcing and staffing do we have upstream versus internally
that are working on these kinds of projects? Again, kind of questions that we have that might be nuanced or related
to the individual project at hand. So, I went through a whole bunch, I read through a lot of readmees,
and started to apply individual labels to each of them, and then tried to group those into logical functional categories.
So, what started as a functional taxonomy, if you look at it more closely, I would say might actually be an organizational taxonomy,
because some of these groupings and labels actually looks a lot like the departmental names and groupings inside of my organization.
We have a security team, we have a team that looks at our physical environment, we have a team that looks at observability,
a team that focuses specifically on QA, or data processing protocols, policies.
And so, I realized while I was trying to design a functional taxonomy, actually made of design and organizational taxonomy,
but in my case, that was actually more helpful because in the organizational context, I wanted to understand where we were using open surf software,
how was that critical or less critical to specific teams, to specific use cases inside of our environment?
And then here's a weird one. This is an example of a mixed taxonomy that was actually a survey question that we use when we were serving
a whole bunch of open-source software projects and organizations.
And you can notice that the categories are mixture of stuff, they're a mixture of tooling, they're a mixture of applications,
and even some industries kind of stuck in there. Why does it look like this?
Well, we had less categories in our prior survey in the last year,
and we had a whole bunch of people checking the same boxes. And so, again, if you think about the distribution of the data,
it wasn't actually telling us that much about the distribution of the entire sample.
So we essentially looked at the categories that had the most responses and said,
can we break this up even further? Can we provide a little bit more context that can allow us from an analysis perspective
to really understand the group and make up of the people taking our survey and the projects that they represent?
So, let's say this is a contextual taxonomy.
Now in practice, have I usually use these things to refine significant results in my analysis projects?
Surprisingly, not that much. What I've compared different types of open-source projects from a functional perspective
and try to compare them, is there any difference in their community dynamics and their growth and their engagement trajectories and their use cases?
And yes, there's difference in use cases, but everything else, it was kind of muddled.
There was so much nuance and context that the actual taxonomy is the end of using an end of being most helpful and functional in my analysis
where more are directed about the people, say the users, if it was like, okay, here's the system and application infrastructure
and end user, but also developer as an individual person or user use case.
And then looking more at contributors and contributor breakdowns, whether or not they were full time contributors, part time,
what how they were contributing, whether or not it was code or other types of tasks that they were taking on and sort of the general level of contribution.
So I ended up using that a lot more.
Here's one of the examples that I love was a project that was looking at trying to find records of non-code contribution in open source software
and they looked at a whole bunch of different artifacts and from the artifacts designed to taxonomy of things that they found and then grouped it.
So here I would say is a contextually built taxonomy based on the data that they had in order to make sense of what they were looking at.
And again, look at levels and types of contributions happening outside of just code.
The other segment that I ended up using a lot was again still looking at the people, except for this time around how we engage with each other as a community.
So looking at say the life cycle of the project, is it brand new, is it years old, is it in growth or decline, general community size?
What kind of processes are in place, governance models, say for example, and particularly technology platforms in use.
If you're on GitLab versus GitHub or in a private Git server, you're interaction and engagement patterns are going a little bit different.
And so that's going to color what your research looks like.
And so these actually ended up being much more effective ways to group projects.
Again, then any of those individual functional technology categories.
Here's another one that I found.
I know I just stuck a lot of taxonomy examples here.
And if you look at the full deck on the, there's actually even more in here that I cut out because I realized just talking about taxonomy examples.
Could eventually get a little bit boring, but there's a lot out there.
And I'm just been trying to collect as many as possible.
Just so we have more references and things to build from.
Here's an example of project status.
This individual proposed a taxonomy to apply badges to your GitHub project to communicate where it was.
Is this something brand new?
Was it a concept?
Is it continually being developed or is it something that you've abandoned?
So as it was realizing and putting all this together that while as a technology researcher, I really wanted to apply.
Functional technical categories in my open source project research finding that it was less actually useful than all of the social characteristics.
That was yet again reminded that open source software is really defined as a social technical ecosystem and model.
The social elements seemingly are the ones that are standing out and grouping us more than these technology categories.
Another taxonomy example because I have to.
Another one of my former colleagues Julie Ferrioli proposed the social model open source, which proposes that we should look at projects by classifying their intent or original purpose.
Because we have to recognize that not every open source software project was intended to be a collaborative growing community.
This of one is a release of demo code or experiment something or prove that something could be done.
And in theory this would be an amazing thing to test and research can we group projects by these things and see maybe that should impact their project trajectory, their sustainability, their engagement patterns.
However, in practice, this is really hard to do.
It's not particularly scalable or on something you can automate necessarily.
I haven't been able to do this outside of reading rabies and still not really being sure.
So I talked about a lot of examples, but I also want to talk about some of the pain.
If I get anyone here has done research like this, you've probably tried to do this and you've probably run into some challenges.
Here are the ones that I've come across the most, particularly my own bias.
It's really hard to get out of your own bias, which is why my general recommendation is don't do this alone.
This is why you have to look for examples, work with your colleagues, work with people who have different perspectives.
Because it's really hard to build a comprehensive view of the world.
One of my favorite examples of where I failed, I was designing a survey for an event feedback.
I'm mechanism we were asking people how the event was, what sessions they liked.
And so we had the question on unemployment, are you currently employed, are you not, are you contractor, are you looking for work, and we put it in the survey.
And I'm looking at the results, and 25% of the other say, student.
I didn't put student in there.
I haven't been a student for 15 years.
I wasn't thinking about that.
That is completely an oversight of my part.
And maybe if I shared this list with someone else before I published a survey, we would have noticed that.
So there could be silly reasons why you forget things, but also the less the ability things in the vacuum,
the more likely you can improve things that you're missing.
We acknowledge that mostly things are built for purpose, so they're not necessarily extensible,
which is possibly why we keep redoing this exercise, because we keep building it for our particular problem,
our particular data set, our particular research question, and we have to keep building them again.
And maybe that's not necessarily a thing that we want to do.
We acknowledge that there's a lot of overlap in taxonomies.
Things don't neatly fit into categories.
I don't know if favorite examples is where would you put a Python library in that functional taxonomy?
It's a language.
It's a library.
It might even be some utilities in that library.
I don't even know what it does.
So what is it then?
Do we put it under all three categories?
Or do we try to group it under one?
And technology keeps changing.
So maybe we have to keep refreshing our taxonomy, and you're always going to have an other bucket.
I still haven't gone around the other problem.
So my general proposal is that it's okay if things have multiple categories.
Perhaps we should start looking at things and acknowledging that we need multiple kinds of things.
Multiple kinds of taxonomies to really understand what the thing is and how we might analyze it.
And just a reminder for a Melvin Conway that a lot of the times are organizational designs and systems.
Mirror on communication structure.
So implying that even if we think we're building something completely logically and systematically,
there's some contextual bias at play, which is probably was causing a lot of these problems.
So can we do this better?
First is trying to just shove a large gooey cat into a small box that he doesn't fit it.
I got really inspired by the Open Demographics project.
If you're not familiar with it, Mickey Stevens proposed this as a way to better write demographic questions for open source surveys of community members.
Because demographic questions can be really sensitive, especially if you start asking about under representation or different types of identity categories.
You really generally don't want to offend your community members.
You want to make sure everyone feels represented and heard but not exposed.
And so ensuring you have the right kind of questions and the right balance of questions is a very delegate exercise.
And so because of that, she proposed this project where other people have been able to come in and provide suggestions for how to write demographic questions.
And I really like that project, which the website might be down so I really hope it's still happening, but they get a page is still active.
So we had an idea in the Chaos community.
I've been working with the data science working group, which works through lots of analysis projects that we propose that we work on together, that we talk about.
And we said, is this something that we could crowdsource? Is this something that we never have to do again?
So what if we shared?
This is sort of, this is again an experiment. I don't know if this is going to work.
But I started to upload some of the tax honomies that I found on to GitHub as just a way to increase more visibility of this.
Open source researchers are going to keep building tax honomies to try to describe open source software in different ways.
And again, if maybe if I were able to apply a literature review, we could start to pull in some from research as well.
But just making it more discoverable, I don't know if this is going to work, but if it does, then hopefully we can stop the next person from building a taxonomy in a vacuum.
Thank you.
Okay.
Yes.
The question or what are the cat's names?
The black and white cat is Newton.
And the orange cat is Moby after Mobius.
Notice there are two themes here around mathematics. My partner and I are really into naming our pets after famous mathematicians.
Moby makes a lot of fun shapes in Euclidean geometry and Newton.
She's just smarter, I don't know.
That was really, really interesting.
And it also, a language that we all do research in the economy and the things that only found that there is no sense of the community.
It's really great that this is electing all of these economies.
Because one thing, so my question is, imagine there is an expert in coming in and saying, I need an economy.
I need to be filled with that.
There are a lot of those standards of that exist.
We've got to standardize that.
Do we have that economy?
Where people can throw it and shop for the specific economy that works out that.
And what's the end of coming from that space that we could never do up from this economy?
Yes.
I guess there's, I'm not quite sure what the question was there.
I guess there was a kind of example.
Is there a place that's a standard hub?
I hope so.
The question is the place that we're gathering tax economies could be interactive and place where people can kind of browse and pursue or peruse tax economies.
That's the end goal for this.
So I think part of the structure of it is also designing a data card if you're familiar with other types of open source data sets or data that's being opened,
whether or not it's open source.
There's sort of this model the data card that provides metadata about what the thing is before you even open it.
And so we're hoping if this structure works well as a way to collect information about tax economies,
then we can turn that into a way to search for tax economies.
So that would essentially build on itself.
This is all depending on whether or not we get enough data.
But again, experiment.
So that would be the end goal that we could just start to build this as a growing reference for anyone to use.
Okay.
Okay?
You have permanent question?
Me?
No?
No.
I'll have one.
I think I heard everything.
whether it is more important, but one of the reasons
is to think there is something that, at the end of the day,
it's one taxonomy of TV.
Don't you see the usefulness of the contextual use
or creation of it, and whether it could be,
then, something important not to pay less,
just to get to the information, because just to keep in mind,
that it was for one purpose, and it's good to pay for that.
Yeah, so the question is around that a lot of the contextual
Bill talks on andies are really designed for that purpose,
and maybe that's just, they continue to be relevant in that purpose.
I can completely agree with that.
I think, especially the ones that I've built,
they're very much reflective of the problem at hand.
I think my inclination for this is that someone can still learn from that.
So I think even if you're never used it again for anything else,
it's still, here's a wonderful example of a taxonomy built for this particular case.
The one that comes to mind is actually Daniel Katz here with me,
a research paper that he might talk about later today,
around research for science, open source research for science,
and I'm blanking on the exact naming, and I apologize for that.
But it actually proposes another taxonomy,
that's specifically for research software versus just open source software.
And I think that is an example of one that is particularly context-focused,
and it's very effective in where it is, but it isn't generally applicable.
So I agree, and I think, again, just more visibility of these things would benefit the general case.
I see on next speaker here.
Thank you. Thank you.
Thank you.