Okay, so let me welcome Alexander, who will be presenting dynamic board-blocking with
web server access log analytics.
Okay, thank you, so my name is Alexander, and instead of starting from who I am, I'm going
to start from what we do.
We develop a team-person W, it's a high performance hybrid of HTTP accelerator and follow directly
embedded into the Linux TCP IP stack.
So essentially we continue the TCP stack with protocols, TLS, and HTTP, everything works
inside the Linux kernel.
The main focus is for the product, high performance and security, we emphasize web security
and complete a GDOS protection, including volumetric GDOS protection, and a 7GDOS protection.
For volumetric GDOS protection, we provide also XEPA model to paste the XFW, which is 5W against volumetric
GDOS.
Thanks to internal implementation, there is no corpus, contacts, which has a lot of overheads
eliminated.
So typically, in paste of W, it performs traditional HTTP service in terms of high throughput
and low latency.
In terms of security, the tempest remains very deep HTTP parser, which uses about magnitude
large number of states done usual, HTTP parsers, and this way we can catch a lot of injection
attacks, right on HTTP parsing stage, so we don't need additional security fish and stage.
There are a lot of great limits from the basic request to advanced HTTP to contact
of wayms, great limits.
JavaScript and cookie changes are out of the box, and this also light key value database,
tempest of the B, which keeps web cache and fish loose.
Thanks to the SPIP stack integration, tempest of W is coupled with net feeder, so you can
write multiple level fish loose, like glucose, traffic from particle IP address, and
having particle value of refered header.
Today I'll be most focused on both and the how tempest of W handles the bots, and the
different types of bots.
The first one is D2, so that could be flat or so HP, with flat, it's very easy to detect,
and block because they are categorized with high volumes of packets or requests.
So HP is more difficult to catch because they are not so high connection rates or requests
rates, but they target memory existing or CPU or socket, but still possible to catch them.
The security scan has password carriers, they also are easy to detect because they usually
imply high rates of HP error responses.
And when we talk about bots, we usually mean specific types of the bots, that content
scarpas, one of the most popular bots, and they are so widespread because of AI, AI needs
a lot of data to learn and usually the data is copied to use the bots.
The e-commerce, a music, or a inventory is copied to use the booking bots, for example,
if some public service provides some appointments, the bot can get those appointments, and
sell them for something.
So there are a lot of types of the bots.
Why is hard to protect against the bots?
It's not a simple security security scan, it's a simple, but for example, for scarpas,
they are cloud vendors which provide boxes for helping bots to buy past protection.
They can provide thousands of IP addresses, for example, WN experiences such scarpas, which
are considered to be dedosatex against them.
They can solve capture JavaScript sharing, as they can pretend, the browser means that if you
just send simple HP request to HTML to the food approaches, they also generate requests
for CSS, JavaScript, and doesn't include resources.
They provide personalization consistency, I'll talk about this later, and that when
this plants provide human behavior, this is a screenshot from the web search, and basically
you can Google any large vendor, could also hire a client, whatever you can name, like
top 10, and there are plenty of companies providing services to buy past services, and
there's one highlighted, and goes provide very nice technical article, how do they actually
buy past Clouds where you can buy the other top 10 service, because that's not the end of
story, and very active community of developing custom was protection, so even if not enough
to use cloud services, we can buy past detection with custom development, to fight with
bots, we need to identify clients that could be IP addresses, the most additional way was that
could be a fingerprint, fingerprint basically computed on a different type of network,
networks, and as a hash values, usually we do not use use of my controller data wide
HP request, next we could specify the traffic it could be, packet race, request race, and
also for some cases like didors, we need some trigger events to not be in protection mode always,
traffic fingerprints just like human fingerprints, a huge number of metrics, which we can
fingerprint, and generate a hash value, as an example is J3, the most popular fingerprint
it's over, JDS parameters, and this md5 over several JLS parameters, you might see it as
hexadempostrin in access lock, J4 is fast development of J3, it has free pass, the first part is
that's the compression version of the data, and the second, second, the third part is a
shade to 56, the third problem with cryptographic hash is that one of the main properties for cryptography
hash is to well distribute the data, so if you have two points, which are different only in one
bit, you get the fingerprint values which are different in those bit, however if we use fingerprints
for some machine classification, we need some similarity metrics, so if you have very similar
values, we need a very similar finger prints, and also inside the kernel cryptography API is very
expensive, we need to store and restore a few state for this, just mention PGOF with also over
all the network layers, but also tries to guess software versions, a video helped our own finger
prints, which do not use cryptography to be fast and to be a machine learning friendly,
this is a small example of how it is computed just like very similar to J3 or J4,
next you see finger prints in access locks, like example on the slide, and usually HPC versus
store access locks in files, and that's for much streams, and the performance problems with
this way of storage, for much stream can slow, and the writing to file is also slow, so it's usually
hard to write the large amount of data on the files, and even more hard to analyze such amount of
data, so to cope with the problem, people usually use relatively sophisticated pipelines to deliver
the locks to some analytic data bases, we end up with the click-out because of high ingestion
performance and powerful analytics, to quickly send our access locks to click-outs, which
have a tip of W, logger, which has perched PU defeat against bots in several cases, the first case
is a common website, the inventory is typing these characters with high rates of request to
cultivate, in that case we saw thousands of IP address and fake user agents, so we use the
click-out square here, where we find all the top TLS fingerprints, which have a ratio between
card URL access to our other user's more than 30%, and we see that the very top results,
not only produced a lot of requests, but also it has much larger ratio between card requests and
our others. The next one is security scanning, we run our website on WordPress, and it still
opens an XML IPC endpoint by default, and this is a popular endpoint for $70 attacks, and if you
would just create a database, how frequently it is, access is the only more frequently accessed
the URL for us is index page, so it's very, a lot of traffic comes from a security scanning
bots. Usually, the endpoint is exported using post request, and we've verified that we really
deploy with successful response to such post request, and in that case, we were really
vulnerable against LSDDOS. Next, we've related that what the boss having the F3 TLS fingerprints
do on our website, and we found that the website at the endpoints, they also visit regular
URLs, shown in green, but also such queries, I've used that with TLS fingerprints to fingerprints
ending with zero bits, and the next significant for bits now TLS fingerprint and code LPN,
it's a limitation of the client, which HP version is going to talk. Usually, a browser has played some
value, but in this case, there is no place, this is quite unusual for normal clients, and we can be
pretty sure that this TLS fingerprints are malicious. In that case, we didn't block the finger
prints because some false positives are possible fingerprints can change, and we blocked the endpoint
just on by your way on TPSW site. The last example is slow HP, it tries to spend as much
several times as possible, so we can't just get the largest community of time because it will
be just the most popular, serious fingerprint, and we cannot just block the top talkers,
because it's also a request, we've produced the longest response time because it will be just
unlikely, clients, but if we try to intersect the top talkers with clients who produce the
longest request, we might see interesting results, and in this query, we also see zero LPN,
and a very high value of which response time. The query is pretty large, it's not handy to
write it by hands, so we developed a small, accessible Python German, which runs such queries
on your behalf. This is such queries named detectors, and we, it can issue, after my
blocking loss for IP said, for NAF type was for TPSW, based on the detector findings,
and that's the detectors and defense mode can be switched on by some triggers like Z-squares or manually.
This is also the detector validation logic, so we can configure a couple of detectors,
or fingerprints by a request per second, or L-R-L response code is pure. Second, and defined
the threshold, this is a score, Z-square value stem, meaning that when we reach 10 standard deviations,
we go through defense mode. Next, the algorithm works like we first make the query for
the request per second, under attack, and we repeat the query one hour ago. The
defense against the bots, but in our experience, we saw that user agencies still
reliable metrics to fit and that a lot of bots still exposing, not trying to
pretend any real-life browsers. Our theory is thinking of prints to work in most of the cases,
we saw in examples. However, the problem with Greece, with Firefox fingerprint protection,
this is a random nose. We can leave with it with just an immunization of fingerprints.
The good news is that even without an immunization, it means that normal browsers will produce
a lot of fingerprints, but if bot doesn't try to impersonate, we will see a single and huge
point for the same TWS fingerprint, and the bad news is that there are a lot of open source libraries
through bypass and to provide implicitization for both developers. I want to conclude with
two key ideas. First one is that it might be not wise to rely solely on large vendors because
large vendors protect a lot of websites, some websites are pretty large and good targets for
both developers. On all the crowd approaches, puts a gift and defaults into bypassing the large
vendors and the custom solution might be more reliable in comparison with large vendor.
With a custom solution, I don't mean just to repeat some of the strategies from the large vendors.
Instead, this is a good community discussion for in Shopify community and the people
developing bots protection logic right on the web resource database. So if you have a
comments website, you have a database for your goods, for your clients and so on, and from the
database, you have your secret weapon. It's a knowledge how your users interact with your
websites. This unique knowledge and unique pattern for only your resources. And this way, you can integrate
that get a simple Python program integrated with access stock with your
database, production database, and develop very strong. We have analytics
solution which will be resilient against bot attacks. That's all. We've shared and
tempest available on GitHub, just in case the large knowledge base, what we can do on the
limited site, that's all. Thank you, and maybe we have a presentation.
We have time for a question, so
Hi, I was wondering how you deal with the difference between like a de-dos and a
hug of death, where you've got like thousands of legitimate clients who've all
got a specific page from red or something. Sorry, difference between de-dos and
the hug of death, hug of death, like when like somebody postalling to red it and so like you
shouldn't get tens of thousands people all going to one page, but they're all really mean like a
fresh call, right? You mean just like a legit user, just like a lot of traffic, right? But legit,
normal users. For now, I'm a simple, much more to do with the attributes, but it's the
possible to do with the heavenauises. One of the point is that during de-dos, you never see
some sophisticated patterns like the first the recurs index page, then traverse to some
good space and make some search and make a more or less normal, normal traffic. So the second
point was to find a way to transition graphs. I believe the good way to differentiate a lot of real
users from de-dos traffic, and de-dos traffic usually very simple just requesting the same
resource. De-dos, I'm sorry. We still have time for a question, I think, but please be quiet,
please don't go any yet. There is the real bit time for that. Any more questions? No questions?
Okay, thank you. Thank you very much.