Of course they have bots... That's how you index the web.
If we took Google offline I bet there would also be a reduction in bot traffic from them too!
Indexing is also probably a low priority task, so if half their datacenter capacity is taken offline, their bot traffic probably reduces by more than 50%.
Logic says google bots do not come disguised, but as google agents.
Or is there other information?
Now there is probably also AI agent spam from google, but not at this level I believe. Also I am not fully clear what they count as spam, I doubt they would include google in this list, so do they include yandex bots here, if they come officially?
I am unsure what you're trying to say. But it seems like you don't understand what Bot means in general and how is it specifically defined for the metric provided in that page.
From that pages point of view, a Google bot, Yandex bot, or your uncle Tom's AI agent spam is the same, a bot.
If official yandex bots ain't included in this data here - then it means spam/scam bots come indeed from yandex servers. As they stopped working, when the yandex data center was hit.
It's a nice reminder that HN is just filled with the same primitive tribal normies and is always getting oneshotted by the same transparent xenophobic hysteria that plagues the rest of the internet.
Yandex has cloud offering, their servers are used for actual bot traffic, not just crawlers. Yandex hasn’t been only a search company for a very long time.
I like to study rhetoric, and appreciate folks asking for specifics. There's a lot of generalities thrown around on this forum that are highly questionable, and the only antidote I have is to request details. It's surprising how often folks resist.
A sibling comment replying to you is using a familiar tactic, which is to try to shame you for asking for specifics, while simultaneously providing none and telling you that it's obvious.
Occasionally people do provide details and I learn something, but that's not the common case, sadly.
Having multiple genuinely different viewpoints in a discussion isn’t evidence of “bots”, it’s the default in any community that isn’t heavily censored to silence dissent. And individuals voicing their different viewpoints isn’t “propaganda”, it’s the definition of discourse.
The russian bots are better.. usually its conspiracy fog, attached to a valid russia opposing point that reveals them. Chinese bots are just this months CCP talking points, so can be quite easily visible if they cram some topic only china cares about into every reply.
The downvotes are because you're resisting easily-observed reality with the seemingly-rational response "Where is this 'air'? If I'm breathing it, it should be easy to point to, no?"
No. If you can't see it, and don't understand it's there, you cannot be explained to. You're someone with no palate asking how anyone else can know if a wine tastes good.
I think you're saying that it's hard to articulate why some particular comment should be judged as having come from a bot. I.e., similar to the SCOTUS opinion regarding pornography: "I know it if I see it".
Is that correct?
If so, wouldn't that still admit someone providing a specific example? The whole point of my original comment was that I knew I wasn't judging these things the same as others were.
I can’t believe there are so many dumb comments in this thread.
All it shows is that crawling from Yandex search engine stopped from one Yandex data center ASN.
No one runs opinion/malicious bots from data center IPs that people mention here.
You are replying to a war propaganda post with a heavily editorialized title. Of course it's swimming in bad faith engagement, that's how these things work.
On the other hand there are plenty of datacenter located command/control systems that puppeteer vast fleets of residential proxies (and Russia is by no means the only one doing so), so it will be very telling to see what social media accounts go silent over the coming days and weeks.
> from one Yandex data center ASN
This is only one of two extant Yandex ASN, not an ASN specific to one geographic region/datacenter.
The HTTP traffic from Yandex is down to ~ 0 % after the third strike, after the first two strikes already degraded it. I wonder how this looks like at the network level because their search page still serves, but presumably their crawler is inactive now
Edit: Here is some more info, they do have availability issues as is to be expected
> Because three of its primary facilities were knocked offline within four days, Yandex’s failover mechanisms broke down[1][8].
> Outages have cascaded across Yandex Go (taxis), Yandex Pay, food delivery, smart home platforms, internal logistics, and third-party corporate services hosted on Yandex Cloud[2][7]. Problems have spilled into neighboring countries using these systems (Belarus, Kazakhstan, Armenia, Serbia)[2][7].
Edit 2: Here is the Yandex Cloud status page, their GCP/AWS equivalent. The entire compute core is offline, and even worse, Cloud Backup
I don't know how useful it will be as a case study since an HA/Ops person in (let's say, the US, Canada, AU, western european countries) has access to do things like buy hypervisor capacity or colocation in just about any location. I could go buy colo in Brussels or Panama City or San Diego or Edmonton or Melbourne tomorrow by contacting the right people.
The Russians are much more restricted from doing anything geographically outside of their own borders (for very well justified sanctions reasons), so their limitations are not the same as an HA person in almost any other country. Maybe some Iranian HA/Ops people will be comparing notes with them.
Sometimes there exist data or processing residence requirements that bind you to a specific region (e.g. EU-only). Also, „all AZs in this region no longer exist“ is not on everyone‘s bingo card, this is a good reminder why multi-AZ deployment in the same region may be not enough.
For data center guys this is definitely a case study, because drone attacks are probably on the threat list for everyone now, for various reasons, including domestic terrorism and hybrid warfare. If you want to defend your investment, time to think what covers you from the air.
I don't think there is much you can do about being targeted by a military, simply because of how unique the attack vector is.
Hacking is smart but happens through software, so if you have hardware (i.e. physical) barriers, that stops the hacking.
Natural disasters happen physically but they are dumb, so if you have software redundancy (i.e. by copying the data to multiple locations), that stops them from destroying everything.
A military is both physical AND smart. They can target multiple locations at once and destroy whole buildings.
You can learn to quickly diversify availability (or at least data safety) beyond the region of conflict when a kinetic war breaks out. The moment “kinetic sanctions” became the strategy, the risk to data centres should have been self-evident. And if it wasn’t immediately obvious, it should have been blindingly so when targeting expanded from oil refineries to Wildberries warehouses.
I don‘t think you can build a new data center in such a short time frame that passed since attacks on WB, definitely not when you can source hardware only from the black market.
I feel like this isn't even a particularly new thing. I worked at a medium-sized SaaS company and we did disaster recovery drills for moving our stack from eu-west-1 to eu-central-1 in case something catastrophic happened in Ireland. If you want HA, you kinda have to assume that any single country in the world can have some kind of a crippling disaster. Unfortunately for Yandex's clients, Yandex only has regions in two countries, and not that many regions either.
I predict a great many more videos coming out of Russia of Firepoint UAVs flying into refineries and such, punctuated by background audio commentary that can be summed up as "cyka blyat"
*Sudden drop in Cloudflare bot traffic from AS13238 YANDEX — Yandex LLC
UPD Definition of bot traffic from Cloudflare's glossary:
>Bot vs. human traffic
>The percentage of HTTP requests classified as bot versus human, based on bot scores. Requests with a bot score between 1 and 29 are classified as likely automated (bot), while requests with a score of 30 or above are classified as likely human. For more information, refer to Bot classes.
>By default, Radar shows bot vs. human traffic for requests to HTML content. This filter is meant to represent traditional web traffic by excluding API calls, asset requests (images, scripts, fonts), and other machine-to-machine requests.
Everyone is mentioning crawlers, but wouldn't most traffic from a datacenter's ASN usually be bots just in general given that there's not many humans there to generate human traffic?
Low-IQ Americans will think these bots are X accounts posting memes against Democrats, which is why the media is astroturfing this news to target American liberals. In reality, most of these bots are just scripts performing normal enterprise jobs, but most people know nothing about how the world works.
I'm going to be really interested how much this cleans up the internet, I mean the damage is done if you read comment sections on youtube etc, but let's see how long the BS persists.
I was thinking, we should all contribute to a Yandex fund for the Ukrainian mil to help keep the level of bot activity on the rest of the internet under control!
Modern day Russia is about the furthest possible thing from "commies". Kleptocracy? Fascism? Dictatorship? Oligarchy? Sure, some bizarro world combination of all of those. But if you want people to take your opinion seriously, try not to sound like somebody's demented grandpa by tossing around "commie" like it's 1959.
I am pretty damn sure that whatever Putin is doing these days is in no way motivated by an adherence to the Marxist-Leninist way of life.
According to orthodox Marxism, the Russian empire had not fully industrialised or created a widespread bourgeoisie. It was still largely feudal and rural, and a monarchy. Lenin tried to paint kulaks as bourgeois en lieu of actual target, and had to modify Marxism heavily to make things fit. Stalin certainly managed to industrialise much of the USSR, so fulfilled that end. Yeltsin-Putin Russia has created an actual bourgeois state, maybe allowing a workers' revolution to take place for real next time.
Or sort of. In reality, Marx was not an infallible prophet. He did not anticipate artificial intelligence or its consequences, genetic engineering, broadcast media or a host of other things. History has not followed all of the paths he suggested.
An eye for an eye for the Russians.
I love western imperial propaganda though, and how HN so eagerly amplifies it. There’s something self-fulfilling about willingly becoming a foot soldier of the Empire.
Here's to a world where the West is monolithic, and may Iran, Iraq, Yugoslavia, Syria, Libya, Lebanon, Palestine be only be the beginning. And may every new crusade for freedom and democracy leave behind another Gaza, in pursuit of upholding human rights and civilizing the world.
Of course they have bots... That's how you index the web.
If we took Google offline I bet there would also be a reduction in bot traffic from them too!
Indexing is also probably a low priority task, so if half their datacenter capacity is taken offline, their bot traffic probably reduces by more than 50%.
https://radar.cloudflare.com/bots/as200350?dateRange=7d
https://www.peeringdb.com/net/20950
EDIT: I was wrong, they're just generic bot vs human traffic numbers and you're right.
Or is there other information?
Now there is probably also AI agent spam from google, but not at this level I believe. Also I am not fully clear what they count as spam, I doubt they would include google in this list, so do they include yandex bots here, if they come officially?
From that pages point of view, a Google bot, Yandex bot, or your uncle Tom's AI agent spam is the same, a bot.
Yes, but it's mostly the opposite tribe of the one you're complaining about.
I've never noticed occurrences of such stuff, so I'm curious what I'm failing to key in on.
Edit: and would someone mind explaining the downvotes? If they're meant to communicate something to me, I'm genuinely not understanding.
A sibling comment replying to you is using a familiar tactic, which is to try to shame you for asking for specifics, while simultaneously providing none and telling you that it's obvious.
Occasionally people do provide details and I learn something, but that's not the common case, sadly.
No. If you can't see it, and don't understand it's there, you cannot be explained to. You're someone with no palate asking how anyone else can know if a wine tastes good.
Regardless, wouldn't it be easy to point me to even one example if it's as prevalent as (I think) you're saying?
Or am I missing your main point, i.e. that what's being described can't be demonstrated via example?
Read and think about my wine analogy, if you care.
Is that correct?
If so, wouldn't that still admit someone providing a specific example? The whole point of my original comment was that I knew I wasn't judging these things the same as others were.
Only then, I'll believe you that wine can taste good or bad.
Is my mind open? Is this not a waste of your time?
If so, it isn't particularly surprising that the line would go down.
EDIT: yes, it is. Looking at the same graph for all of Russia, and there's no discernible pattern:
https://radar.cloudflare.com/bots/ru?dateRange=7d
same for the 48h view:
https://radar.cloudflare.com/bots/ru?dateRange=2d
No one runs opinion/malicious bots from data center IPs that people mention here.
Did people actually stop thinking?
> from one Yandex data center ASN
This is only one of two extant Yandex ASN, not an ASN specific to one geographic region/datacenter.
https://www.peeringdb.com/net/1751
The other one is this: https://www.peeringdb.com/net/20950
And the 200350 also shows a near complete drop off in bot activity:
https://radar.cloudflare.com/bots/as200350?dateRange=7d
That is a drop in traffic specifically from Yandex bots (which includes their crawlers and other apps hosted in their DC)
Now compare it with Bot Traffic in:
* Russia: https://radar.cloudflare.com/bots/ru?dateRange=7d
* Europe: https://radar.cloudflare.com/bots/europe?dateRange=7d
* Worldwide: https://radar.cloudflare.com/bots?dateRange=7d
Dip is barely noticeable even in Russia.
Its like saying, "API requests are completely down" (and small disclaimer: only from your server, because your server is down)
Edit: Here is some more info, they do have availability issues as is to be expected
> Because three of its primary facilities were knocked offline within four days, Yandex’s failover mechanisms broke down[1][8].
> Outages have cascaded across Yandex Go (taxis), Yandex Pay, food delivery, smart home platforms, internal logistics, and third-party corporate services hosted on Yandex Cloud[2][7]. Problems have spilled into neighboring countries using these systems (Belarus, Kazakhstan, Armenia, Serbia)[2][7].
Edit 2: Here is the Yandex Cloud status page, their GCP/AWS equivalent. The entire compute core is offline, and even worse, Cloud Backup
https://status.yandex.cloud/ru/dashboard
> ru Compute Cloud, ru Kubernetes, ru PostgreSQL, ru ClickHouse, ru MySQL, ru YDB, ru Kafka, ru Application Load Balancer
Interestingly, Yandex tells its users on the status page to switch to competitors Selectel, K2Cloud, and VK Cloud. Which will probably be hit next
The Russians are much more restricted from doing anything geographically outside of their own borders (for very well justified sanctions reasons), so their limitations are not the same as an HA person in almost any other country. Maybe some Iranian HA/Ops people will be comparing notes with them.
For data center guys this is definitely a case study, because drone attacks are probably on the threat list for everyone now, for various reasons, including domestic terrorism and hybrid warfare. If you want to defend your investment, time to think what covers you from the air.
Hacking is smart but happens through software, so if you have hardware (i.e. physical) barriers, that stops the hacking.
Natural disasters happen physically but they are dumb, so if you have software redundancy (i.e. by copying the data to multiple locations), that stops them from destroying everything.
A military is both physical AND smart. They can target multiple locations at once and destroy whole buildings.
You could choose not to invade other countries.
No, you can choose to collaborate or you can shut it down.
Collaborators don't get to point back in time and say 'oh we had no choice', even if the choice was a painful one. Sorry it does not work that way.
("Ept" and "suka" are two other swearwords, that are also used much like punctuation.)
UPD Definition of bot traffic from Cloudflare's glossary:
>Bot vs. human traffic
>The percentage of HTTP requests classified as bot versus human, based on bot scores. Requests with a bot score between 1 and 29 are classified as likely automated (bot), while requests with a score of 30 or above are classified as likely human. For more information, refer to Bot classes.
>By default, Radar shows bot vs. human traffic for requests to HTML content. This filter is meant to represent traditional web traffic by excluding API calls, asset requests (images, scripts, fonts), and other machine-to-machine requests.
https://developers.cloudflare.com/radar/glossary/#bot-vs-hum...
https://developers.cloudflare.com/bots/concepts/bot-score/
Pretty benign AS as far as actual exploit or malicious crawling is concerned.
https://radar.cloudflare.com/bots
https://cofense.com/blog/how-cloudflare-services-are-abused-...
https://www.heise.de/en/news/Report-Cybercriminals-use-Cloud...
Or maybe they would just want to get rid of the competition.
(I’d welcome a data center and oil infrastructure targeting specific fund)
https://www.peeringdb.com/net/1751
I am pretty damn sure that whatever Putin is doing these days is in no way motivated by an adherence to the Marxist-Leninist way of life.
Or sort of. In reality, Marx was not an infallible prophet. He did not anticipate artificial intelligence or its consequences, genetic engineering, broadcast media or a host of other things. History has not followed all of the paths he suggested.
Here's to a world where the West is monolithic, and may Iran, Iraq, Yugoslavia, Syria, Libya, Lebanon, Palestine be only be the beginning. And may every new crusade for freedom and democracy leave behind another Gaza, in pursuit of upholding human rights and civilizing the world.