Skip to content

Synchronized spider config with ufal #635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
milanmajchrak merged 11 commits into from
Apr 23, 2024
Merged

Synchronized spider config with ufal #635

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
1d49887
Copied `ufal.ignore.txt` spider
milanmajchrak Apr 16, 2024
77b1c2d
Did ufal.ignore.txt cause failing tests?
milanmajchrak Apr 16, 2024
a946720
Trying to have only one IP in the `ufal.ignore.txt`
milanmajchrak Apr 16, 2024
df34fa3
Trying to have some IP in the `ufal.ignore.txt`
milanmajchrak Apr 16, 2024
c8bfeb5
Trying to have some IP in the `ufal.ignore.txt`
milanmajchrak Apr 16, 2024
25824e7
Trying decrease number of IP in the `ufal.ignore.txt`
milanmajchrak Apr 16, 2024
03c9e12
Added a few IP addresses
milanmajchrak Apr 16, 2024
79d4d41
Maybe I found problematic IP address.
milanmajchrak Apr 16, 2024
6df3c5b
Removed conflict IP address
milanmajchrak Apr 16, 2024
7e57334
Fixed IP with `/` instead of `.`
milanmajchrak Apr 22, 2024
246f8ca
The IP address must be in the format `74.86.158.0/24`
milanmajchrak Apr 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions dspace/config/spiders/ufal.ignore.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# monitoring from ufal-point-dev
ufal-point-dev.ms.mff.cuni.cz
195.113.20.155
# monitoring from GWDG
134.76.10.179
134.76.10.175
134.76.4.132
#lindat-monitoring + stuff from piwik filters (2015/11/19)
195.113.20.66
195.113.20.155
134.76.10.179
134.76.10.175
134.76.4.132
74.86.158.0/24
46.137.190.132
122.248.234.23
188.226.183.141
178.62.52.237
54.79.28.129
54.94.142.218
104.131.107.63
54.67.10.127
54.64.67.106
139.162.154.49
74.86.158.106
74.86.158.107
74.86.158.109
74.86.158.110
69.162.124.226
69.162.124.227
69.162.124.228
69.162.124.229
69.162.124.230
69.162.124.231
69.162.124.232
69.162.124.233
69.162.124.234
69.162.124.235
69.162.124.236
69.162.124.237
69.162.124.238
74.86.158.108
46.137.190.132
122.248.234.23
188.226.183.141
178.62.52.237
54.79.28.129
54.94.142.218
104.131.107.63
54.67.10.127
54.64.67.106