Это только у меня каждый день регистрируется несколько спамеров и вешают ссылки? Началось недавно.
Понятно, что ссылки идут через /go, но все равно неприятно.
Кто как борется и какие результаты?
регистрируются и ссылки вешают
#1
24 января 2012 в 21:37
#2
24 января 2012 в 21:59
отключаем стену, баним по ip
проверяйте ip юзера под именем вашсайт.ru/users/pavdyuchenko — это спамер, его ip сразу в бан
проверяйте ip юзера под именем вашсайт.ru/users/pavdyuchenko — это спамер, его ip сразу в бан
еще эти ip баньте сразу
178.76.226.219
178.125.141.116
178.125.53.172
178.76.226.219
178.125.141.116
178.125.53.172
Сегодня в 15:08
#4
24 января 2012 в 22:48
а где в настройках отключается стена?
#5
24 января 2012 в 23:39
Попадете в топ и прощупывание, допустим иньекции, будут самое малое. Спамеры были и будут, просто надо быть готовым к этому.
Отключить можно все что угодно, включая регу пользователей. Раскрученный проект должен быть под наблюдением 24 часа, для этого существует слово "КОМАНДА". А то что в начале пути вам неудаеться чистить вовремя, что делать, принимайте как должное или как плату за некоторую популярность.
Вот когда нет спамеров и логи чистенькие, нет даже попыток того же хрумера, вот тут надо задуматься.
Отключить можно все что угодно, включая регу пользователей. Раскрученный проект должен быть под наблюдением 24 часа, для этого существует слово "КОМАНДА". А то что в начале пути вам неудаеться чистить вовремя, что делать, принимайте как должное или как плату за некоторую популярность.
Вот когда нет спамеров и логи чистенькие, нет даже попыток того же хрумера, вот тут надо задуматься.
в профиле пользователей (настройка профилей)а где в настройках отключается стена ?
#7
25 января 2012 в 00:52
появился новый. пишет ссылки на ваз клуб.
#8
25 января 2012 в 00:53
его ip 178.76.226.219
## USER IP BANNING <Limit GET POST> order allow,deny deny from 178.76.226.219 allow from all </Limit>
Кстати, актуально для всяких качалок, но добавить агента.
Для тех кто аж...
В роботс
User-agent: OmniExplorer_Bot
Disallow: /
User-agent: FreeFind
Disallow: /
User-agent: BecomeBot
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: Jetbot/1.0
Disallow: /
User-agent: Jetbot
Disallow: /
User-agent: WebVac
Disallow: /
User-agent: Stanford
Disallow: /
User-agent: naver
Disallow: /
User-agent: dumbot
Disallow: /
User-agent: Hatena Antenna
Disallow: /
User-agent: grub-client
Disallow: /
User-agent: grub
Disallow: /
User-agent: looksmart
Disallow: /
User-agent: WebZip
Disallow: /
User-agent: larbin
Disallow: /
User-agent: b2w/0.1
Disallow: /
User-agent: Copernic
Disallow: /
User-agent: psbot
Disallow: /
User-agent: Python-urllib
Disallow: /
User-agent: Googlebot-Image
Disallow: /
User-agent: NetMechanic
Disallow: /
User-agent: URL_Spider_Pro
Disallow: /
User-agent: CherryPicker
Disallow: /
User-agent: EmailCollector
Disallow: /
User-agent: EmailSiphon
Disallow: /
User-agent: WebBandit
Disallow: /
User-agent: EmailWolf
Disallow: /
User-agent: ExtractorPro
Disallow: /
User-agent: CopyRightCheck
Disallow: /
User-agent: Crescent
Disallow: /
User-agent: SiteSnagger
Disallow: /
User-agent: ProWebWalker
Disallow: /
User-agent: CheeseBot
Disallow: /
User-agent: LNSpiderguy
Disallow: /
User-agent: Mozilla
Disallow: /
User-agent: mozilla
Disallow: /
User-agent: mozilla/3
Disallow: /
User-agent: mozilla/4
Disallow: /
User-agent: mozilla/5
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000)
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: ia_archiver/1.6
Disallow: /
User-agent: Alexibot
Disallow: /
User-agent: Teleport
Disallow: /
User-agent: TeleportPro
Disallow: /
User-agent: Stanford Comp Sci
Disallow: /
User-agent: MIIxpc
Disallow: /
User-agent: Telesoft
Disallow: /
User-agent: Website Quester
Disallow: /
User-agent: moget/2.1
Disallow: /
User-agent: WebZip/4.0
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: WebSauger
Disallow: /
User-agent: WebCopier
Disallow: /
User-agent: NetAnts
Disallow: /
User-agent: Mister PiX
Disallow: /
User-agent: WebAuto
Disallow: /
User-agent: TheNomad
Disallow: /
User-agent: WWW-Collector-E
Disallow: /
User-agent: RMA
Disallow: /
User-agent: libWeb/clsHTTP
Disallow: /
User-agent: asterias
Disallow: /
User-agent: httplib
Disallow: /
User-agent: turingos
Disallow: /
User-agent: spanner
Disallow: /
User-agent: InfoNaviRobot
Disallow: /
User-agent: Harvest/1.5
Disallow: /
User-agent: Bullseye/1.0
Disallow: /
User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
Disallow: /
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
Disallow: /
User-agent: CherryPickerSE/1.0
Disallow: /
User-agent: CherryPickerElite/1.0
Disallow: /
User-agent: WebBandit/3.50
Disallow: /
User-agent: NICErsPRO
Disallow: /
User-agent: Microsoft URL Control — 5.01.4511
Disallow: /
User-agent: DittoSpyder
Disallow: /
User-agent: Foobot
Disallow: /
User-agent: WebmasterWorldForumBot
Disallow: /
User-agent: SpankBot
Disallow: /
User-agent: BotALot
Disallow: /
User-agent: lwp-trivial/1.34
Disallow: /
User-agent: lwp-trivial
Disallow: /
User-agent: www.WebmasterWorld.com bot
Disallow: /
User-agent: BunnySlippers
Disallow: /
User-agent: Microsoft URL Control — 6.00.8169
Disallow: /
User-agent: URLy Warning
Disallow: /
User-agent: Wget/1.6
Disallow: /
User-agent: Wget/1.5.3
Disallow: /
User-agent: Wget
Disallow: /
User-agent: LinkWalker
Disallow: /
User-agent: cosmos
Disallow: /
User-agent: moget
Disallow: /
User-agent: hloader
Disallow: /
User-agent: humanlinks
Disallow: /
User-agent: LinkextractorPro
Disallow: /
User-agent: Offline Explorer
Disallow: /
User-agent: Mata Hari
Disallow: /
User-agent: LexiBot
Disallow: /
User-agent: Web Image Collector
Disallow: /
User-agent: The Intraformant
Disallow: /
User-agent: True_Robot/1.0
Disallow: /
User-agent: True_Robot
Disallow: /
User-agent: BlowFish/1.0
Disallow: /
User-agent: www.SearchEngineWorld.com bot
Disallow: /
User-agent: www.WebmasterWorld.com bot
Disallow: /
User-agent: JennyBot
Disallow: /
User-agent: MIIxpc/4.2
Disallow: /
User-agent: BuiltBotTough
Disallow: /
User-agent: ProPowerBot/2.14
Disallow: /
User-agent: BackDoorBot/1.0
Disallow: /
User-agent: toCrawl/UrlDispatcher
Disallow: /
User-agent: WebEnhancer
Disallow: /
User-agent: suzuran
Disallow: /
User-agent: VCI WebViewer VCI WebViewer Win32
Disallow: /
User-agent: VCI
Disallow: /
User-agent: Szukacz/1.4
Disallow: /
User-agent: QueryN Metasearch
Disallow: /
User-agent: Openfind data gathere
Disallow: /
User-agent: Openfind
Disallow: /
User-agent: Xenu's Link Sleuth 1.1c
Disallow: /
User-agent: Xenu's
Disallow: /
User-agent: Zeus
Disallow: /
User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /
User-agent: RepoMonkey
Disallow: /
User-agent: Microsoft URL Control
Disallow: /
User-agent: Openbot
Disallow: /
User-agent: URL Control
Disallow: /
User-agent: Zeus Link Scout
Disallow: /
User-agent: Zeus 32297 Webster Pro V2.9 Win32
Disallow: /
User-agent: Webster Pro
Disallow: /
User-agent: EroCrawler
Disallow: /
User-agent: LinkScan/8.1a Unix
Disallow: /
User-agent: Keyword Density/0.9
Disallow: /
User-agent: Kenjin Spider
Disallow: /
User-agent: Iron33/1.0.2
Disallow: /
User-agent: Bookmark search tool
Disallow: /
User-agent: GetRight/4.2
Disallow: /
User-agent: FairAd Client
Disallow: /
User-agent: Gaisbot
Disallow: /
User-agent: Aqua_Products
Disallow: /
User-agent: Radiation Retriever 1.1
Disallow: /
User-agent: WebmasterWorld Extractor
Disallow: /
User-agent: Flaming AttackBot
Disallow: /
User-agent: Oracle Ultra Search
Disallow: /
User-agent: MSIECrawler
Disallow: /
User-agent: PerMan
Disallow: /
User-agent: searchpreview
Disallow: /
User-agent: sootle
Disallow: /
User-agent: es
Disallow: /
User-agent: Enterprise_Search/1.0
Disallow: /
User-agent: Enterprise_Search
Disallow: /
И в .htaccess
RewriteEngine on RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR] RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot[at]yahoo.com [OR] RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR] RewriteCond %{HTTP_USER_AGENT} ^Custo [OR] RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR] RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR] RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR] RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR] RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR] RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR] RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR] RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR] RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR] RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR] RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR] RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR] RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR] RewriteCond %{HTTP_USER_AGENT} ^HMView [OR] RewriteCond %{HTTP_USER_AGENT} ^HTTrack [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR] RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^Indy\ Library [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR] RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR] RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR] RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR] RewriteCond %{HTTP_USER_AGENT} ^larbin [OR] RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR] RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR] RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR] RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR] RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR] RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR] RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR] RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR] RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR] RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR] RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR] RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR] RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR] RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR] RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR] RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR] RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR] RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR] RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR] RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR] RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR] RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR] RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR] RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^Wget [OR] RewriteCond %{HTTP_USER_AGENT} ^Widow [OR] RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR] RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Zeus RewriteRule ^.* - [F,L]
#10
25 января 2012 в 01:48
обалдеть! спасибо!
#11
25 января 2012 в 02:08
у меня от хостера сегодня пришло следующее письмо
Вам не кажется что это спам? похоже что с вашего сайта постоянно шлют почту.....,
Это ваш емейл? мояпочта@mail.ru
-auth_id сайт
-auth_sender сайт@s06.хостер.com
-allow_unqualified_recipient
-allow_unqualified_sender
-local
XX
1
don.ford@mail.ru
190P Received: from сайт by s06.хостер.com with local (Exim 4.77)
(envelope-from <сайт@s06.хостер.com>)
id 1Rpgqs-0000RM-Jj
for don.ford@mail.ru; Tue, 24 Jan 2012 17:55:58 +0400
021T To: don.ford@mail.ru
081 Subject: =?windows-1251?B?0yDi4PEg7e7i7uUg8e7u4fnl7ejlISAtIEtvbGhvemFuZXQuUnU=?=
057 X-PHP-Script: сайт.ru/index.php for 41.208.167.170
038 Date: Tue, 24 Jan 2012 17:55:58 +0400
035* Return-Path: мояпочта@mail.ru
046F From: "Сайт.Ru" <мояпочта@mail.ru>
061I Message-ID: <5421dd6017eef7fbbf79fbb11dc7d83a@сайт.ru>
014 X-Priority: 3
035 X-Mailer: PHPMailer [version 1.73]
018 MIME-Version: 1.0
034 Content-Transfer-Encoding: base64
049 Content-Type: text/plain; charset="windows-1251"
Кто нибудь может подсказать что это?
Вам не кажется что это спам? похоже что с вашего сайта постоянно шлют почту.....,
Это ваш емейл? мояпочта@mail.ru
-auth_id сайт
-auth_sender сайт@s06.хостер.com
-allow_unqualified_recipient
-allow_unqualified_sender
-local
XX
1
don.ford@mail.ru
190P Received: from сайт by s06.хостер.com with local (Exim 4.77)
(envelope-from <сайт@s06.хостер.com>)
id 1Rpgqs-0000RM-Jj
for don.ford@mail.ru; Tue, 24 Jan 2012 17:55:58 +0400
021T To: don.ford@mail.ru
081 Subject: =?windows-1251?B?0yDi4PEg7e7i7uUg8e7u4fnl7ejlISAtIEtvbGhvemFuZXQuUnU=?=
057 X-PHP-Script: сайт.ru/index.php for 41.208.167.170
038 Date: Tue, 24 Jan 2012 17:55:58 +0400
035* Return-Path: мояпочта@mail.ru
046F From: "Сайт.Ru" <мояпочта@mail.ru>
061I Message-ID: <5421dd6017eef7fbbf79fbb11dc7d83a@сайт.ru>
014 X-Priority: 3
035 X-Mailer: PHPMailer [version 1.73]
018 MIME-Version: 1.0
034 Content-Transfer-Encoding: base64
049 Content-Type: text/plain; charset="windows-1251"
Кто нибудь может подсказать что это?
Дык смотрите в какой папке шелл лежит и льет мыло. Иначе попадете в блек листы, чем скорее найдете тем лучше.
Хотя хостер должен был предоставить откуда, точнее место исполнения скрипта в вашей директории.
Хотя хостер должен был предоставить откуда, точнее место исполнения скрипта в вашей директории.
добавил в htaccess- так сайт вообще вырубилоИ в .htaccess
Дык надо смотреть что у вас в файле, плясать от этого нужно. Например вот эта строчка RewriteEngine on
Должна быть один раз и тд.
Должна быть один раз и тд.
#15
25 января 2012 в 05:05
garry,
Допустимо-ли после этого
Следующая запись
User-Agent: *
Allow: /
Sitemap: site.org/sitemap.xml
Disallow: /index.php
Disallow: *?*
Disallow: /actions
Disallow: /users$
Disallow: /isearch
Disallow: /ysearch
Disallow: /admin
Disallow: /backups
Disallow: /core
Disallow: /components
Disallow: /wysiwyg
Disallow: /search
Disallow: /includes
Disallow: /modules
Disallow: /filters
Disallow: /languages
Disallow: /logout*
Disallow: /passremind.html
Disallow: /registration
Disallow: /login
Disallow: /catalog$
Disallow: /photos/latest.html
Disallow: /photos/100$
Disallow: /users$
Disallow: */messages.html
Disallow: */messages-sent.html
Disallow: */messages-new.html
Disallow: */newpost.html
Disallow: */newcat.html
Disallow: */editblog.html
Disallow: /*ajax
Disallow: */blogs/*/cat-*
Disallow: */addphoto.html
Disallow: */add.html
Disallow: */bookmarks/*
Disallow: */avatar.html
Disallow: */editprofile.html
Disallow: */karma.html
Disallow: /users/city/*
Disallow: /users/*/comments.html
Disallow: /users/*/forumposts.html
Disallow: /users/hobby/*
Disallow: /users/*/board.html
Disallow: /clubs/*/config.html
Disallow: /clubs/*/join_member.html
Disallow: */newthread.html
Disallow: */pinthread*.html
Disallow: */forum/reply*.html
Disallow: */forum/thread*-quote*.html
Disallow: */forum/subscribe*.html
Disallow: */subscribe/
Disallow: /go
Disallow: /cgi-bin
Disallow: /cache
Disallow: /templates
Disallow: /druzja
Disallow: /my.html
Disallow: /blogs/0
Disallow: /content/0
Disallow: /photos/0
Disallow: /board/0
Disallow: /faq/0
Disallow: /forum/0
Disallow: /price/0
Disallow: /0
Disallow: /rss
Disallow: /subscribe
Disallow: /board/*city
Disallow: /board/*type
Disallow: /catalog/*find
Disallow: /catalog/*find-first
Disallow: /*ajax
Disallow: /templates/
Disallow: /view-registration/
Disallow: /view-users/
Disallow: /upload/
Disallow: /search/
Host: site.orgТо есть,User-Agent: *
или теперь нужно указывать для Яндекса, Гугла, и т.п. отдельно?
Допустимо-ли после этого
User-agent: OmniExplorer_Bot
Disallow: /
User-agent: FreeFind
Disallow: /
User-agent: BecomeBot
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: Jetbot/1.0
Disallow: /
User-agent: Jetbot
Disallow: /
User-agent: WebVac
Disallow: /
User-agent: Stanford
Disallow: /
User-agent: naver
Disallow: /
User-agent: dumbot
Disallow: /
User-agent: Hatena Antenna
Disallow: /
User-agent: grub-client
Disallow: /
User-agent: grub
Disallow: /
User-agent: looksmart
Disallow: /
User-agent: WebZip
Disallow: /
User-agent: larbin
Disallow: /
User-agent: b2w/0.1
Disallow: /
User-agent: Copernic
Disallow: /
User-agent: psbot
Disallow: /
User-agent: Python-urllib
Disallow: /
User-agent: Googlebot-Image
Disallow: /
User-agent: NetMechanic
Disallow: /
User-agent: URL_Spider_Pro
Disallow: /
User-agent: CherryPicker
Disallow: /
User-agent: EmailCollector
Disallow: /
User-agent: EmailSiphon
Disallow: /
User-agent: WebBandit
Disallow: /
User-agent: EmailWolf
Disallow: /
User-agent: ExtractorPro
Disallow: /
User-agent: CopyRightCheck
Disallow: /
User-agent: Crescent
Disallow: /
User-agent: SiteSnagger
Disallow: /
User-agent: ProWebWalker
Disallow: /
User-agent: CheeseBot
Disallow: /
User-agent: LNSpiderguy
Disallow: /
User-agent: Mozilla
Disallow: /
User-agent: mozilla
Disallow: /
User-agent: mozilla/3
Disallow: /
User-agent: mozilla/4
Disallow: /
User-agent: mozilla/5
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000)
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: ia_archiver/1.6
Disallow: /
User-agent: Alexibot
Disallow: /
User-agent: Teleport
Disallow: /
User-agent: TeleportPro
Disallow: /
User-agent: Stanford Comp Sci
Disallow: /
User-agent: MIIxpc
Disallow: /
User-agent: Telesoft
Disallow: /
User-agent: Website Quester
Disallow: /
User-agent: moget/2.1
Disallow: /
User-agent: WebZip/4.0
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: WebSauger
Disallow: /
User-agent: WebCopier
Disallow: /
User-agent: NetAnts
Disallow: /
User-agent: Mister PiX
Disallow: /
User-agent: WebAuto
Disallow: /
User-agent: TheNomad
Disallow: /
User-agent: WWW-Collector-E
Disallow: /
User-agent: RMA
Disallow: /
User-agent: libWeb/clsHTTP
Disallow: /
User-agent: asterias
Disallow: /
User-agent: httplib
Disallow: /
User-agent: turingos
Disallow: /
User-agent: spanner
Disallow: /
User-agent: InfoNaviRobot
Disallow: /
User-agent: Harvest/1.5
Disallow: /
User-agent: Bullseye/1.0
Disallow: /
User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
Disallow: /
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
Disallow: /
User-agent: CherryPickerSE/1.0
Disallow: /
User-agent: CherryPickerElite/1.0
Disallow: /
User-agent: WebBandit/3.50
Disallow: /
User-agent: NICErsPRO
Disallow: /
User-agent: Microsoft URL Control — 5.01.4511
Disallow: /
User-agent: DittoSpyder
Disallow: /
User-agent: Foobot
Disallow: /
User-agent: WebmasterWorldForumBot
Disallow: /
User-agent: SpankBot
Disallow: /
User-agent: BotALot
Disallow: /
User-agent: lwp-trivial/1.34
Disallow: /
User-agent: lwp-trivial
Disallow: /
User-agent: www.WebmasterWorld.com bot
Disallow: /
User-agent: BunnySlippers
Disallow: /
User-agent: Microsoft URL Control — 6.00.8169
Disallow: /
User-agent: URLy Warning
Disallow: /
User-agent: Wget/1.6
Disallow: /
User-agent: Wget/1.5.3
Disallow: /
User-agent: Wget
Disallow: /
User-agent: LinkWalker
Disallow: /
User-agent: cosmos
Disallow: /
User-agent: moget
Disallow: /
User-agent: hloader
Disallow: /
User-agent: humanlinks
Disallow: /
User-agent: LinkextractorPro
Disallow: /
User-agent: Offline Explorer
Disallow: /
User-agent: Mata Hari
Disallow: /
User-agent: LexiBot
Disallow: /
User-agent: Web Image Collector
Disallow: /
User-agent: The Intraformant
Disallow: /
User-agent: True_Robot/1.0
Disallow: /
User-agent: True_Robot
Disallow: /
User-agent: BlowFish/1.0
Disallow: /
User-agent: www.SearchEngineWorld.com bot
Disallow: /
User-agent: www.WebmasterWorld.com bot
Disallow: /
User-agent: JennyBot
Disallow: /
User-agent: MIIxpc/4.2
Disallow: /
User-agent: BuiltBotTough
Disallow: /
User-agent: ProPowerBot/2.14
Disallow: /
User-agent: BackDoorBot/1.0
Disallow: /
User-agent: toCrawl/UrlDispatcher
Disallow: /
User-agent: WebEnhancer
Disallow: /
User-agent: suzuran
Disallow: /
User-agent: VCI WebViewer VCI WebViewer Win32
Disallow: /
User-agent: VCI
Disallow: /
User-agent: Szukacz/1.4
Disallow: /
User-agent: QueryN Metasearch
Disallow: /
User-agent: Openfind data gathere
Disallow: /
User-agent: Openfind
Disallow: /
User-agent: Xenu's Link Sleuth 1.1c
Disallow: /
User-agent: Xenu's
Disallow: /
User-agent: Zeus
Disallow: /
User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /
User-agent: RepoMonkey
Disallow: /
User-agent: Microsoft URL Control
Disallow: /
User-agent: Openbot
Disallow: /
User-agent: URL Control
Disallow: /
User-agent: Zeus Link Scout
Disallow: /
User-agent: Zeus 32297 Webster Pro V2.9 Win32
Disallow: /
User-agent: Webster Pro
Disallow: /
User-agent: EroCrawler
Disallow: /
User-agent: LinkScan/8.1a Unix
Disallow: /
User-agent: Keyword Density/0.9
Disallow: /
User-agent: Kenjin Spider
Disallow: /
User-agent: Iron33/1.0.2
Disallow: /
User-agent: Bookmark search tool
Disallow: /
User-agent: GetRight/4.2
Disallow: /
User-agent: FairAd Client
Disallow: /
User-agent: Gaisbot
Disallow: /
User-agent: Aqua_Products
Disallow: /
User-agent: Radiation Retriever 1.1
Disallow: /
User-agent: WebmasterWorld Extractor
Disallow: /
User-agent: Flaming AttackBot
Disallow: /
User-agent: Oracle Ultra Search
Disallow: /
User-agent: MSIECrawler
Disallow: /
User-agent: PerMan
Disallow: /
User-agent: searchpreview
Disallow: /
User-agent: sootle
Disallow: /
User-agent: es
Disallow: /
User-agent: Enterprise_Search/1.0
Disallow: /
User-agent: Enterprise_Search
Disallow: /
Disallow: /
User-agent: FreeFind
Disallow: /
User-agent: BecomeBot
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: Jetbot/1.0
Disallow: /
User-agent: Jetbot
Disallow: /
User-agent: WebVac
Disallow: /
User-agent: Stanford
Disallow: /
User-agent: naver
Disallow: /
User-agent: dumbot
Disallow: /
User-agent: Hatena Antenna
Disallow: /
User-agent: grub-client
Disallow: /
User-agent: grub
Disallow: /
User-agent: looksmart
Disallow: /
User-agent: WebZip
Disallow: /
User-agent: larbin
Disallow: /
User-agent: b2w/0.1
Disallow: /
User-agent: Copernic
Disallow: /
User-agent: psbot
Disallow: /
User-agent: Python-urllib
Disallow: /
User-agent: Googlebot-Image
Disallow: /
User-agent: NetMechanic
Disallow: /
User-agent: URL_Spider_Pro
Disallow: /
User-agent: CherryPicker
Disallow: /
User-agent: EmailCollector
Disallow: /
User-agent: EmailSiphon
Disallow: /
User-agent: WebBandit
Disallow: /
User-agent: EmailWolf
Disallow: /
User-agent: ExtractorPro
Disallow: /
User-agent: CopyRightCheck
Disallow: /
User-agent: Crescent
Disallow: /
User-agent: SiteSnagger
Disallow: /
User-agent: ProWebWalker
Disallow: /
User-agent: CheeseBot
Disallow: /
User-agent: LNSpiderguy
Disallow: /
User-agent: Mozilla
Disallow: /
User-agent: mozilla
Disallow: /
User-agent: mozilla/3
Disallow: /
User-agent: mozilla/4
Disallow: /
User-agent: mozilla/5
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000)
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: ia_archiver/1.6
Disallow: /
User-agent: Alexibot
Disallow: /
User-agent: Teleport
Disallow: /
User-agent: TeleportPro
Disallow: /
User-agent: Stanford Comp Sci
Disallow: /
User-agent: MIIxpc
Disallow: /
User-agent: Telesoft
Disallow: /
User-agent: Website Quester
Disallow: /
User-agent: moget/2.1
Disallow: /
User-agent: WebZip/4.0
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: WebSauger
Disallow: /
User-agent: WebCopier
Disallow: /
User-agent: NetAnts
Disallow: /
User-agent: Mister PiX
Disallow: /
User-agent: WebAuto
Disallow: /
User-agent: TheNomad
Disallow: /
User-agent: WWW-Collector-E
Disallow: /
User-agent: RMA
Disallow: /
User-agent: libWeb/clsHTTP
Disallow: /
User-agent: asterias
Disallow: /
User-agent: httplib
Disallow: /
User-agent: turingos
Disallow: /
User-agent: spanner
Disallow: /
User-agent: InfoNaviRobot
Disallow: /
User-agent: Harvest/1.5
Disallow: /
User-agent: Bullseye/1.0
Disallow: /
User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
Disallow: /
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
Disallow: /
User-agent: CherryPickerSE/1.0
Disallow: /
User-agent: CherryPickerElite/1.0
Disallow: /
User-agent: WebBandit/3.50
Disallow: /
User-agent: NICErsPRO
Disallow: /
User-agent: Microsoft URL Control — 5.01.4511
Disallow: /
User-agent: DittoSpyder
Disallow: /
User-agent: Foobot
Disallow: /
User-agent: WebmasterWorldForumBot
Disallow: /
User-agent: SpankBot
Disallow: /
User-agent: BotALot
Disallow: /
User-agent: lwp-trivial/1.34
Disallow: /
User-agent: lwp-trivial
Disallow: /
User-agent: www.WebmasterWorld.com bot
Disallow: /
User-agent: BunnySlippers
Disallow: /
User-agent: Microsoft URL Control — 6.00.8169
Disallow: /
User-agent: URLy Warning
Disallow: /
User-agent: Wget/1.6
Disallow: /
User-agent: Wget/1.5.3
Disallow: /
User-agent: Wget
Disallow: /
User-agent: LinkWalker
Disallow: /
User-agent: cosmos
Disallow: /
User-agent: moget
Disallow: /
User-agent: hloader
Disallow: /
User-agent: humanlinks
Disallow: /
User-agent: LinkextractorPro
Disallow: /
User-agent: Offline Explorer
Disallow: /
User-agent: Mata Hari
Disallow: /
User-agent: LexiBot
Disallow: /
User-agent: Web Image Collector
Disallow: /
User-agent: The Intraformant
Disallow: /
User-agent: True_Robot/1.0
Disallow: /
User-agent: True_Robot
Disallow: /
User-agent: BlowFish/1.0
Disallow: /
User-agent: www.SearchEngineWorld.com bot
Disallow: /
User-agent: www.WebmasterWorld.com bot
Disallow: /
User-agent: JennyBot
Disallow: /
User-agent: MIIxpc/4.2
Disallow: /
User-agent: BuiltBotTough
Disallow: /
User-agent: ProPowerBot/2.14
Disallow: /
User-agent: BackDoorBot/1.0
Disallow: /
User-agent: toCrawl/UrlDispatcher
Disallow: /
User-agent: WebEnhancer
Disallow: /
User-agent: suzuran
Disallow: /
User-agent: VCI WebViewer VCI WebViewer Win32
Disallow: /
User-agent: VCI
Disallow: /
User-agent: Szukacz/1.4
Disallow: /
User-agent: QueryN Metasearch
Disallow: /
User-agent: Openfind data gathere
Disallow: /
User-agent: Openfind
Disallow: /
User-agent: Xenu's Link Sleuth 1.1c
Disallow: /
User-agent: Xenu's
Disallow: /
User-agent: Zeus
Disallow: /
User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /
User-agent: RepoMonkey
Disallow: /
User-agent: Microsoft URL Control
Disallow: /
User-agent: Openbot
Disallow: /
User-agent: URL Control
Disallow: /
User-agent: Zeus Link Scout
Disallow: /
User-agent: Zeus 32297 Webster Pro V2.9 Win32
Disallow: /
User-agent: Webster Pro
Disallow: /
User-agent: EroCrawler
Disallow: /
User-agent: LinkScan/8.1a Unix
Disallow: /
User-agent: Keyword Density/0.9
Disallow: /
User-agent: Kenjin Spider
Disallow: /
User-agent: Iron33/1.0.2
Disallow: /
User-agent: Bookmark search tool
Disallow: /
User-agent: GetRight/4.2
Disallow: /
User-agent: FairAd Client
Disallow: /
User-agent: Gaisbot
Disallow: /
User-agent: Aqua_Products
Disallow: /
User-agent: Radiation Retriever 1.1
Disallow: /
User-agent: WebmasterWorld Extractor
Disallow: /
User-agent: Flaming AttackBot
Disallow: /
User-agent: Oracle Ultra Search
Disallow: /
User-agent: MSIECrawler
Disallow: /
User-agent: PerMan
Disallow: /
User-agent: searchpreview
Disallow: /
User-agent: sootle
Disallow: /
User-agent: es
Disallow: /
User-agent: Enterprise_Search/1.0
Disallow: /
User-agent: Enterprise_Search
Disallow: /
User-Agent: *
Allow: /
Sitemap: site.org/sitemap.xml
Disallow: /index.php
Disallow: *?*
Disallow: /actions
Disallow: /users$
Disallow: /isearch
Disallow: /ysearch
Disallow: /admin
Disallow: /backups
Disallow: /core
Disallow: /components
Disallow: /wysiwyg
Disallow: /search
Disallow: /includes
Disallow: /modules
Disallow: /filters
Disallow: /languages
Disallow: /logout*
Disallow: /passremind.html
Disallow: /registration
Disallow: /login
Disallow: /catalog$
Disallow: /photos/latest.html
Disallow: /photos/100$
Disallow: /users$
Disallow: */messages.html
Disallow: */messages-sent.html
Disallow: */messages-new.html
Disallow: */newpost.html
Disallow: */newcat.html
Disallow: */editblog.html
Disallow: /*ajax
Disallow: */blogs/*/cat-*
Disallow: */addphoto.html
Disallow: */add.html
Disallow: */bookmarks/*
Disallow: */avatar.html
Disallow: */editprofile.html
Disallow: */karma.html
Disallow: /users/city/*
Disallow: /users/*/comments.html
Disallow: /users/*/forumposts.html
Disallow: /users/hobby/*
Disallow: /users/*/board.html
Disallow: /clubs/*/config.html
Disallow: /clubs/*/join_member.html
Disallow: */newthread.html
Disallow: */pinthread*.html
Disallow: */forum/reply*.html
Disallow: */forum/thread*-quote*.html
Disallow: */forum/subscribe*.html
Disallow: */subscribe/
Disallow: /go
Disallow: /cgi-bin
Disallow: /cache
Disallow: /templates
Disallow: /druzja
Disallow: /my.html
Disallow: /blogs/0
Disallow: /content/0
Disallow: /photos/0
Disallow: /board/0
Disallow: /faq/0
Disallow: /forum/0
Disallow: /price/0
Disallow: /0
Disallow: /rss
Disallow: /subscribe
Disallow: /board/*city
Disallow: /board/*type
Disallow: /catalog/*find
Disallow: /catalog/*find-first
Disallow: /*ajax
Disallow: /templates/
Disallow: /view-registration/
Disallow: /view-users/
Disallow: /upload/
Disallow: /search/
Host: site.org
или теперь нужно указывать для Яндекса, Гугла, и т.п. отдельно?