User-agent: 007AC9
User-agent: 8
User-agent: 0xFF Domain Crawler
User-agent: 13TABS
User-agent: 192.comAgent
User-agent: 1stwarning bot
User-agent: 200PleaseBot
User-agent: 2Bone
User-agent: 2gisbot
User-agent: 360Spider
User-agent: 404 Checker
User-agent: 4seohuntBot
User-agent: 50.nu
User-agent: A6-Indexer
User-agent: abby
User-agent: Aboundexbot
User-agent: aboutthedomain
User-agent: AboutUsBot
User-agent: Abrave Spider
User-agent: Accelobot
User-agent: AcoonBot
User-agent: Adbeat bot
User-agent: AddThis.com
User-agent: ADmantX crawler
User-agent: AdnormCrawler
User-agent: adressendeutschland.de
User-agent: adstxtlab.com Crawler
User-agent: agentslug
User-agent: AhrefsBot
User-agent: aiHitBot
User-agent: Akamai crawler
User-agent: akula
User-agent: Alertra
User-agent: Alexabot
User-agent: AliveAdvisor crawler
User-agent: AlphaSeoBot
User-agent: alyze.info
User-agent: Amagit.COM
User-agent: Amazon bot
User-agent: amibot
User-agent: AMP validator
User-agent: AndersPinkBot
User-agent: AntBot
User-agent: Anturis Agent
User-agent: Apercite
User-agent: Apexoo Spider
User-agent: AportWorm
User-agent: AppBeat
User-agent: Applebot
User-agent: AraBot
User-agent: arachnode.net
User-agent: Arachnophilia
User-agent: Aranea
User-agent: archive.org_bot
User-agent: ArchiveTeam crawler
User-agent: argenfybot
User-agent: ArielisBot
User-agent: Arquivo-web-crawler
User-agent: asafaweb.com
User-agent: Ask Jeeves/Teoma
User-agent: AskQuickly
User-agent: Audisto Crawler
User-agent: auskunftbot
User-agent: Automattic Analytics Crawler
User-agent: Autonomy
User-agent: Avira crawler
User-agent: Awario crawler
User-agent: AWeber bot
User-agent: aylienbot
User-agent: BabalooSpider
User-agent: backlink-check.de
User-agent: BacklinkCrawler
User-agent: Bad-Neighborhood
User-agent: Baiduspider
User-agent: Barkrowler
User-agent: basicstate.com
User-agent: baypup
User-agent: BCKLINKS
User-agent: BDCbot
User-agent: BegunAdvertising
User-agent: Bender
User-agent: BigBozz
User-agent: bingbot
User-agent: biNu crawler
User-agent: bitlybot
User-agent: bixocrawler
User-agent: BizwikiBot
User-agent: bl.uk_lddc_bot
User-agent: Blackboard crawler
User-agent: BlinkaCrawler
User-agent: BlogPulse
User-agent: BlogSearch
User-agent: Blogtrottr
User-agent: bnf.fr_bot
User-agent: BoardReader Favicon Fetcher
User-agent: bot-pge.chlooe.com
User-agent: bot.wsowner.com
User-agent: botmobi
User-agent: BotOnParade
User-agent: BotRobin
User-agent: BOTW Spider
User-agent: BrainbruBot
User-agent: Braintree-Webhooks
User-agent: BrandProtect bot
User-agent: BrandVerity
User-agent: Brodie
User-agent: BrokenLinkCheck.com
User-agent: Browsershots
User-agent: BrowserSpyBot
User-agent: BUbiNG
User-agent: BusinessSeek.biz
User-agent: Butterfly
User-agent: Buzzbot
User-agent: BuzzSumo
User-agent: CamontSpider
User-agent: CareerBot
User-agent: CarianBot
User-agent: Castabot
User-agent: CatchBot
User-agent: CATExplorador
User-agent: CCBot
User-agent: censys.io crawler
User-agent: CERT.at-Statistics-Survey
User-agent: cg-eye
User-agent: ChangeDetection
User-agent: Charlotte
User-agent: Check-Domains.com bot
User-agent: Check-Host
User-agent: checkgzipcompression.com robot
User-agent: CheckMarkNetwork
User-agent: Checksem
User-agent: Chirp
User-agent: CirrusExplorer
User-agent: CISPA Vulnerability Notification
User-agent: CJ.com crawler
User-agent: classbot
User-agent: Clickagy Intelligence Bot
User-agent: CligooRobot
User-agent: CliqzBot
User-agent: Closure Compiler Service
User-agent: Cloud mapping
User-agent: CloudEndure Scanner
User-agent: CloudFlare crawler
User-agent: Cloudinary
User-agent: CloudServerMarketSpider
User-agent: CMS Crawler
User-agent: coccocbot
User-agent: Codewise bot
User-agent: CoinCornerBot
User-agent: CommaFeed
User-agent: COMODOSpider
User-agent: Company News Search engine
User-agent: Companybook Crawler
User-agent: CompSpyBot
User-agent: COMSYS crawler
User-agent: Contacts Crawler
User-agent: contentDetection
User-agent: ContentKing
User-agent: CopperEgg
User-agent: copyright sheriff
User-agent: CorpusCrawler
User-agent: Covario-IDS
User-agent: crawler for netopian
User-agent: crawler National Library of Israel
User-agent: Crawler4j
User-agent: Crawlera
User-agent: CrawlForMe
User-agent: CRIM Crawler
User-agent: cron-job.org
User-agent: CronDroid
User-agent: Cronjob.de bot
User-agent: Cronless
User-agent: Crowsnest
User-agent: csirt.cz
User-agent: CSS Certificate Spider
User-agent: CukBot
User-agent: Curious George
User-agent: cuwhois
User-agent: cXensebot
User-agent: damn404 crawler
User-agent: DareBoost
User-agent: datagnionbot
User-agent: Datanyze
User-agent: DataparkSearch
User-agent: Dataprovider Spider
User-agent: Daumoa
User-agent: DAWINCI ANTIPLAG
User-agent: Dazoobot
User-agent: DBLBot
User-agent: DCPbot
User-agent: Dead Link Checker
User-agent: deadlinkchecker
User-agent: DealGates Bot
User-agent: deepcrawl
User-agent: demandbase-bot
User-agent: DER-bot
User-agent: Detectify bot
User-agent: DeuSu
User-agent: Dex Media Bot
User-agent: DialogSearch.com Bot
User-agent: Diffbot
User-agent: Digg Deeper
User-agent: Digincore bot
User-agent: Discordbot
User-agent: discoverybot
User-agent: DIY-SEOBot
User-agent: dj-research
User-agent: DJEcoBot
User-agent: DKIMRepBot
User-agent: dlcbot
User-agent: dlvr.it
User-agent: DNS-Digger
User-agent: DNS-Tools Header-Analyzer
User-agent: DNSPod crawler
User-agent: Docoloc
User-agent: DocSpotBot
User-agent: Domain Re-Animator Bot
User-agent: DomainAppender
User-agent: DomainCheck.io Crawler
User-agent: DomainDB
User-agent: DomainMacroCrawler
User-agent: domainsbot
User-agent: DomainSigmaCrawler
User-agent: DomainSONOCrawler
User-agent: DomainStatsBot
User-agent: DomainTunoCrawler
User-agent: Donuts Content Explorer
User-agent: Dot TK - spider
User-agent: Dotcom-Monitor bot
User-agent: dotSemantic
User-agent: Dow Jones Searchbot
User-agent: downnotifier.com monitoring
User-agent: DowntimeDetector
User-agent: DripfeedBot
User-agent: drupact
User-agent: dubaiindex
User-agent: DuckDuckBot
User-agent: DWDS-crawler
User-agent: DwnldBot
User-agent: E. Orliac
User-agent: e.ventures crawler
User-agent: EasouSpider
User-agent: Easy-Thumb
User-agent: EasyBib crawler
User-agent: EasyCron
User-agent: eBot
User-agent: eCairn-Grabber
User-agent: eCommerceBot
User-agent: EdisterBot
User-agent: electricmonk
User-agent: Elefent
User-agent: Embedly
User-agent: emefgebot
User-agent: ENLLE bot
User-agent: Enolyst Crawler
User-agent: envolk
User-agent: Esribot
User-agent: etoolsbot
User-agent: EuripBot
User-agent: Eurobot
User-agent: evc-batch
User-agent: EventGuruBot
User-agent: EveryoneSocialBot
User-agent: EvriNid
User-agent: ExactSeek
User-agent: ExB Language Crawler
User-agent: ExchangleBot
User-agent: exif-search
User-agent: Experibot
User-agent: ExpertSearchSpider
User-agent: Exploratodo
User-agent: ExtLinksBot
User-agent: Ezooms
User-agent: FacebookBot
User-agent: factbot
User-agent: FairShare
User-agent: Falconsbot
User-agent: FAST bot
User-agent: fastbot crawler
User-agent: FatBot
User-agent: FauBot
User-agent: Faveeo
User-agent: Feedage
User-agent: Feedbin
User-agent: FeedBooster
User-agent: FeedBucket
User-agent: FeedBunch
User-agent: FeedBurner
User-agent: FeedCatBot
User-agent: feeder.co
User-agent: Feedfetcher-Google
User-agent: FeedFinder
User-agent: Feedly
User-agent: Feedspotbot
User-agent: Feedwind
User-agent: FemtosearchBot
User-agent: Fetch-Guess
User-agent: FFbot
User-agent: findlinks
User-agent: Findxbot
User-agent: firmilybot
User-agent: flatlandbot
User-agent: FlightDeckReportsBot
User-agent: FlipboardBot
User-agent: Flocke bot
User-agent: FollowSite Bot
User-agent: Fooooo_Web_Video_Crawl
User-agent: forensiq
User-agent: FoundSeoTool
User-agent: freefind
User-agent: FreeWebMonitoring SiteChecker
User-agent: FyberSpider
User-agent: g2reader-bot
User-agent: GAChecker
User-agent: Gaisbot
User-agent: ganarvisitas
User-agent: GarlikCrawler
User-agent: GC3pro
User-agent: Geek-Tools
User-agent: GeliyooBot
User-agent: Genderanalyzer
User-agent: Genieo Web filter
User-agent: GentleSource Short URL Checker
User-agent: GetFoundBot
User-agent: GetintentCrawler
User-agent: GetLinkInfo.com
User-agent: GetProxi.es-bot
User-agent: getroot.sh crawler
User-agent: Ghost Inspector
User-agent: ghostery bot
User-agent: GIDBot
User-agent: GiftBot
User-agent: gimmeUSAbot
User-agent: GingerCrawler
User-agent: gipo-crawler
User-agent: Girafabot
User-agent: github-camo
User-agent: glindahl-cocrawler
User-agent: GloomarBot
User-agent: Gluten Free Crawler
User-agent: gocrawl
User-agent: GOFORITBOT
User-agent: GomezAgent
User-agent: gonzo
User-agent: gooblog
User-agent: Goodzer
User-agent: Google AppsViewer
User-agent: Google-HotelAdsVerifier
User-agent: Google-Youtube-Links
User-agent: Googlebot
User-agent: GoogleImageProxy
User-agent: GoogleDocs
User-agent: GoogleSites
User-agent: GoSquared
User-agent: GotSiteMonitor
User-agent: GozaikBot
User-agent: Grabify.link
User-agent: Grahambot
User-agent: Grammarly
User-agent: GrapeshotCrawler
User-agent: Grobbot
User-agent: GroupHigh
User-agent: GTmetrix
User-agent: GurujiBot
User-agent: Hailoobot
User-agent: Hatena
User-agent: Hatena
User-agent: hawkReader
User-agent: hCardValidator
User-agent: HeartRailsBot
User-agent: heritrix
User-agent: HetrixTools crawler
User-agent: Heurekabot
User-agent: historious
User-agent: hledejLevne.cz
User-agent: Hlidam.to robot
User-agent: Holmes
User-agent: HolmesBot
User-agent: HomeTags
User-agent: HootSuite crawler
User-agent: hosterstats
User-agent: HostTracker
User-agent: houzzbot
User-agent: htrix
User-agent: HTTP-Header-Abfrage
User-agent: HuaweiSymantecSpider
User-agent: HubPages
User-agent: HubSpot Crawler
User-agent: HybridBot
User-agent: Hypefactors crawler
User-agent: HyperCrawl
User-agent: HyperZbozi.cz Feeder
User-agent: HypeStat
User-agent: hyScore.io crawler
User-agent: IAB crawler
User-agent: IAS crawler
User-agent: IBM Watson
User-agent: ICC-Crawler
User-agent: ichiro
User-agent: iCjobs
User-agent: IDBot
User-agent: IdeelaborPlagiaat
User-agent: IDG/IT
User-agent: idmarch
User-agent: IDwhois
User-agent: Iframely
User-agent: IlTrovatore
User-agent: ImageFetcher
User-agent: ImaggaBot
User-agent: immediatenet thumbnails
User-agent: ImplisenseBot
User-agent: import.io
User-agent: InbyBot
User-agent: IndeedBot
User-agent: Infegy
User-agent: Influencebot
User-agent: Infohelfer
User-agent: inoreader.com
User-agent: Inspingbot
User-agent: IntegromeDB
User-agent: Interdose AntiSpamBot
User-agent: InternetSeer
User-agent: internetVista monitor
User-agent: IP-Guide.com Crawler
User-agent: ips-agent
User-agent: ipv6-test.com validator
User-agent: iqdb
User-agent: Irokez.cz monitoring
User-agent: Is it up
User-agent: isa
User-agent: iskanie
User-agent: ISOWQ
User-agent: IstellaBot
User-agent: isUp.li Website Monitoring
User-agent: it2media-domain-crawler
User-agent: iThemes crawler
User-agent: IXEbot
User-agent: iZSearch.com
User-agent: Jabse.com Crawler
User-agent: JadynAveBot
User-agent: JAHHO crawler
User-agent: Jambot
User-agent: JamesBOT
User-agent: janforman.com
User-agent: Jeffrey Exif Viewer
User-agent: JikeSpider
User-agent: Job Roboter Spider
User-agent: JobboerseBot
User-agent: JobdiggerSpider
User-agent: JobKereso
User-agent: johnhew crawler
User-agent: Jooblebot
User-agent: jpg-newsbot
User-agent: JUST-CRAWLER
User-agent: Jyxobot
User-agent: K7 Bot
User-agent: kakao crawler
User-agent: KaloogaBot
User-agent: Karneval-Bot
User-agent: Kaspersky Lab bot
User-agent: KAZ.KZ_Bot
User-agent: kazbtbot
User-agent: KD Bot
User-agent: KeepRight OpenStreetMap Checker
User-agent: Kemvibot
User-agent: KeyCDN bot
User-agent: KimonoLabs
User-agent: kinshoo
User-agent: KOCMOHABT bot
User-agent: KomodiaBot
User-agent: KrOWLer
User-agent: kulturarw
User-agent: Kyoto-Crawler
User-agent: L.webis
User-agent: Laserlikebot
User-agent: LCC
User-agent: LDSpider
User-agent: Lead Inspector Crawler
User-agent: Leikibot
User-agent: LemurWebCrawler
User-agent: letsencrypt bot
User-agent: LexxeBot
User-agent: librabot
User-agent: Licorne crawler
User-agent: LightspeedSystems Crawler
User-agent: Lijit
User-agent: lincobot
User-agent: LinguaBot
User-agent: linguatools
User-agent: Linguee Bot
User-agent: Link Valet Online
User-agent: LinkAider
User-agent: LinkAlarm
User-agent: linkapediabot
User-agent: linkdexbot
User-agent: LinkedInBot
User-agent: linkfluence
User-agent: LinkMarket-Bot
User-agent: LinkpadBot
User-agent: linkpeek
User-agent: LinkStats Bot
User-agent: LinkTiger
User-agent: link_thumbnailer
User-agent: LinqiaBot
User-agent: livedoor ScreenShot
User-agent: LivelapBot
User-agent: LoadImpact
User-agent: LoadTimeBot
User-agent: LongURL
User-agent: ltbot
User-agent: ltx71
User-agent: LuminateBot
User-agent: LumpImageSearch
User-agent: LXRbot
User-agent: LYT.SR
User-agent: MagiBot
User-agent: magpie-crawler
User-agent: Mail.Ru bot
User-agent: MailDBBot
User-agent: make.myUrl
User-agent: manualzz bot
User-agent: Mappy
User-agent: MarketBrewBot
User-agent: masscan
User-agent: MaxPointCrawler
User-agent: MB-SiteCrawler
User-agent: mbot
User-agent: meanpathbot
User-agent: Mediatoolkitbot
User-agent: Mediumbot
User-agent: MegaIndex
User-agent: melodibot
User-agent: MeMoNewsBot
User-agent: memoryBot
User-agent: Mergadobot
User-agent: MergeFlow-PageReader
User-agent: metacdn
User-agent: MetaCommentBot
User-agent: MetaGeneratorCrawler
User-agent: metager2-bot
User-agent: MetaHeadersBot
User-agent: MetaInspector
User-agent: MetaJobBot
User-agent: MetamojiCrawler
User-agent: METASpider
User-agent: Metaspinner
User-agent: MetaURI
User-agent: MFGPagesBot
User-agent: mfibot
User-agent: MIA Bot
User-agent: MiaDev
User-agent: Microsearch.ru Bot
User-agent: mindUpBot
User-agent: Miniflux
User-agent: MixBot
User-agent: Mixmax crawler
User-agent: Mixnode
User-agent: MixrankBot
User-agent: MJ12bot
User-agent: MLBot
User-agent: MnoGoSearch
User-agent: Moatbot
User-agent: moba-crawler
User-agent: MojeekBot
User-agent: monitis
User-agent: Monitive Uptime Monitoring Bot
User-agent: Monitority
User-agent: Monsidobot
User-agent: montastic-monitor
User-agent: MonTools.Com
User-agent: MOQU ADV crawlers
User-agent: Moreover
User-agent: MotoMinerBot
User-agent: Motoricerca-Robots.txt-Checker
User-agent: mowser
User-agent: Mozilla Metadata Service
User-agent: Mp3Bot
User-agent: MrakyPraceCzBot
User-agent: MS Cognitive Services
User-agent: MSRBOT
User-agent: Multiviewbot
User-agent: musobot
User-agent: Najdi.si
User-agent: NalezenCzBot
User-agent: NaverBot
User-agent: Nekstbot
User-agent: NerdByNature.Bot
User-agent: NerdyBot
User-agent: Netcraft crawler
User-agent: netEstate Crawler
User-agent: netk6 crawler
User-agent: Netpeak bot
User-agent: Netpursual
User-agent: NetResearchServer
User-agent: Netseer
User-agent: NetShelter ContentScan
User-agent: NetTrack.info bot
User-agent: Netvibes
User-agent: NeumobBot
User-agent: Neustar WPM
User-agent: NeutrinoAPI
User-agent: NewShareCounts.com
User-agent: NG
User-agent: nic.it crawler
User-agent: NIF
User-agent: Nigma.ru
User-agent: NIXStatsbot
User-agent: NLNZ_IAHarvester
User-agent: Nmap
User-agent: Nodemeter
User-agent: NodePing
User-agent: nodestackbot
User-agent: nomore404.com robot
User-agent: Norton-Safeweb
User-agent: NotifyNinja
User-agent: NTENTbot
User-agent: Nuesbyte
User-agent: Nuhk
User-agent: Nutch
User-agent: Nuzzel
User-agent: nworm
User-agent: Nymesis
User-agent: oBot
User-agent: Ocelli
User-agent: Office 365 viewer
User-agent: oku-taka-lab-bot
User-agent: omgilibot
User-agent: OmniExplorer_Bot
User-agent: Online Domain Tools
User-agent: online-webceo-bot
User-agent: OnPageBot
User-agent: Open Web Analytics Bot
User-agent: OpenCalaisSemanticProxy
User-agent: Opendi Bot
User-agent: OpenDocumentCrawler
User-agent: OpenfosBot
User-agent: OpenHoseBot
User-agent: OpenindexSpider
User-agent: Openstat
User-agent: OpenWebSpider
User-agent: OptimizationCrawler
User-agent: OrangeBot
User-agent: Orbiter
User-agent: OrgbyBot
User-agent: OrgProbe
User-agent: orion-semantics.com
User-agent: OsObot
User-agent: outbrain
User-agent: ownCloud Crawler
User-agent: Page Analyzer
User-agent: Page Valet
User-agent: Page2RSS
User-agent: PagePeeker
User-agent: PagesInventory
User-agent: Pagespeed
User-agent: page_verifier
User-agent: Pandeo Bot
User-agent: Panopta
User-agent: panscient.com
User-agent: PaperLiBot
User-agent: ParchBot
User-agent: parsijoo-bot
User-agent: PayPal IPN
User-agent: PDFDriveCrawler
User-agent: pdffillerbot
User-agent: Pearltrees
User-agent: Peeplo Screenshot Bot
User-agent: Peepowbot
User-agent: peerindex
User-agent: Peew
User-agent: PercolateCrawler
User-agent: PhantomJsCloud.com
User-agent: Photon
User-agent: Pi-Monster
User-agent: Pingability.com bot
User-agent: Pingdom.com bot
User-agent: PINGOMETER
User-agent: Pingoscope
User-agent: PingSpot
User-agent: Pinterest bot
User-agent: PiplBot
User-agent: PiStato
User-agent: Pixray-Seeker
User-agent: Plukkie
User-agent: pmoz.info ODP link checker
User-agent: PocketParser
User-agent: Pompos
User-agent: Porkbun
User-agent: Port Monitor check service
User-agent: PostPost
User-agent: PowerMapper Crawler
User-agent: pr-cy.ru Bot
User-agent: Prerender
User-agent: Prerender.cloud
User-agent: Priceonomics Analysis Engine
User-agent: Princetonbot
User-agent: PrintFriendly.com
User-agent: PritTorrent
User-agent: PrivacyAwareBot
User-agent: PrivateSearch
User-agent: probethenet.com scanner
User-agent: ProCogBot
User-agent: ProCogSEOBot
User-agent: ProductoDownloadUrlBot
User-agent: Profound scrapyproject
User-agent: Project 25499
User-agent: Promotion_Tools
User-agent: Protopage
User-agent: proximic
User-agent: psbot
User-agent: PTST
User-agent: Pulsepoint crawler
User-agent: Pu_iN
User-agent: pyspider
User-agent: Qirina Hurdler
User-agent: Qseero
User-agent: Qualidator.com Bot
User-agent: QuerySeekerSpider
User-agent: Quick-Crawler
User-agent: quickobot
User-agent: QuoraBot
User-agent: Qwantify
User-agent: R6 bot
User-agent: RADaR-Bot
User-agent: RankActive Bot
User-agent: RankFlex.com Webspider
User-agent: RankingBot
User-agent: RankSonicSiteAuditor
User-agent: RankurBot
User-agent: RankvalBot
User-agent: RavenCrawler
User-agent: Re-re crawler
User-agent: Readability
User-agent: REDbot
User-agent: RedesScrapy
User-agent: RelateIQ Crawler
User-agent: ResponseCodeTest
User-agent: Riddlerbot
User-agent: Rival IQ
User-agent: RobotsChecker
User-agent: Robots_Tester
User-agent: Robozilla
User-agent: RockhallFundingCrawler
User-agent: rogerbot
User-agent: Ronzoobot
User-agent: ROR Sitemap Generator
User-agent: RSSingBot
User-agent: RSSMicro.com RSS/Atom Feed Robot
User-agent: RukiCrawler
User-agent: Ruky-Roboter
User-agent: Runet-Research-Crawler
User-agent: RyzeCrawler
User-agent: s4a
User-agent: SafeAds.xyz bot
User-agent: SafeDNSBot
User-agent: SAI Crawler
User-agent: SalesIntelligent
User-agent: Saleslift
User-agent: SanszBot
User-agent: SauceNAO
User-agent: SBA Research bot
User-agent: SBIder
User-agent: SBSearch
User-agent: scanbot
User-agent: Scarlett
User-agent: SCFCrawler
User-agent: Scoop
User-agent: Scopia crawler
User-agent: ScoutJet
User-agent: Scrapy
User-agent: scrapy-redis
User-agent: ScreenerBot Crawler
User-agent: ScreenripBot
User-agent: Scrubby
User-agent: search.KumKie.com
User-agent: Search17Bot
User-agent: SearchmetricsBot
User-agent: SecurityResearchBot
User-agent: seegnifybot
User-agent: Seeker
User-agent: Semager
User-agent: Semantic Health Web Crawler
User-agent: semantic-visions.com crawler
User-agent: semanticbot
User-agent: SemanticScholarBot
User-agent: Semantifire
User-agent: Semasio crawler
User-agent: Semiocast crawler
User-agent: SentiBot
User-agent: SEO Browser
User-agent: Seo Servis
User-agent: seo-nastroj.cz
User-agent: Seobility
User-agent: SEOCentro bot
User-agent: seocharger-robot
User-agent: SeoCheckBot
User-agent: SEODat
User-agent: SEOdiver
User-agent: SEOENGBot
User-agent: SEOkicks-Robot
User-agent: Seologies
User-agent: SEOlyticsCrawler
User-agent: SeopultContentAnalyzer
User-agent: Seoterritory.com
User-agent: seplinkbot
User-agent: Server Density
User-agent: servernfo.com
User-agent: ServiceUptime.robot
User-agent: SetCronJob
User-agent: Setoozbot
User-agent: Setoozbot
User-agent: SeznamBot
User-agent: sg-Orbiter
User-agent: Shareaholicbot
User-agent: Shelob
User-agent: ShopAlike
User-agent: Shoppimon
User-agent: ShopWiki
User-agent: showsite crawler
User-agent: ShowyouBot
User-agent: SimplyFast.info
User-agent: Sistrix crawler
User-agent: Site-Shot
User-agent: Site24x7
User-agent: SiteAnalyzerBot
User-agent: SiteCheckerBot
User-
100 31472  100 31472    0     0   4150      0  0:00:07  0:00:07 --:--:--  7248
agent: SiteCondor
User-agent: Sitedomain-Bot
User-agent: SiteExplorer
User-agent: SiteGuardian
User-agent: Siteimprove.com bot
User-agent: Siteliner
User-agent: Siteluxbot
User-agent: SiteProbe
User-agent: SiteTruth.com
User-agent: SiteUptime.com
User-agent: SiteXY Webmaster tools
User-agent: SKB Kontur bot
User-agent: SkypeUriPreview
User-agent: Slackbot
User-agent: SmabblerBot
User-agent: smart.apnoti.com Robot
User-agent: SMTBot
User-agent: Snapbot
User-agent: SnapSearch
User-agent: SniffRSS
User-agent: sniptracker
User-agent: SnowHaze SearcH
User-agent: socialbm_bot
User-agent: Sogou web spider
User-agent: SOLOFIELD bot
User-agent: SolomonoBot
User-agent: sonarwhal crawler
User-agent: Sosospider
User-agent: spbot
User-agent: special_archiver
User-agent: Specificfeeds
User-agent: Speedy Spider
User-agent: SpiderLing
User-agent: Spinn3r
User-agent: SpuhexBot
User-agent: SputnikBot
User-agent: SpyOnWeb
User-agent: SSL Labs
User-agent: SSL-Crawler
User-agent: ssl-tools.net
User-agent: SSLBot
User-agent: StackRambler
User-agent: startmebot
User-agent: Statastico
User-agent: StatoolsBot
User-agent: StatusCake
User-agent: Steeler
User-agent: Stephan Kopp bot
User-agent: STINGbot
User-agent: Storm-crawler
User-agent: StorygizeBot
User-agent: stq_bot
User-agent: Streamline3Bot
User-agent: Strokebot
User-agent: StudioFACA Search
User-agent: studylib bot
User-agent: StumbleUpon crawler
User-agent: Sucuri bot
User-agent: Sufog
User-agent: suggybot
User-agent: sukibot_heritrix
User-agent: Summify
User-agent: Super Monitoring
User-agent: Superarama
User-agent: SuperPagesUrlVerifyBot
User-agent: SurcentroBot
User-agent: SurdotlyBot
User-agent: Surphace Scout
User-agent: SurveyBot
User-agent: SWEBot
User-agent: Swobblspider
User-agent: SygolBot
User-agent: Symfony Spider
User-agent: SynTrafficBot
User-agent: Sysomos
User-agent: Szukacz
User-agent: TA SEO Crawler
User-agent: Taboolabot
User-agent: Tagoobot
User-agent: tagSeoBot
User-agent: taptubot
User-agent: Taringa
User-agent: TarmotGezgin
User-agent: TcpTools Bot
User-agent: TeamSpeak bot
User-agent: techleadzbot
User-agent: Technoratibot
User-agent: TeeRaidBot
User-agent: TelegramBot
User-agent: Testomatobot
User-agent: Tetrahedron
User-agent: theoldreader.com
User-agent: Thinglink ImageBot
User-agent: Thumbnail.CZ robot
User-agent: ThumbShots-Bot
User-agent: thumbshots-de-Bot
User-agent: Thumbshots.ru
User-agent: ThumbSniper
User-agent: TinEye
User-agent: TipTop
User-agent: tldstat.com crawler
User-agent: TomTom places company search
User-agent: Tools4noobs.com Spider
User-agent: Topicbot
User-agent: topster_de-crawler
User-agent: Toread-Crawler
User-agent: Touche
User-agent: ToutiaoSpider
User-agent: Toweyabot
User-agent: Traackr.com
User-agent: trendictionbot
User-agent: trovitBot
User-agent: TurnitinBot
User-agent: TweetedTimes Bot
User-agent: TweetmemeBot
User-agent: TwengaBot
User-agent: Twiceler
User-agent: Twikle
User-agent: Twingly Recon
User-agent: Twitterbot
User-agent: TWMBot
User-agent: UASlinkChecker
User-agent: uclassify.com
User-agent: uCrawler
User-agent: uMBot
User-agent: UnisterBot
User-agent: Unshorten.It
User-agent: Untiny
User-agent: UnwindFetchor
User-agent: updated
User-agent: updown.io bot
User-agent: Updownerbot
User-agent: Uptimebot
User-agent: UptimeDog
User-agent: UptimeRobot
User-agent: Uptimia
User-agent: URLAppendBot
User-agent: urlfan-bot
User-agent: Urlfilebot (Urlbot)
User-agent: URLitor.com
User-agent: UrlTrends
User-agent: UXCrawlerBot
User-agent: Vagabondo
User-agent: vebidoobot
User-agent: VeBot
User-agent: Vedma
User-agent: VegeBot
User-agent: Vegi bot
User-agent: Venafi crawler
User-agent: Veoozbot
User-agent: VideoSurf_bot
User-agent: Virusdie crawler
User-agent: virustotal
User-agent: Visbot
User-agent: vkShare
User-agent: voltron
User-agent: Vorboss Web Crawler
User-agent: voyager
User-agent: VSAgent
User-agent: VSB-TUO
User-agent: w3dt.net httphr
User-agent: Wappalyzer
User-agent: WASALive-Bot
User-agent: WatchMouse
User-agent: WBSearchBot
User-agent: WbSrch
User-agent: WDT.io
User-agent: web-capture.net
User-agent: Web-Monitoring
User-agent: Web-sniffer
User-agent: Web-sniffer.me/1.0.0
User-agent: WebAlta Crawler
User-agent: WebarooBot
User-agent: Webauskunft.org
User-agent: WebCookies
User-agent: WebCorp
User-agent: Webcron.org crawler
User-agent: WebImages
User-agent: webinatorbot
User-agent: webmastercoffee
User-agent: WebNL
User-agent: webnumbrFetcher
User-agent: WebRankSpider
User-agent: Webshot
User-agent: websitepulse bot
User-agent: Websnapr
User-agent: WebTarantula.com Crawler
User-agent: WebThumb
User-agent: WebThumbnail
User-agent: WebWatch/Robot_txtChecker
User-agent: WEPA
User-agent: WeSEE
User-agent: WeViKaBot
User-agent: wf_crawler
User-agent: WGETbot
User-agent: WhatsMyIP.org
User-agent: Whibse
User-agent: Who.is Bot
User-agent: Whoismindbot
User-agent: Whoiswebsitebot
User-agent: Whynder Magnet
User-agent: WikiDo
User-agent: WikioFeedBot
User-agent: wikiwix-bot
User-agent: Willow Internet Crawler
User-agent: WillyBot
User-agent: WinWebBot
User-agent: WMCAI_robot
User-agent: wmtips.com
User-agent: Woko
User-agent: woobot
User-agent: WordPress.com bots
User-agent: woriobot
User-agent: WorldBrewBot
User-agent: WormlyBot
User-agent: WoTBoT
User-agent: Wotbox
User-agent: WP Engine
User-agent: wprecon.com survey
User-agent: WPScan
User-agent: WPScans
User-agent: wsAnalyzer
User-agent: wscheck.com
User-agent: WURFL ImageEngine
User-agent: x28-job-bot
User-agent: XING-contenttabreceiver
User-agent: XmarksFetch
User-agent: XML Schema Validator
User-agent: XML Sitemaps Generator
User-agent: XmlSitemapGenerator
User-agent: XoviBot
User-agent: XRL
User-agent: Yaanb
User-agent: yacybot
User-agent: Yahoo Ad monitoring
User-agent: Yahoo!
User-agent: YamanaLab-bot
User-agent: YandeG
User-agent: YandexBot
User-agent: Yanga
User-agent: YellowLabTools
User-agent: yelpspider
User-agent: YesupBot
User-agent: YetiBOT
User-agent: Yieldbot
User-agent: YioopBot
User-agent: YisouSpider
User-agent: Yoleo Consumer
User-agent: YOOBI.de bot
User-agent: yoozBot
User-agent: YottaaMonitor
User-agent: YottaBot
User-agent: YoudaoBot
User-agent: YowedoBot
User-agent: YP crawler
User-agent: YRSpider
User-agent: YYSpider
User-agent: ZeerchBot
User-agent: Zemanta
User-agent: zitebot
User-agent: ZnajdzFoto/ImageBot
User-agent: Zombiebot
User-agent: ZookaBot
User-agent: ZoomBot
User-agent: ZoomInfo bot
User-agent: ZumBot
User-agent: ZyBorg
User-agent: ^Nail
User-agent: Alexibot
User-agent: AppEngine
User-agent: Aqua_Products
User-agent: archive
User-agent: asterias
User-agent: b2w/0.1
User-agent: BackDoorBot/1.0
User-agent: BecomeBot
User-agent: BlekkoBot
User-agent: Blexbot
User-agent: BlowFish/1.0
User-agent: Bookmark search tool
User-agent: BotALot
User-agent: BuiltBotTough
User-agent: Bullseye/1.0
User-agent: BunnySlippers
User-agent: CheeseBot
User-agent: CherryPicker
User-agent: CherryPickerElite/1.0
User-agent: CherryPickerSE/1.0
User-agent: chroot
User-agent: Copernic
User-agent: CopyRightCheck
User-agent: cosmos
User-agent: Crescent
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
User-agent: DittoSpyder
User-agent: dotbot
User-agent: dumbot
User-agent: EmailCollector
User-agent: EmailSiphon
User-agent: EmailWolf
User-agent: Enterprise_Search
User-agent: Enterprise_Search/1.0
User-agent: EroCrawler
User-agent: es
User-agent: exabot
User-agent: ExtractorPro
User-agent: FairAd Client
User-agent: Flaming AttackBot
User-agent: Foobot
User-agent: GetRight/4.2
User-agent: gigabot
User-agent: grub
User-agent: grub-client
User-agent: Go-http-client
User-agent: Harvest/1.5
User-agent: Hatena Antenna
User-agent: hloader
User-agent: http://www.SearchEngineWorld.com bot
User-agent: http://www.WebmasterWorld.com bot
User-agent: httplib
User-agent: humanlinks
User-agent: ia_archiver
User-agent: ia_archiver/1.6
User-agent: InfoNaviRobot
User-agent: Iron33/1.0.2
User-agent: JennyBot
User-agent: Jetbot
User-agent: Jetbot/1.0
User-agent: Jorgee
User-agent: Kenjin Spider
User-agent: Keyword Density/0.9
User-agent: larbin
User-agent: LexiBot
User-agent: libWeb/clsHTTP
User-agent: LinkextractorPro
User-agent: LinkScan/8.1a Unix
User-agent: LinkWalker
User-agent: LNSpiderguy
User-agent: looksmart
User-agent: lwp-trivial
User-agent: lwp-trivial/1.34
User-agent: Mata Hari
User-agent: Megalodon
User-agent: Microsoft URL Control
User-agent: Microsoft URL Control - 5.01.4511
User-agent: Microsoft URL Control - 6.00.8169
User-agent: MIIxpc
User-agent: MIIxpc/4.2
User-agent: Mister PiX
User-agent: moget
User-agent: moget/2.1
User-agent: mozilla
User-agent: mozilla/3
User-agent: mozilla/4
User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP)
User-agent: mozilla/5
User-agent: MSIECrawler
User-agent: naver
User-agent: NetAnts
User-agent: NetMechanic
User-agent: NICErsPRO
User-agent: Offline Explorer
User-agent: Openbot
User-agent: Openfind
User-agent: Openfind data gathere
User-agent: Oracle Ultra Search
User-agent: PerMan
User-agent: ProPowerBot/2.14
User-agent: ProWebWalker
User-agent: Python-urllib
User-agent: QueryN Metasearch
User-agent: Radiation Retriever 1.1
User-agent: RepoMonkey
User-agent: RepoMonkey Bait & Tackle/v1.01
User-agent: RMA
User-agent: scooter
User-agent: searchpreview
User-agent: SemrushBot
User-agent: SemrushBot-SA
User-agent: SiteSnagger
User-agent: sootle
User-agent: SpankBot
User-agent: spanner
User-agent: Stanford
User-agent: Stanford Comp Sci
User-agent: Stanford CompClub
User-agent: Stanford CompSciClub
User-agent: Stanford Spiderboys
User-agent: SurveyBot_IgnoreIP
User-agent: suzuran
User-agent: Szukacz/1.4
User-agent: Teleport
User-agent: TeleportPro
User-agent: Telesoft
User-agent: Teoma
User-agent: The Intraformant
User-agent: TheNomad
User-agent: toCrawl/UrlDispatcher
User-agent: True_Robot
User-agent: True_Robot/1.0
User-agent: turingos
User-agent: Typhoeus
User-agent: URL Control
User-agent: URL_Spider_Pro
User-agent: URLy Warning
User-agent: VCI
User-agent: VCI WebViewer VCI WebViewer Win32
User-agent: Web Image Collector
User-agent: WebAuto
User-agent: WebBandit
User-agent: WebBandit/3.50
User-agent: WebCopier
User-agent: WebEnhancer
User-agent: WebmasterWorld Extractor
User-agent: WebmasterWorldForumBot
User-agent: WebSauger
User-agent: Website Quester
User-agent: Webster Pro
User-agent: WebStripper
User-agent: WebVac
User-agent: WebZip
User-agent: WebZip/4.0
User-agent: Wget
User-agent: Wget/1.5.3
User-agent: Wget/1.6
User-agent: WWW-Collector-E
User-agent: Xenu's
User-agent: Xenu's Link Sleuth 1.1c
User-agent: Zeus
User-agent: Zeus 32297 Webster Pro V2.9 Win32
User-agent: Zeus Link Scout
User-agent: MBCrawler
User-agent: MauiBot
User-agent: Attracta

Disallow: /