Welcome to PhantomPilots.com

Sign up for a weekly email of the latest drone news & information

Robots online? What?

Discussion in 'General Discussion' started by Phantomix, May 22, 2016.

  1. Phantomix

    Joined:
    Sep 6, 2014
    Messages:
    334
    Likes Received:
    51
    Location:
    Weston, MA
    When you look at "members online now", it says the members, guests, and robots. What are the robots? :confused:
     
  2. dirkclod

    dirkclod Moderator
    Staff Member

    Joined:
    May 9, 2014
    Messages:
    10,652
    Likes Received:
    5,705
    Location:
    Amory Mississippi
  3. Zerone

    Joined:
    Jul 28, 2015
    Messages:
    140
    Likes Received:
    70
    Location:
    S.Wales, UK
    The robot count is created by what are commonly known as web-crawlers, web-spiders, or just bots. They are usually created by search engines (google, yahoo, bing, etc) to update their search indexes and provide you with reasonably accurate and uptodate results. The bots are constantly crawling the web and although you can stop then crawling your own site if you wish by using "robot.txt" codes etc, they don't do any damage and actually help to propagate a new web-sites.
     
    dirkclod likes this.
  4. msinger

    Approved Vendor

    Joined:
    Oct 30, 2014
    Messages:
    18,990
    Likes Received:
    5,594
    Location:
    US
  5. NotARubicon

    Joined:
    Jan 29, 2016
    Messages:
    174
    Likes Received:
    48
    Location:
    Southern California / Inland Empire
    "bots" are also used to scrape (copy) content to be used on other websites without permission, slurp up email address for use by spammers, and to probe for security vulnerabilities to hack.. So they are also used for bad things..
    There are actually (approximately) 10000X more 'bad' bots roaming the web than there are 'good' bots..
     
  6. Zerone

    Joined:
    Jul 28, 2015
    Messages:
    140
    Likes Received:
    70
    Location:
    S.Wales, UK
    The OP specifically refered to the "Members Online Now" section, at which time I checked and there were 52 Robots listed and my response was to that specific query. It was not a technical tutorial on email harvesting, security vulnerabilities, key mapping, database excavation, or any other nasty which does not enter through the front door and so avoid being counted in the census.

    The 52 listed were Google 14, Bing 12, Yahoo 7, Baidu 19. All of which are recognised search engine bots, the last one being the main search engine in China.
     
  7. DefaultIT

    Joined:
    Jan 23, 2016
    Messages:
    188
    Likes Received:
    74
    Funny I just noticed that stat yesterday...and my mind went elsewhere
    T800onPP.gif
     
  8. NotARubicon

    Joined:
    Jan 29, 2016
    Messages:
    174
    Likes Received:
    48
    Location:
    Southern California / Inland Empire
    My response was to the other comment that "bots do no harm", (or something similar...)
    I apologize if my post has, in any way, reduced the quality of your life and am sorry that you had to take up so much time and energy in your response.

    PS: I would not be so confident about the nasty bots not getting through the door.. Especially the ones that spoof the Agent-ID and show up as Google, Bing, Yahoo, or Baidu... Especially since they totally ignore ROBOTS.TXT as you mentioned..
     
  9. Zerone

    Joined:
    Jul 28, 2015
    Messages:
    140
    Likes Received:
    70
    Location:
    S.Wales, UK
    duck-fly.jpg
     
    dirkclod likes this.