Forum Stats

  • 3,734,269 Users
  • 2,246,935 Discussions
  • 7,857,216 Comments

Discussions

Public homepage - google indexation problem

p.belanger
p.belanger Member Posts: 37
edited Nov 5, 2013 7:45AM in APEX Discussions

Hi,

I have some problem getting a description for a public Apex application homepage in Google. Right now, I get "A description for this result is not available because of this site's robots.txt – learn more.". We disabled Googlebots when not on session 0 to avoid being hit too many times and slowing the server down. Here's the content of robots.txt :

User-agent: *

Allow: /f?p=105:*:0

Allow: /pls/htmldb/f?p=105:*:0

Disallow: /

So in Google we see only this link :

  1. my-domain.com - A description for this result is not available because of this site's robots.txt – learn more.

my-domain.com redirects to the actual homepage (my-domain.com/pls/htmldb/f?p=105:34:0) using Apache RewriteRule :

RewriteRule ^/$ /pls/htmldb/f?p=105:34:0 [NS,R,L]

How can I change the display of Google search results to get the actual homepage with a description ?

--

Patrick

Tagged:

Answers

  • Jorge Rimblas
    Jorge Rimblas Member Posts: 710 Bronze Trophy

    Hi Patrick, I'm no expert on this, but I'm wondering, where is your robot.txt file located?

    If it's in the root (/) then because of your rewrite rule I don't think you can get to it. Did you try it  (my-domain.com/robots.txt) ?

    Perhaps you can have a explicit rule for robots.txt -> apex page with robots.txt info.  Or... a path with the actual file.

    Thanks

    -Jorge

  • p.belanger
    p.belanger Member Posts: 37

    Hi Jorge, there's no problem locating the robots.txt file.  In fact, i'm actually testing it on webmaster tools, so Google can find it.  I did another test this morning on webmaster tools using this robots.txt :

    User-agent: *

    Allow: /f?p=105:*:0

    Allow: /pls/htmldb/f?p=105:*:0

    Disallow: /f*

    Disallow: /pls/htmldb/f*

    instead of :

    User-agent: *

    Allow: /f?p=105*:*:0

    Allow: /pls/htmldb/f?p=105*:*:0

    Disallow: /

    I'm trying to block every Apex URL except application 105 when using session 0.  It now authorize my-domain.com/ so I guess that I might see a description on Google search results.  I'll modify my robots.txt and let you know if that works.

    Thanks

    --

    Patrick

  • jariola
    jariola Member Posts: 10,264 Bronze Crown

    Hi,

    I'm not expert on this area, and maybe I did miss something on your first post.

    But, isn't description come to search result from meta tag ?

    https://support.google.com/webmasters/answer/35624?hl=en

    Also, if you try fight that Google do not index all pages with session id, use link rel="canonical"

    https://support.google.com/webmasters/answer/139066?hl=en

    Regards,

    Jari

This discussion has been closed.