New hacked site notifications in search results

Webmaster level: All

Today we’ve added a new notification to our search results that helps people know when a site may have been hacked. We’ve provided notices for malware for years, which also involve a separate warning page. Now we’re expanding the search results notifications to help people avoid sites that may have been compromised and altered by a third party, typically for spam. When a user visits a site, we want her to be confident the information on that site comes from the original publisher.

Here’s what the notification looks like:

Clicking the “This site may be compromised” link brings you to an article in our Help Center which explains more about the notice. Meanwhile, clicking the result itself brings you to the target website, as expected.

We use a variety of automated tools to detect common signs of a hacked site as quickly as possible. When we detect something suspicious, we’ll add the notification to our search results. We’ll also do our best to contact the site’s webmaster via their Webmaster Tools account and any contact email addresses we can find on the webpage. We hope webmasters will also appreciate these notices, because it will help you more quickly discover when someone may be abusing your site so you can correct the problem.

Of course, we also understand that webmasters may be concerned that these notices are impacting their traffic from search. Rest assured, once the problem has been fixed, the warning label will be automatically removed from our search results, usually in a matter of days. You can also request a review of your site to accelerate removal of the notice.

If you see this notification appearing on your site’s listing, please take a look at the instructions in our Help Center to learn how you can begin to address the problem. Together, we can make the web a safer place.

Posted by Gideon Wald, Associate Product Manager

Webmaster Tools’ Holiday Update

Webmaster Level: All

Just in time for the holidays, the Webmaster Tools team has updated the "Search queries" and "Links to your site" features.

Search queries with top pages:
Throughout the past year we’ve made some significant changes to the search queries feature in Webmaster Tools. We've received lots of feedback about this tremendously popular feature. One frequent request we heard was that people wanted to be able to see search queries data for their site’s individual pages. Well, we totally agreed that this would be useful and promptly set out to add this functionality to search queries. The fruits of our effort have finally ripened enough on the vine and are ready for you to enjoy. Now when you visit the search queries feature in Webmaster Tools you'll see a new tab titled "Top Pages." The "Top Pages" tab lists impression, click, and position data for the top pages on your site based on their performance in Google's search results.


If you click on one of the individual pages listed you’ll see a list of the queries driving traffic to that page, along with impressions and number of clicks for each query.


Just like in the "Top queries" view, you can click on a specific query to see more detailed data and evaluate how the query is performing across the whole site.


To make filtering in search queries even easier, we've added pie charts to show visually the proportions of search type, location and traffic. Also in the "Top queries" view, you can now specify “containing” or “not containing” when filtering queries.


Links to your site with intermediate links:
Links to your site now shows when a particular URL redirects. If there's a link to your site that links to URL1 which then redirects to URL2, we are now showing the link from URL1 to URL2 as an intermediate link. We also added a "Download all links" option to all the tables in the Links to your site feature.


Now that you know about these updates, please take a few minutes to check them out for your site. We hope it will provide you with a little bit of extra joy this holiday season. Let us know what you think by submitting a comment here or posting in our Webmaster Help Forum. Happy Holidays from the Webmaster Tools team!

Posted by Jonathan Simon, Webmaster Trends Analyst

Controlling crawling and indexing now documented on code.google.com

Webmaster level: All

Do you know how Google's crawler, Googlebot, handles conflicting directives in your robots.txt file? Do you know how to prevent a PDF file from being indexed? Do you know Googlebot’s favorite song? The answers to these questions (except for the last one :)), along with lots of other information about controlling the crawling and indexing of your site, are now available on code.google.com:

Controlling crawling and indexing



Now site owners have a comprehensive resource where they can learn about robots.txt files, robots meta tags, and X-Robots-Tag HTTP header directives. Please share your comments, and if you have questions you can post them in our Webmaster Help Forum.

Posted by Jonathan Simon, Webmaster Trends Analyst

Petits fours in your search results

Webmaster Level: All

Recently we made a change to show more results from a domain for certain types of queries -- this helped searchers get to their desired result even faster. Today we’re expanding the feature so that, when appropriate, more queries show additional results from a domain. As a webmaster, you’ll appreciate the fact that these results may bring targeted visitors directly to the pages they’re interested in.

Here’s an example: in the past, the query [moma] (the Museum of Modern Art), might have triggered two results from the official site:

With this iteration, our search results may show:

• Up to four web results from each domain (i.e., several domains may have multiple results)
• Single-line snippets for the additional results, to keep them compact

As before, we still provide links to results from a variety of domains to ensure people find a diverse set of sources relevant to their searches. However, when our algorithms predict pages from a particular site are likely to be most relevant, it makes sense to provide additional direct links in our search results.

Like all the hundreds of changes we make a year, we’re trying to help users quickly reach their desired result. Even though we’re constantly improving our algorithms, our general advice still holds true: create compelling, search-engine friendly sites in order to attract users, buzz, and often targeted traffic!

Written by Harvey Jones, Software Engineer

Call for webspam reports in Thai, Indonesian, Romanian, Czech and Farsi

Webmaster Level: All

Update on May 19, 2010: We have several translated versions of this post! If you're more comfortable reading Thai, Indonesian, Romanian, Czech, or Farsi, the links above will take you to your preferred version. Thanks again for your help.

We pay attention to dozens of different languages in our spam fighting, but sometimes we really want to drill down and concentrate on a small number of languages. We’d like to ask for your help to identify webspam in Thai, Indonesian, Romanian, Czech and Farsi. If you know of sites that violate our webmaster guidelines in these languages, please send us a spam report. We use this information not only to look at the sites listed in reports, but also to improve our effectiveness in the rest of your language on the web.

Thanks in advance for any data you send our way about spam in these languages. Of course, you’re always welcome to submit spam reports in other languages too!

Written by Viktor Nebehaj and Matt Cutts, Search Quality Team

GENERIC CIALIS on my website? I think my site has been hacked!

How to use "Fetch as Googlebot", part 1
Webmaster level: Intermediate

Has your site ever dropped suddenly from the index or disappeared mysteriously from search results? Have you ever received a notice that your site is using cloaking techniques? Unfortunately, sometimes a malicious party "hacks" a website: they penetrate the security of a site and insert undesirable content. Sophisticated attackers can camouflage this spammy or dangerous content so that it doesn't appear for normal users, and appears only to Googlebot, which could negatively impact your site in Google's results.

In such cases it used to be very difficult to detect the problem, because the site would appear normal in the eyes of the user. It may be possible that only requests with a User-agent: of Googlebot and coming from Googlebot's IP could see the hidden content. But that's over: with Fetch as Googlebot, the new Labs feature in Webmaster Tools, you can see exactly what Googlebot is seeing, and avoid any kind of cloaking problems. We'll show you how:

Let's imagine that Bob, the administrator of www.example.com, is searching for his site but he finds this instead:



That's strange, because when he looks at the source code of www.example.com, it looks fine:



With much surprise Bob may receive a notice from Google warning him that his site is not complying with Google's quality guidelines. Fortunately he has his site registered with Webmaster Tools, let's see how he can check what Googlebot sees:

First Bob logs into Webmaster Tools and selects www.example.com. The Fetch as Googlebot feature will be at the bottom of the navigation menu, in the Labs section:



The page will contain a field where you can insert the URL to fetch. It can also be left blank to fetch the homepage.



Bob can simply click Fetch and wait a few seconds. After refreshing the page, he can see the status of the fetch request. If it succeeds, he can click on the "Success" link...



...and that will show the details, with the content of the fetched page:



Aha! There's the spammy content! Now Bob can be certain that www.example.com has been hacked.

Confirming that the website has been hacked (and perhaps is still hacked) is an important step. It is, however, only the beginning. For more information, we strongly suggest getting help from your server administrator or hoster and reading our previous blog posts on the subject of hacked sites:

• My site's been hacked - now what?
• A quick security checklist for webmasters
• Message center warnings for hackable sites
• Best practices against hacking

If you have any questions about how to use the Fetch as Googlebot feature, feel free to drop by the Webmaster Help Forum. If you feel that your website might be hacked but are having problems resolving it, you might want to ask the experts in our "Malware and Hacked sites" category.

PS Keep in mind that once you have removed hacked content from your site, it will generally still take time for us to update our search results accordingly. There are a number of factors that affect crawling and indexing of your content so it's impossible to give a time-frame for that.

Written by Javier Tordable, Software Engineer and John Mueller, Webmaster Trends Analyst

Pros and cons of watermarked images

Webmaster Level: All

What's our take on watermarked images for Image Search? It's a complicated topic. I talked with Peter Linsley—my friend at the 'plex, video star, and Product Manager for Image Search—to hear his thoughts.

Maile: So, Peter... "watermarked images". Can you break it down for us?

Peter: It's understandable that webmasters find watermarking images beneficial.

Pros of watermarked images

• Photographers can claim credit/be recognized for their art.
• Unknown usage of the image is deterred.

If search traffic is important to a webmaster, then he/she may also want to consider some of our findings:

Findings relevant to watermarked images

• Users prefer large, high-quality images (high-resolution, in-focus).
• Users are more likely to click on quality thumbnails in search results. Quality pictures (again, high-res and in-focus) often look better at thumbnail size.
• Distracting features such as loud watermarks, text over the image, and borders are likely to make the image look cluttered when reduced to thumbnail size.

In summary, if a feature such as watermarking reduces the user-perceived quality of your image or your image's thumbnail, then searchers may select it less often. Preview your images at thumbnail size to get an idea of how the user might perceive it.

Maile: Ahh, I see: Webmasters concerned with search traffic likely want to balance the positives of watermarking with the preferences of their users -- keeping in mind that sites that use clean images without distracting artifacts tend to be more popular, and that this can also impact rankings. Will Google rank an image differently just because it's watermarked?

Peter: Nope. The presence of a watermark doesn't itself cause an image to be ranked higher or lower.

Do you have questions or opinions on the topic? Let's chat in the webmaster forum.

Written by Maile Ohye, Developer Programs Tech Lead