Ways to bypass a temporary Google ban for frequent access to the database - Profit Hunter

* Guest Post

This post is devoted to the peculiarities of working with the nTrendsMaster script, the mass verification of Google output for the number of competitors for the necessary queries and the time-out problem from Google in general. The post is very good in terms of expanding horizons (I had no idea about many things written here before), and in some way explains the causes of problems that you may have while using Google scripts and programs (including Market Samurai ).

The post is a compilation of comments and correspondence with the Internet entrepreneur Oleg Savchenko and is published with his permission.


Alignment is as follows (in any case, I have):

  • we launch the script and give it 100-200 words - everything works without problems ...
  • we start it again with the same amount ... and more ... if you don’t stand between the launches of the spouse at least half an hour, then literally after 3 launches (that is, after 400-600 checked words) errors on checking competitive pages ; - ((

After I immediately try to search for something in Google, I see a captcha. The script was launched from Denver, so the captcha says that my aypishnik temporarily went to the ban That is, in manual mode I can enter the code of this picture further for some time quietly use Google. But for about 5-6 hours more, this same captcha may randomly appear during any of my calls to Google (or may not appear). In any case, at this time using the script from the same address will not work, because it will not be able to recognize the image or drive in the code ...

What I see solutions:

1. Increased pauses between Google requests in script . But not just an increase in pauses, but also their randomization. Many desktop applications ( KRA PRO , IBP , and others) do not fall under the ban of Google for exactly this reason. However, they put between 7 to 30 seconds break between each request !!! If you just set 5 (7, 10, 15) seconds and make all the pauses the same, Google all the same cuts through automation and gives out a captcha; - ((

Why is this path bad? What time does the script check become comparable? over time to check manually. But there are pluses: at this time you’re still doing not checking, but something else. It’s “long but without you”, so the option has the right to exist. Unfortunately, sometimes it’s It’s probably failing, probably because I’m pulling the pages with a curl (a function such curl in PHP), or maybe I’ll need more It (there are also such libraries that more or less accurately allow to emulate a browser). In short, there is still something to think about ...

2. Using keys from google . But the problem here is Google officially no longer supports SOAP API, but recommends everyone to switch to the AJAX API. I noticed that the results in this case are very different from the search in manual mode (at times, and even dozens of times !!!). (http: // www. google. com / search? hl = en. safe = off. q = ajax + google + api + estimatedResultCount) you will see that this problem is ubiquitous ... So I highly recommend not using the new API for any kind of SEO research.

As for the old method, using the Google SOAP Search API, then everything is fine: the results diverge “within reasonable limits” (they also diverge on different Google servers, so everything is OK). BUT! Google is no longer gives users these same "old" API-shnye keys. So to write a script that will be public is simply impossible.

The people of the glades made the difference between the old and the new keys and rushed the old ones to trade ... And I was still surprised - who needs them for $ 25 something ??? But on the forums. digitalpoint. com they are just such a price assigned. And with one key only 1000 requests per day can be made. They have always had such a limit. That is, owning one key and a script, you need to spend 10 days on a base of 10,000. Well, or buy 10 keys, but not worth the candle. A free script that requires an additional $ 250 to use for its own keys is not serious ...

However, for those who, like me, have old stock of keys, it makes sense to pay attention to this possibility. But this is not for a public release ... Alas ...

Besides, buying online does not guarantee that at the same time you have not bought a couple of dozen buyers from the same seller ;-)) and then this key becomes generally useless. That is, you need to buy only from verified persons who can guarantee the uniqueness of the sale. And where to take them?

3. Combination of the first with work through a proxy . You can create several simultaneous streams (from different proxies, i.e. from different ip addresses, they will come to Google), each of which will withstand reasonable pauses and then the speed of the entire script will increase as many times as these streams involved.Here, the difficulty lies in finding the WORKERS and at the same time truly ANONYMOUS proxies that would not exactly shine a real IP address before Google. Among the dozens, hundreds, and even thousands of free proxy lists available on the Internet after checking remain working percent fractions. Yesterday, I tested more than 2,000 proxy software - only about a hundred worked, and anonymous 20 pieces were working ... That is, the script must also be equipped with such a component as grabbing the lists of proxies and their regular performance checks. And this is a separate monster is obtained. I know such working systems, I even use a paid script, but I cannot directly integrate it into this script - only through import-export. However, this problem is solved, but again, the use of mass in this case, you can forget. Why am I all this? In addition, to make the script "for yourself" is quite real. Only you need your own proxy checker or purchased proxies. Both cost money.

4. Alternative "for maniacs" . 5-6 hosting is purchased with working open connections for PHP scripts (not every hosting service allows it, i.e. PHP scripts themselves are now universally available, but for these scripts to have a “road” to establish connections with external servers, far from everyone). But it’s quite possible to find such providers. Then a script is written that divides, say, 1000 words into 5 portions of 200 each and spreads them to 5 hosts, where they are quietly checked as usual without any alterations. 200 words usually all passes ... And by nd they throw off the results back to the "central" script that you and provides a unified picture. It’s like a proxy and not a proxy at the same time 😉 But the price is 5 hosting providers - you know ... and monthly ... It's easier to buy a proxy checker and run everything from Denver ...

5. Using data from Wordtracker or another similar service . They usually have Competitors fairly close to Google data for the United States. The popularity of keywords does not interest us, here we have Google trends to help, only data on competitors ... But, again, the financial side of the question ...

Here’s a deal. I tend to last. That is, in the script itself, turn off the competitors check. And chase everything to find the right number of requests. And we’ll send out the “competitors” (we reject everything that we look for too rarely or too often) to check our competitors. The only thing that turns out to be a multi-pass combination, but I would like “all at once”, but for now this option seems to me quite attractive considering the fact that it is still desirable to have a subscription to Wordtracker or any other similar service). But if the wordtracker is not affordable - then the options are arranged in the following way:

  • with old keys (not suitable for everyone);
  • with proxies and multi-threaded query execution (difficult to make, but really );
  • with several hosts (the price can be comparable to the price of Wordtracker).

Related posts:

  • The whole truth about cloaking
  • How to quickly find out about hacking a blog
  • How Profit Hunter check cashed
  • Increase online shop conversion

Do you like articles? Subscribe to the newsletter!


Related Articles