Are crawlers and bots con­sum­ing too many re­sources?

OpenSource For You - - HOW TO DEVELOPERS -

An­other rather painful be­hindJtheJscenes cause for la­tency is the work of bots, crawlers, spi­ders and other ‘dig­i­tal crea­tures’. jore of­ten, crawlers and bots can creep into your site un­no­ticed, and though not all of them do so with a ma­li­cious in­tent, they can con­sume valu­able re­sources and slow down the re­sponse times for users. fn some of the cor­po­rate sites that we built, we found the doogle bot had in­dexed the site even be­fore

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.