Ya­hoo open sources its Big Data sup­ported search tech

OpenSource For You - - Fossbytes -

Months af­ter be­ing ac­quired by Ver­i­zon, Ya­hoo has de­cided to open source its Big Data pro­cess­ing and serv­ing en­gine called Vespa. The tech­nol­ogy was used ex­clu­sively for search queries on key Ya­hoo prod­ucts, in­clud­ing Ya­hoo

News and Flickr, among oth­ers.

Ver­i­zon-owned Oath, which serves as the par­ent com­pany of Ya­hoo, claims that Vespa pro­cesses and serves con­tent and ads al­most 90,000 times ev­ery sec­ond, with la­ten­cies in the tens of mil­lisec­onds. It is even sup­posed to han­dle key­word and im­age searches on a huge scale, with a few hun­dred queries per sec­ond on tens of bil­lions of im­ages.

De­vel­oper teams can lever­age Vespa to pick con­tent through SQL­like queries and text search, or­gan­ise matches and gen­er­ate data-driven pages, as well as write data in real­time. The tech­nol­ogy is ca­pa­ble of dis­tribut­ing data and com­pu­ta­tion over sev­eral ma­chines at once.

“By re­leas­ing Vespa, we are mak­ing it easy for any­one to build ap­pli­ca­tions that can com­pute re­sponses to user re­quests, over large data sets, in real-time and at In­ter­net scale — ca­pa­bil­i­ties that up un­til now have been within the reach of only a few large com­pa­nies,” Vespa’s dis­tin­guished ar­chi­tect Jon Brat­seth wrote in a blog post. Vespa can be run on-premise or in the cloud and comes both in Docker im­ages and rpm pack­ages. Its code is avail­able in a GitHub repos­i­tory along with de­tailed doc­u­men­ta­tion.

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.