{"id":5031,"date":"2021-02-02T04:30:32","date_gmt":"2021-02-02T04:30:32","guid":{"rendered":"https:\/\/blog.verbat.com\/?p=2556"},"modified":"2024-05-27T06:47:42","modified_gmt":"2024-05-27T06:47:42","slug":"google-smith-algorithm-outranks-bert","status":"publish","type":"post","link":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/","title":{"rendered":"Google\u2019s SMITH Algorithm Outranks BERT."},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Recently Google published a research paper on a new algorithm called SMITH that claims to outdo BERT in understanding long queries and documents. What makes this new model excel better is that it can comprehend passages within documents in the same way that BERT interprets words and sentences, which enables the <strong><a href=\"https:\/\/www.verbat.com\/blog\/google-december-core-update\/\" target=\"_blank\" rel=\"noreferrer noopener\">Google algorithms<\/a><\/strong> to read and understand longer documents.\u00a0<\/p>\n\n\n\n<!--more-->\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Limitations\nof Google\u2019s BERT Algorithm<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">BERT algorithm uses Transformer, an attention mechanism that understands the contextual relation between words in a text. In its simplest form, Transformer includes two separate mechanisms- an encoder that reads the text and a decoder that predict the hidden words from the context. In the past few years, such self-attention based mechanisms like Transformers\u2026 and BERT have achieved tremendous performance in text matching. Because of the quadratic computational complexity of self-attention concerning input text length they are still limited to a few sentences or one paragraph.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Why Is It\nDifficult To Comprehend Long Documents?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The researchers Liu Yang Mingyang, Zhang Cheng Li, Michael Bendersky, Marc Najork in the paper quotes that&nbsp;&nbsp;<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-style-large is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"> <br> &#8220;<em>Compared to semantic matching between short texts, or between short and long texts, semantic matching between long texts is a more challenging task due to a few reasons:\u00a0<\/em><br> \u00a0<br> <em>1)\u00a0\u00a0\u00a0\u00a0\u00a0 When both texts are long, matching them requires a more thorough understanding of semantic relations including matching pattern between text fragments with long distance;<\/em><br> \u00a0<br> <em>2)\u00a0\u00a0\u00a0\u00a0\u00a0 Long documents contain internal structure like sections, passages, and sentences. For human readers, document structure usually plays a key role for content understanding. Similarly, a model also needs to take document structure information into account for better document matching performance;\u00a0<\/em><br> <em>\u00a0<\/em><br> <em>3) The processing of long texts is more likely to trigger practical issues like out of TPU\/GPU memories without careful model design<\/em> &#8220;<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">According to the researchers, the issue of matching long queries to long content has not been adequately explored which they seek to resolve using the SMITH algorithm.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What is the\nSMITH Algorithm?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The current model BERT (Bidirectional Encoder\nRepresentations from Transformers) is designed to understand the full context\nof a word by understanding the context of sentences. Thereby allowing the\nalgorithm to fully comprehend the intent behind each search query. Such algorithms are also trained on data sets to\npredict hidden words from the context within the sentences.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Likewise, the SMITH model is trained to\nunderstand passages within the context of the complete document and to predict\nthe next block of sentences are. Under the opinion of researchers, such\ntraining helps the algorithm understand larger documents a lot better than the\nBERT algorithm.&nbsp;<\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\"> <strong>Also Read: <\/strong><a href=\"https:\/\/www.verbat.com\/blog\/international-seo-checklist\/\" target=\"_blank\" rel=\"noreferrer noopener\"><strong>The Ultimate Checklist for International SEO<\/strong><\/a><strong> <\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">They also claim that the SMITH model outperforms many states of the art models, including BERT, for understanding long-form content. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">They say&nbsp;<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-style-large is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"> &#8220;<em>The experimental results on several benchmark datasets show that our proposed SMITH model outperforms previous state-of-the-art Siamese matching models including HAN, SMASH, and BERT for long-form document matching.<\/em><br> <em>Moreover, our proposed model increases the maximum input text length from 512 to 2048 when compared with BERT-based baseline methods.<\/em>&#8221; <\/p>\n<\/blockquote>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Is Google Using the SMITH Algorithm?<\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Generally, Google does not specify the algorithm it is using. Hence it would be purely speculative to say whether or not it is in use unless Google announces formally that the SMITH algorithm is in use to comprehend passages within web pages.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Building a strong brand recognition needs work on SEO, content marketing, social media marketing among other things. <strong><a href=\"https:\/\/www.verbat.com\/contact\/\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\"Contact us (opens in a new tab)\">Contact us<\/a><\/strong> to know how we can help you build your brand visibility.&nbsp; <\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently Google published a research paper on a new algorithm called SMITH that claims to outdo BERT in understanding long queries and documents. What makes this new model excel better is that it can comprehend passages within documents in the same way that BERT interprets words and sentences, which enables the Google algorithms to read [&hellip;]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-5031","post","type-post","status-publish","format-standard","hentry","category-others"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Google\u2019s SMITH Algorithm Outranks BERT | Verbat Technologies<\/title>\n<meta name=\"description\" content=\"Learn how Google&#039;s SMITH algorithm surpasses BERT in understanding long-form content, enhancing search accuracy and relevance.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Google\u2019s SMITH Algorithm Outranks BERT | Verbat Technologies\" \/>\n<meta property=\"og:description\" content=\"Learn how Google&#039;s SMITH algorithm surpasses BERT in understanding long-form content, enhancing search accuracy and relevance.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/\" \/>\n<meta property=\"og:site_name\" content=\"Software Development Company Dubai UAE - Verbat Technologies\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/verbatltd\" \/>\n<meta property=\"article:published_time\" content=\"2021-02-02T04:30:32+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-27T06:47:42+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@verbatltd\" \/>\n<meta name=\"twitter:site\" content=\"@verbatltd\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/\"},\"author\":{\"name\":\"\",\"@id\":\"\"},\"headline\":\"Google\u2019s SMITH Algorithm Outranks BERT.\",\"datePublished\":\"2021-02-02T04:30:32+00:00\",\"dateModified\":\"2024-05-27T06:47:42+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/\"},\"wordCount\":589,\"publisher\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/#organization\"},\"articleSection\":[\"Others\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/\",\"url\":\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/\",\"name\":\"Google\u2019s SMITH Algorithm Outranks BERT | Verbat Technologies\",\"isPartOf\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/#website\"},\"datePublished\":\"2021-02-02T04:30:32+00:00\",\"dateModified\":\"2024-05-27T06:47:42+00:00\",\"description\":\"Learn how Google's SMITH algorithm surpasses BERT in understanding long-form content, enhancing search accuracy and relevance.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.verbat.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Google\u2019s SMITH Algorithm Outranks BERT.\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.verbat.com\/blog\/#website\",\"url\":\"https:\/\/www.verbat.com\/blog\/\",\"name\":\"Verbat Technologies\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.verbat.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.verbat.com\/blog\/#organization\",\"name\":\"Verbat Technologies\",\"url\":\"https:\/\/www.verbat.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.verbat.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.verbat.com\/blog\/wp-content\/uploads\/2024\/04\/verbatltd_logo.jpg\",\"contentUrl\":\"https:\/\/www.verbat.com\/blog\/wp-content\/uploads\/2024\/04\/verbatltd_logo.jpg\",\"width\":200,\"height\":200,\"caption\":\"Verbat Technologies\"},\"image\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/verbatltd\",\"https:\/\/x.com\/verbatltd\",\"https:\/\/www.linkedin.com\/company\/verbatltd\"]},{\"@type\":\"Person\",\"@id\":\"\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Google\u2019s SMITH Algorithm Outranks BERT | Verbat Technologies","description":"Learn how Google's SMITH algorithm surpasses BERT in understanding long-form content, enhancing search accuracy and relevance.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/","og_locale":"en_US","og_type":"article","og_title":"Google\u2019s SMITH Algorithm Outranks BERT | Verbat Technologies","og_description":"Learn how Google's SMITH algorithm surpasses BERT in understanding long-form content, enhancing search accuracy and relevance.","og_url":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/","og_site_name":"Software Development Company Dubai UAE - Verbat Technologies","article_publisher":"https:\/\/www.facebook.com\/verbatltd","article_published_time":"2021-02-02T04:30:32+00:00","article_modified_time":"2024-05-27T06:47:42+00:00","twitter_card":"summary_large_image","twitter_creator":"@verbatltd","twitter_site":"@verbatltd","twitter_misc":{"Written by":"","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/#article","isPartOf":{"@id":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/"},"author":{"name":"","@id":""},"headline":"Google\u2019s SMITH Algorithm Outranks BERT.","datePublished":"2021-02-02T04:30:32+00:00","dateModified":"2024-05-27T06:47:42+00:00","mainEntityOfPage":{"@id":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/"},"wordCount":589,"publisher":{"@id":"https:\/\/www.verbat.com\/blog\/#organization"},"articleSection":["Others"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/","url":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/","name":"Google\u2019s SMITH Algorithm Outranks BERT | Verbat Technologies","isPartOf":{"@id":"https:\/\/www.verbat.com\/blog\/#website"},"datePublished":"2021-02-02T04:30:32+00:00","dateModified":"2024-05-27T06:47:42+00:00","description":"Learn how Google's SMITH algorithm surpasses BERT in understanding long-form content, enhancing search accuracy and relevance.","breadcrumb":{"@id":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.verbat.com\/blog\/google-smith-algorithm-outranks-bert\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.verbat.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Google\u2019s SMITH Algorithm Outranks BERT."}]},{"@type":"WebSite","@id":"https:\/\/www.verbat.com\/blog\/#website","url":"https:\/\/www.verbat.com\/blog\/","name":"Verbat Technologies","description":"","publisher":{"@id":"https:\/\/www.verbat.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.verbat.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.verbat.com\/blog\/#organization","name":"Verbat Technologies","url":"https:\/\/www.verbat.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.verbat.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.verbat.com\/blog\/wp-content\/uploads\/2024\/04\/verbatltd_logo.jpg","contentUrl":"https:\/\/www.verbat.com\/blog\/wp-content\/uploads\/2024\/04\/verbatltd_logo.jpg","width":200,"height":200,"caption":"Verbat Technologies"},"image":{"@id":"https:\/\/www.verbat.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/verbatltd","https:\/\/x.com\/verbatltd","https:\/\/www.linkedin.com\/company\/verbatltd"]},{"@type":"Person","@id":""}]}},"_links":{"self":[{"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/posts\/5031","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/comments?post=5031"}],"version-history":[{"count":1,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/posts\/5031\/revisions"}],"predecessor-version":[{"id":6190,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/posts\/5031\/revisions\/6190"}],"wp:attachment":[{"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/media?parent=5031"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/categories?post=5031"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/tags?post=5031"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}