Recurrent Neural Networks by ThatWare AI

Recurrent Neural Networks (RNNs):
Models that process sequential data –
Next Gen SEO with Hyper-Intelligence
This project aims to detect content similarity across multiple web pages using
machine learning techniques. The approach uses a Recurrent Neural Network (RNN)
to analyze web page data, including titles, meta descriptions, and content. By
processing this information, the model generates embeddings that represent the
semantic meaning of the content. Cosine similarity is then calculated to determine
how similar different web pages are.
Recurrent Neural Network (RNN)
In this project, a Recurrent Neural Network (RNN) is used to understand and represent
the meaning of web page content in a structured and intelligent way. RNN is a type of
machine learning model that is designed to handle sequential data — in other words,
data that has a natural order, like text. Since sentences and paragraphs follow a
specific sequence of words, RNN is particularly well-suited for processing language.
What the RNN Does
The RNN works like a smart reader. It goes through the text content from each web
page, word by word, and tries to understand the patterns and context of those words.

This helps in building a compressed form of understanding called an embedding.
Embeddings are numerical representations that capture the overall meaning of the
content.
These embeddings are not just based on word frequency or keyword matching; they
capture deeper relationships such as tone, structure, and overall semantic meaning.
That makes them far more powerful for comparing content across different websites.
Why the RNN Is Needed
Web pages often have varying content styles, wording, and layouts. Traditional
methods might only match exact words, missing out on the underlying meaning. The
RNN, on the other hand, is capable of learning the structure and flow of language.
This allows it to generate embeddings that can reflect how similar or different two
web pages really are — not just based on surface-level text, but based on meaning.
In this project, the RNN is used as part of an autoencoder — a structure that tries to
reconstruct the input content from its learned embedding. This training process helps
the model understand content more efficiently, even without needing labeled
examples.
How the RNN Fits into the Workflow
● Text from the web pages (title, meta, and body content) is first cleaned and
converted into tokens (numerical values).
● These tokens are then sent to the RNN-based autoencoder.
● The encoder part of the RNN learns how to compress the content into a
meaningful vector (embedding).
● This embedding is used as the basis for comparing one piece of content with
another using cosine similarity.
● The closer the embeddings are, the more similar the content is considered to
be.
Benefits of Using RNN
● Understands language flow: Unlike simpler models, RNN captures the
sequence of words, improving the understanding of context.
● Captures meaning: The model doesn’t just compare words, but compares the
meaning behind the words.
● Learns from unlabeled data: There is no need for manual tagging or
supervision. The RNN trains itself to reconstruct input and thus learns valuable
patterns automatically.
What Is an Embedding

In the context of this project, an embedding is a special type of data representation
that captures the overall meaning of a web page’s content in the form of numbers.
Instead of comparing raw text directly, the content is transformed into a structured
format that a machine learning model can understand and work with.
Why Embeddings Are Needed
Raw text from web pages is complex and difficult to compare in its original form.
Every page has different writing styles, word choices, and formatting. Embeddings
make it possible to reduce this complexity by converting the text into a numerical
vector — essentially, a set of values that hold the essence of the content.
These vectors allow for easier and more accurate comparisons between different web
pages. Two web pages with similar topics and themes will have embeddings that are
closer together, while very different pages will have embeddings that are farther apart.
How Embeddings Are Created
● First, the content (including title, meta description, and paragraphs) is cleaned
and converted into tokens, which are numbers representing words.
● These tokens are passed through a trained Recurrent Neural Network (RNN).
● The RNN analyzes the sequence of words and learns patterns from them.
● From this, the encoder part of the model creates a compact vector — the
embedding — that summarizes the meaning of the entire input content.
● This vector can then be used to represent the original web page in a
meaningful and comparable way.
Key Qualities of a Good Embedding
● Compact and informative: Instead of storing every word, embeddings store the
essence of meaning in a small set of numbers.
● Context-aware: Embeddings consider the order and context of words, not just
their presence.
● Comparable: Similar embeddings mean similar content — making them ideal
for similarity scoring.
● Normalized: Embeddings are adjusted to have a consistent scale, which helps
in measuring accurate similarity.
Role of Embeddings in the Project
Embeddings are at the core of content similarity detection. Once the embeddings for
two different pages are available, their similarity can be calculated using a
mathematical formula called cosine similarity. The closer the two embeddings are, the
more semantically similar the two web pages are considered to be — even if the
words used are not exactly the same.

Cosine Similarity
Cosine similarity is a mathematical method used in this project to measure how
similar two web pages are based on their content embeddings. After a web page is
converted into an embedding — a vector of numbers — cosine similarity provides a
score that tells how closely two of these vectors point in the same direction.
Why Cosine Similarity Is Used
When comparing two web pages, it’s important to measure how similar their
meanings are, not just whether they share some words. Cosine similarity helps
identify whether two embeddings — the compact representations of those pages —
are aligned in terms of their overall context and themes.
This method is ideal for textual comparisons because it focuses on the orientation of
the content vector, rather than just the raw distance or count of words. It helps
capture the semantic similarity, even when two pages use different vocabulary.
How Cosine Similarity Works
● ach web page’s content is transformed into a numerical vector (embedding).
● These vectors can be visualized as arrows pointing in multi-dimensional
space.
● Cosine similarity calculates the angle between these two arrows:
○ If the vectors point in the same direction (angle close to 0 degrees), the
similarity score is close to 1.0 — meaning very similar content.
○ If they point in completely opposite directions (angle close to 180
degrees), the score is close to -1.0 — meaning very different content.
○ If the angle is around 90 degrees, the score is close to 0.0 — meaning
unrelated content.
In this project, all similarity scores fall between 0.0 (no similarity) and 1.0 (high
similarity).
Masking Padding Tokens
To ensure accurate results, cosine similarity calculations ignore any padding values
that were added during preprocessing. These padding tokens are zeros inserted to
make all sequences the same length, but they don’t carry any real meaning, so they
are excluded from the similarity scoring.
Role in the Project
After the RNN model generates embeddings for all the web pages, cosine similarity
compares each pair of embeddings. This results in a numerical score for every

comparison, showing how similar two pages are in terms of their content. These
scores are then presented as output, helping to understand which pages are aligned
in terms of SEO themes, service offerings, or informational depth.
What is the purpose of this project?
The project is built to analyze and compare the actual content of webpages from
different SEO companies. This helps uncover how similar or different their pages are
in terms of topics, depth, and overall focus.
This comparison enables useful actions such as:
● Identifying content overlaps between competitors
● Spotting unique content that one company has but another doesn’t
● Discovering content gaps where a company may improve or expand
● Supporting better content planning and SEO strategy
Instead of relying on surface-level checks like keyword density or titles, the system
works with the actual full content to understand how one page compares to another.
How does the comparison actually work?
The system follows a structured process:
● Scrapes the webpage and pulls content such as title, description, and main
body.
● Cleans the text to remove noise (like symbols or extra spaces).
● Transforms the cleaned content into a numerical summary (embedding) using
a machine learning model.
● Compares these summaries across webpages to measure how similar or
different they are.
The model used is designed to understand how the words are structured over time —
learning from the overall flow and context of each page, not just isolated keywords.
Library Overview
requests
The requests library is used to perform HTTP requests. In this project, it is
responsible for sending requests to given blog URLs and retrieving the HTML content

of those pages. This step forms the foundation of the entire process, as it allows
automated access to the page data that will later be processed and analyzed.
Think of it as the tool that collects raw material from each URL.
BeautifulSoup (from bs4)
BeautifulSoup is used to parse and extract content from the HTML structure of a
webpage. Once a page is fetched, this tool identifies and isolates relevant content
such as titles and body text, while ignoring irrelevant sections like navigation bars,
advertisements, or page scripts. This ensures the analysis focuses only on the actual
article content, which is essential for accurate content similarity detection.
re (Regular Expressions)
The re module is used for advanced string processing and content cleanup. It helps
remove unwanted characters, multiple spaces, special symbols, and other noise
elements from the extracted text. Clean and consistent content is important for
generating reliable results during the model’s similarity comparison process.
numpy
numpy supports numerical operations and is commonly used in machine learning
workflows. It is utilized here for managing vector data (numerical representation of
text), mathematical calculations, and similarity score computation. It plays an
essential role in maintaining efficiency and accuracy during content comparison
tasks.
pandas
pandas is a data management library. It is used to organize, format, and store outputs
such as extracted titles, body texts, and similarity results. Data is structured in tabular
formats, which can then be exported to CSV or Excel files for easy sharing, review,
and further analysis.
tensorflow
TensorFlow is the primary machine learning framework used in the project. It provides
the environment and tools necessary to define, train, and run the deep learning
model. Its integration ensures the model is scalable, efficient, and reliable for
processing complex content similarity tasks.

Function: scrape_page_data(url)
This function extracts key information (title, meta description, and the first 10
paragraphs of content) from a webpage to aid in content similarity detection. Here’s a
breakdown of how it works:
Fetch Webpage Content: response = requests.get(url, timeout=10) Fetches the
webpage content by sending an HTTP GET request to the provided URL.
Parse HTML with BeautifulSoup: soup = BeautifulSoup(response.text, ‘html.parser’)
Uses BeautifulSoup to parse the raw HTML, making it easier to navigate and extract
information.
Extract the Title: title = soup.title.get_text() if soup.title else None Extracts the
webpage’s title from the <title> tag, which indicates the page’s focus.
Extract the Meta Description: meta_description = soup.find(‘meta’, attrs={‘name’:
‘description’})
meta_description = meta_description[‘content’] if meta_description else None
Extracts the meta description (a brief summary of the page) from the <meta> tag.
Extract the Content (First 10 Paragraphs): content = ” for paragraph in
soup.find_all(‘p’, limit=10): content += paragraph.get_text() + ‘ ‘ Gathers the first 10
paragraphs of the page’s content to provide a snapshot of the text.
Return Combined Data: return f”{title}. {meta_description}. {content}” Combines the
title, meta description, and content into a single string to return.

Error Handling: except Exception as e: return “” Returns an empty string if there’s an
error fetching or parsing the webpage.
Purpose in the Project:
This function collects essential elements (title, meta description, and content) from a
webpage, providing the necessary data for content comparison in the similarity
detection model. It ensures that the collected data is clean, structured, and consistent
for further analysis.
Extract Data from URL
The function scrape_page_data(url1) fetches the page content from the provided URL.
It will extract the title, meta description, and the first 10 paragraphs of the content.
Extract Data from URL2
Similarly, scrape_page_data(url2) fetches content from the second URL.
Function clean_text
The clean_text function is responsible for preparing the scraped text for further
processing by cleaning and normalizing the content.
The clean_text function performs three primary tasks on the input text:
Converts Text to Lowercase:

text = text.lower() This is done to standardize the text, as text analysis should be
case-insensitive. By converting everything to lowercase, it ensures that words like
“SEO” and “seo” are treated as the same word.
Removes Special Characters:
text = re.sub(r'[^a-z0-9s]’, ”, text) The regular expression r'[^a-z0-9s]’ matches
anything that is not a lowercase letter (a-z), a digit (0-9), or a whitespace (s). It
removes punctuation marks, symbols, and any other non-alphanumeric characters.
Strips Extra Whitespace:
text = re.sub(r’s+’, ‘ ‘, text).strip() This step ensures that any extra spaces between
words or at the beginning and end of the text are removed. It consolidates multiple
spaces into a single space and removes leading or trailing spaces.
The function clean_text(text1) cleans the page content using provided content.
It will lowercase the charecters and remove unwanted space and texts.
The Tokenizer class from Keras is used to vectorize a corpus of text, turning each
word into a unique token (numerical representation). This is an essential step in
preparing text data for use in neural networks. Here’s how the tokenizer works:
Fitting the Tokenizer: Before using the Tokenizer, it needs to “learn” the vocabulary
from the text data. This is done by calling the fit_on_texts() method, which processes
the text and assigns a unique integer to each word based on its frequency of
occurrence.
Converting Text to Tokens: Once the tokenizer is fitted, it can then be used to convert
text into a sequence of integers. Each word in the text gets replaced by its
corresponding integer value, which is useful for input into machine learning models.

The code involves two important steps: padding the sequences and preparing the
decoder targets.
● Padding the Sequences: The pad_sequences() function is used to ensure that
all input sequences (texts in this case) have the same length. In natural
language processing (NLP) tasks, it’s crucial for all input data to have a
consistent length so that it can be fed into models efficiently. Here’s a
breakdown:
○ tokenizer.texts_to_sequences([cleaned_text1, cleaned_text2]) converts
the cleaned texts into sequences of integers, where each word in the
texts is replaced by its corresponding integer ID from the tokenizer’s
vocabulary.

○ maxlen=max_len specifies the maximum length of the sequences. Any
sequences longer than this length are truncated, while shorter ones are
padded (with zeros) to meet this length.
○ padding=’post’ indicates that padding will be added at the end of the
sequence. This is commonly done in NLP to maintain the semantic
content at the beginning of the sequence.
○ truncating=’post’ means if the sequence exceeds the specified length
(max_len), it will truncate the sequence from the end (instead of from
the start, which could potentially remove important context).
● Preparing Decoder Targets:
decoder_targets = np.expand_dims(padded_texts, -1) reshapes the padded text
sequence into the shape required for the decoder in the sequence-to-sequence
model. Here, the expand_dims() function adds an extra dimension at the end, which is
necessary for the decoder’s input format.
The output of padded_texts is a 2D array (for the two input texts) where each row
represents a sequence of tokenized words, padded to max_len length.
Function build_rnn_model
This function is responsible for building an RNN-based autoencoder model, which
plays a crucial role in learning meaningful representations of webpage content in an
unsupervised manner. The autoencoder learns to compress (encode) the text data
into a compact form using the encoder, and then reconstruct (decode) it back into the
original format. Although the final goal is not to reconstruct the content, this training
helps the encoder learn how to capture the most important features of the content.
The encoder part of this model is later used to convert any given webpage text into a
numerical embedding—a dense, fixed-size vector—that can be used to calculate
similarity between different webpages. This process allows the system to go beyond
simple keyword matching and understand the underlying semantics of the content.
Explanation

encoder_inputs = Input(shape=(max_len,), name=”encoder_input”) Creates the input
layer for the encoder. It expects a sequence of integers (token IDs) of length max_len.
x = Embedding(input_dim=vocab_size, output_dim=embedding_dim,
mask_zero=True)(encoder_inputs) Adds an Embedding layer, which turns each
integer token into a dense vector of size embedding_dim. The mask_zero=True tells
the model to ignore padding tokens (zeros).
x = SimpleRNN(rnn_units, return_sequences=False, name=”encoder_rnn”)(x) This
SimpleRNN layer processes the embedded sequence and summarizes it into a single
vector (since return_sequences=False), effectively compressing the entire sequence.
x = Dropout(0.3)(x) Applies dropout to prevent overfitting by randomly turning off 30%
of the neurons during training.
repeat = RepeatVector(max_len)(x) Repeats the encoded vector for max_len times to
prepare it as input for the decoder RNN. This mirrors the input shape of the original
sequence.
decoder = SimpleRNN(rnn_units, return_sequences=True,
name=”decoder_rnn”)(repeat) This decoder RNN attempts to reconstruct the original
sequence from the repeated encoded vector.
decoder_dense = TimeDistributed(Dense(vocab_size, activation=’softmax’),
name=”decoder_output”)(decoder) Applies a dense layer (with softmax activation) at
each timestep of the decoder output. It converts RNN outputs into a sequence of
predicted word IDs, matching the original input format.
autoencoder = Model(encoder_inputs, decoder_dense)
autoencoder.compile(optimizer=Adam(), loss=’sparse_categorical_crossentropy’)
Combines encoder and decoder into one model and compiles it using a loss suitable
for sequence prediction.
encoder_model = Model(encoder_inputs, x) Separately creates the encoder model
(used later for similarity analysis), which maps input sequences to the learned dense
representations.
return autoencoder, encoder_model Returns both the complete autoencoder model
for training and the encoder-only model for later use.

This code block initializes the model training phase. Before comparing any content,
the system must first learn how to understand it — and this is where model training
plays a crucial role. The goal is to train an RNN-based autoencoder using cleaned and
tokenized versions of two web pages. The encoder learns how to compress content
into a meaningful representation, and the decoder attempts to reconstruct the original
sequence from that compressed form.
This training process is essential to ensure the encoder captures the structure and
semantics of SEO-related web content. The final encoder becomes the main
component used for content similarity detection later in the project.
keyboard_arrow_down
Explanation:
vocab_size = len(tokenizer.word_index) + 1 Calculates the total number of unique
tokens (words) found in the content, based on the tokenizer used earlier. Adding 1
accounts for padding or out-of-vocabulary tokens.
Defines the model’s hyperparameters:
embedding_dim: Size of each word vector.
rnn_units: Number of units in the RNN layer, determining how much capacity it has to
capture patterns.
epochs: Number of full training cycles over the input data.
autoencoder, encoder_model = build_rnn_model(vocab_size, embedding_dim,
rnn_units, max_len)
Uses the earlier-defined function build_rnn_model to build two models:

autoencoder: Full model with encoder + decoder, used only for training.
encoder_model: Encoder only, used after training to extract embeddings for similarity
comparison.
autoencoder.fit(padded_texts, decoder_targets, epochs=epochs, batch_size=2) Starts
training the autoencoder model:
Input: padded_texts, the cleaned and padded sequence data from both pages.
Target: decoder_targets, the expected output sequence used to guide the training.
This training helps the encoder model learn how to compress meaningful parts of the
webpage content into dense vectors.
Function preprocess_text
This function prepares webpage content for use in the RNN-based encoder by
transforming raw cleaned text into a standardized numerical format. Specifically, it
converts text into sequences of numbers (tokens), and then ensures that all
sequences have the same fixed length using padding. This standardization is
essential for deep learning models, which require input data of consistent shape.
By ensuring uniform length and format, this function allows for accurate and efficient
content embedding, which is crucial for comparing different webpages or analyzing
semantic similarity.
Explanation:
tokens = tokenizer.texts_to_sequences([text]) Converts the input text into a sequence
of integers using the tokenizer.
Each word is replaced with a corresponding index from the tokenizer’s vocabulary.
The input is wrapped in a list ([text]) to maintain consistency in shape for batch
processing.
padded_tokens = pad_sequences(tokens, padding=’post’, maxlen=max_len,
truncating=’post’)
Ensures that all token sequences have the same length by:

Padding shorter sequences with zeros at the end (post padding).
Truncating longer sequences from the end (post truncating).
max_len defines the standard length — ensuring compatibility with the encoder
model’s expected input.
return padded_tokens Returns the processed and standardized sequence, ready for
input into the encoder.
Function get_embedding
This function transforms a block of cleaned and tokenized text into a numerical vector
representation using the trained encoder model. This vector — known as an
embedding — captures the key characteristics and contextual meaning of the content
in a form that machine learning models can understand. Embeddings are essential for
comparing content similarity, identifying clusters, or feeding content into other
downstream SEO analytics tasks.
Once generated, the embedding is normalized, meaning its length (magnitude) is
adjusted to 1. This normalization ensures that similarity comparisons between two
embeddings (such as using cosine similarity) are consistent and not influenced by
differences in scale.
Explanation:
preprocessed_text = preprocess_text(text, tokenizer, max_len) Uses the
preprocess_text function which converts raw input text into a sequence of numbers
based on the tokenizer.
Also pads or truncates the sequence to a fixed length (max_len) so the model can
process it properly.
embedding = model.predict(preprocessed_text, verbose=0) Sends the preprocessed
sequence to the trained encoder model.

The encoder returns the embedding — a compressed numerical summary of the
content.
norm = np.linalg.norm(embedding) Calculates the magnitude (Euclidean length) of the
embedding vector.
This is used to normalize the embedding.
if norm == 0: return embedding Handles edge cases: If the magnitude is zero (which
might happen with empty or malformed input), it returns the unmodified embedding to
avoid division by zero.
normalized_embedding = embedding / norm Normalizes the embedding vector to have
a unit length (length = 1).
This allows for consistent comparisons between different content vectors.
return normalized_embedding Returns the final normalized embedding vector, ready
to be used in similarity comparison or clustering tasks.

This step generates a numerical embedding for the cleaned webpage content using
the trained RNN-based encoder model. The embedding captures the semantic
meaning of the text rather than just a sequence of words.
● Calls the get_embedding function to convert cleaned texts into a meaningful
vector representation.
● Uses the trained encoder model to generate a condensed representation of the
content.
● The output is a normalized numerical array that represents the text in a
multi-dimensional space.
Understanding the Output
The output is a 64-dimensional vector:
Each value represents a feature of the content, and together they define its position in
the embedding space.

● Positive & Negative Values
○ The presence of both positive and negative values indicates that the
model has learned distinct characteristics about the content.
○ Some features contribute positively to certain similarities, while others
contribute negatively.
● Importance of Normalization
○ Since the embedding is normalized, the magnitude of the vector does
not affect similarity calculations.
○ This ensures that similarity is based on direction rather than absolute
size.
Function cosine_similarity
The cosine_similarity function is designed to take two such embeddings and return a
score that reflects their similarity. This score ranges from -1 to 1, where:
1 means the embeddings are identical in direction (high similarity),
0 means they are orthogonal or unrelated,
-1 means they are opposite in direction (very dissimilar).
Explanation
mask1 = embedding1 != padding_value mask2 = embedding2 != padding_value Before
comparing, any padding values (commonly 0s added during preprocessing) are

ignored. These don’t carry semantic information and would distort the similarity score
if left in.
embedding1 = embedding1[mask1] embedding2 = embedding2[mask2] The
embeddings are filtered to retain only valid values, ensuring the calculation is based
on real content.
embedding1.reshape(1, -1) Reshapes the 1D embedding array into a 2D row vector
format, which is required for matrix operations.
np.dot(embedding1, embedding2.T) / (norm1 * norm2) This is the actual cosine
similarity formula, comparing the angle between the two vectors. The dot product
measures their alignment, while the denominator ensures they are compared
proportionally.
return score[0][0] The final similarity score is returned in scalar form for easy
interpretation.
This function is central to the content similarity analysis part of the project. It
translates the abstract embeddings into a measurable signal that determines how
much two URLs’ content align in meaning. The result helps in identifying duplicates,
similar intent pages, or opportunities for content clustering.
Output Analysis:
This similarity score represents the degree of content-level similarity between two
webpages based on their underlying semantic structures. The score is calculated
using cosine similarity, which ranges between -1 and 1:
1.0 indicates perfect semantic alignment (identical or extremely similar content),
0.0 indicates no similarity (unrelated content),
-1.0 would indicate opposite semantics, though negative values are rare in
well-structured embeddings due to normalization.
What a Score of ~0.78 Implies: A similarity score of 0.778 suggests that the two
webpages have a high degree of semantic similarity. While they are not identical, they
likely share similar themes, concepts, or topics. Some key insights this score may
indicate:
The two pages may be targeting the same user intent or covering closely related
subject matter.

There may be redundancy or overlap between the two pages in terms of informational
value.
The content might be suitable for consolidation or for internal linking, depending on
the SEO strategy.
SEO Context: In SEO, identifying pages with high content similarity is important for
several reasons:
It helps reduce keyword cannibalization, where multiple pages compete for the same
search terms.
It enables content pruning or merging, enhancing content depth and authority.
It improves crawl efficiency and user experience by avoiding unnecessary
duplication.
Conclusion: A similarity score of this magnitude is a strong signal that both pages
serve closely aligned purposes. Depending on their positioning and traffic
performance, a content strategy decision—such as merging, differentiating, or
internally linking—can be made to optimize SEO outcomes.

Content Similarity Analysis Across Multiple Webpages
Overview
This section evaluates the semantic similarity between a group of webpages by
comparing their underlying content embeddings. These embeddings capture the
meaning and thematic structure of each page. Cosine similarity is used as the core
metric to quantify how closely related the content of each page is to the others.
Similarity Score Patterns and Interpretation
High Similarity (Above 0.70)

Scores in this range indicate strong thematic or topical alignment between pages.
Examples in this group suggest that the respective webpages likely:
Cover overlapping subject areas.
Target the same or similar user intent.
May be part of a focused content series or thematic group.
SEO Implication:
These pages may benefit from internal linking or even consolidation, depending on
performance. However, care should be taken to maintain keyword coverage diversity
and avoid cannibalization.
Examples:
A similarity score of 0.74 between two pages implies highly similar updates or
algorithm-related discussions.
A 1.00 score confirms identical content, often due to comparing the same page twice.
Moderate Similarity (0.50 – 0.69)
Pages in this group share notable thematic elements but are not redundant. They
may:
Discuss different facets of the same broader topic.
Target adjacent keyword clusters.
Use different formats (e.g., guides vs. analyses) for related themes.
SEO Implication:
This range is ideal for internal linking strategies and structuring pillar-cluster content
architecture. It reflects an organized topical strategy where related content exists
without duplication.
Examples:
Pages scoring 0.55 to 0.64 likely address similar challenges or tools in SEO, each with
distinct focus areas.
Low Similarity (0.20 – 0.49)
This range indicates that the pages are loosely related. Some shared concepts may be
present, but the primary focus and value proposition differ.
SEO Implication:

These pages should not be merged or altered for similarity reasons, but they may
occasionally be linked when offering complementary perspectives or tools.
Examples:
Scores like 0.28 or 0.20 show partial thematic overlap—such as mentioning a shared
algorithm or framework—but with overall different goals.
Very Low Similarity (Below 0.20)
Pages in this group are semantically distinct. They cover unrelated topics, target
different search intents, or serve entirely different user journeys.
SEO Implication:
These pages should remain independent. Linking is only relevant if there’s a strategic
need (e.g., site navigation or auxiliary resources), not based on content similarity.
Examples:
A score of 0.07 or 0.16 typically implies very minimal content overlap.
Strategic Takeaways for SEO Optimization
Identify Redundancies: Pages with high similarity can be reviewed for potential
consolidation or differentiation.
Support Internal Linking: Moderate similarity scores highlight opportunities to
improve internal linking across related topics.
Maintain Diversity: Low similarity is not negative—it ensures coverage across a broad
range of topics and avoids overlap.
Content Structuring: Clustering pages based on similarity can inform topical silos and
pillar content strategies.

Cross-Domain Content Similarity Analysis
Objective:
To assess semantic alignment between content published across different websites,
each with distinct pages. This comparison provides insights into how closely the
themes, topics, or messaging of different websites align with each other — useful for
competitive analysis, partnership evaluation, or strategic content benchmarking.
Key Observations by Similarity Score Ranges
Very High Similarity (0.85 – 1.00)
This range indicates almost identical or extremely aligned content.

It may suggest shared templates, overlapping descriptions, or extremely similar
messaging.
Some contact or homepage pages across different websites fall in this range, often
due to standard formats or common service language.
SEO Insight:
This could imply redundancy across sites (if owned by the same entity) or close
competitive alignment. Consider content uniqueness strategies if duplication is
unintentional.
High Similarity (0.70 – 0.84)
These scores suggest notable thematic overlap across companies. This could involve
shared service categories, updates, or SEO topics being covered in similar depth and
structure.
For instance, comparison between an update-related post on one website and a
service description on another may yield high similarity if both touch on algorithm
strategies or site auditing.
SEO Insight:
This is ideal for content gap identification or collaborative benchmarking. Such
similarity might also uncover which websites are competing on similar keyword sets.
Moderate Similarity (0.50 – 0.69) Pages in this range share broad themes but diverge
in specifics.
For example, educational SEO content might share a framework or terminology with a
service page, but differ in intent (informative vs. transactional).
SEO Insight:
This signals adjacent topical coverage — perfect for discovering opportunities to
either expand content depth or enhance internal linking strategies between related
but unique content.
Low Similarity (Below 0.50)
Indicates distinct themes or intent. For instance, a detailed technical guide compared
with a general contact page or broad business overview.
Still, low scores don’t mean poor quality — just different subject matter.
SEO Insight:

Content in this zone typically serves different purposes or audiences. It may highlight
an opportunity to expand into under-covered thematic areas when comparing across
competitors.
Cross-Domain Strategy Insights
Competitive Alignment Pages between companies scoring in the 0.80+ range may
indicate similar messaging or even content duplication, either unintentionally or due
to industry standardization.
Content Differentiation Moderate similarity scores reveal areas where websites
approach similar topics differently. These are opportunities to learn from competitors
or differentiate more clearly.
Structural Templates Detected Pages like “Contact Us” and general service overviews
often show high cross-site similarity, suggesting standard layout use or shared
service vocabularies.
Strategic Benchmarking This analysis can inform whether a company’s content is
overlapping, missing, or uniquely positioned relative to others in the space.
What does this project actually measure?
This project evaluates how similar two pieces of web content are based on their
semantic meaning — not just matching keywords or surface-level similarities. By
converting page content into vector-based embeddings and comparing them using
cosine similarity, we measure how closely aligned the underlying topics or messaging
are.
Why should content similarity matter to our SEO
strategy?
Understanding content similarity is crucial for:
● Avoiding duplication across internal pages (which can confuse search
engines)
● Benchmarking content against competitors (to see how similar or different
your messaging is)
● Identifying content gaps where your site could expand or strengthen coverage
● Improving topical authority by ensuring content clusters cover a diverse but
related set of themes
What does a high similarity score mean?

A high similarity score (e.g., 0.85 or above) means two pages are very similar in
meaning. This can happen if:
● The same topic is explained using similar language
● Pages have overlapping content structures
● The messaging or purpose (e.g., product descriptions, services) are nearly
identical
In SEO terms, this might help detect:
● Duplicate or near-duplicate content (internally or externally)
● Competitive content overlap
● Pages that may benefit from consolidation or differentiation
What does a low similarity score indicate?
A low similarity score (e.g., 0.2 or below) means the two pages are semantically
unrelated. This typically happens when:
● One page is educational and another is transactional
● Topics are entirely different (e.g., SEO vs. web development)
● The writing tone or structure is not aligned
This is often expected and useful to ensure content serves diverse intents across
your site.
Is this comparison based on keyword matching or
something more?
This goes far beyond keyword matching. The comparison is done using deep
learning-based embeddings, which capture the actual meaning of the text — not just
individual words. Even if two pages use different vocabulary, they will be marked as
similar if they convey the same ideas.
Can this analysis detect duplicate content issues within
your own site?
Yes. By running this analysis internally (within your own set of URLs), it becomes
possible to:
● Detect content that’s too similar
● Flag areas for rewriting or merging
● Improve crawl efficiency and on-site structure

How is this useful when comparing your content with
competitors’?
By comparing your content with external competitor sites, this analysis can help you:
● Spot overlapping content (where you compete head-to-head)
● Discover new angles or topics competitors cover that you don’t
● Differentiate your voice and approach in highly competitive spaces
● Benchmark your content strategy across industries or markets
What should you do?
After reviewing the similarity scores, consider these next steps:
● Audit for duplication: Identify pages with very high similarity (especially if
within your own domain) and consolidate or rewrite them.
● Fill content gaps: If your pages are very different from competitors’, assess
whether you’re missing important topics.
● Differentiate key content: Ensure your top-converting or product-focused
pages are unique in tone and content.
● Improve semantic coverage: Where similarity is low, think about building
supporting content to bridge the gap in topic coverage.
● Prioritize updates: Use the insights to prioritize which pages need work —
based on overlap or competitive similarity.
Final Thoughts
This project demonstrates a practical and scalable approach to analyzing website
content through semantic similarity techniques. By leveraging contextual
embeddings powered by a trained Recurrent Neural Network (RNN) model, it becomes
possible to assess how closely related different webpages are — not just through
surface keywords, but through deeper patterns in language and structure.
The RNN architecture, designed to understand sequential data, enables the model to
capture context and flow within webpage content, producing more accurate and
meaningful content representations. These embeddings are then compared using
cosine similarity to produce quantifiable similarity scores.
Whether used for internal audits, competitive analysis, or content planning, this
method offers valuable insights into content overlap, uniqueness, and opportunity
areas. It empowers businesses to move beyond traditional SEO strategies and adopt
a more intelligent, data-driven approach to content optimization.
The outcome is not just a set of similarity scores, but a roadmap for:
● Strengthening content quality
● Reducing duplication

● Identifying content gaps
● Making smarter editorial decisions
As search engine algorithms continue to evolve and prioritize meaningful,
high-quality, and distinct content, adopting such methodologies — especially those
enhanced by deep learning models like RNNs — will help websites stay competitive
and search-relevant in the modern SEO landscape. Learn more.

Recurrent Neural Networks by ThatWare AI

More Related Content

Similar to Recurrent Neural Networks by ThatWare AI

More from ThatWare

Recently uploaded

Recurrent Neural Networks by ThatWare AI