Natural Language Processing (NLP) is set to become a key part of how search engines rank web pages with Google already leading the field. So what is it and what should you be doing about it?
Early attempts to get computers to answer questions from humans assumed humans asked carefully worded questions and used correct grammar and spelling. Of course ... they don't and languages (especially English) can evolve rapidly with thousands of new words coming into use every year, not to mention new ways of using them. NLP uses Machine Learning to gather data about how people really talk and then try and figure out what they really mean.
Google is well placed to do this because users carry out billions of queries every month on the search engine and by watching which websites seem to meet their needs it gets to know those questions better.
As humans we find understanding vague questions fairly easy because we usually have context. If someone says to you "I want to buy a coat" we know the general weather conditions. If its high summer they probably want a light coat or jacket. We may have some historical context - yesterday they told you that they were going to a wedding so perhaps they need something elegant. You may know something about their tastes - color, cut, etc.
Traditionally search engines had to cope without any of this background although Google already considers our location and websites we have visited in the past. So its already starting to build context but what happens when someone asks a question it has never seen before?
Natural Language Processing aims to overcome this without requiring a human to adjust bits of code. Rather than being told the grammar rules it learns them. Instead of considering each word in a question individually it considers the sentence and the way the words are used in that sentence.
"I want to buy a book", "I want to book a flight" - here 'book' has very different meanings depending where it is used in the string of words.
This is just the tip of the iceberg with all sorts of fun things like stemming, segmentation, lemmatization, etc. If you really want to discover what goes into NLP here's the Wikipedia Page!
My focus here is what this means for Search Engine Optimization.
Search engines are primarily focused on answering "What is the best website page to answer this?" for any search that a user carries out. This came in two steps:
Using links was a great way to sort the quality from the rubbish in the early days of the Internet but relying on backlinks has fundamental flaws.
Websites which rank highly for a particular search term tend to gain more links over time. If I want to make a point, I go to Google and find the first article which supports my point, I link to it. But that does not mean it is the 'best' page on the Internet.
Getting good backlinks is easy when you have cash for link builders or plenty of time to do link building. The 'best' page may have been added to the web last month but Google is ignoring it because it has no links. The author of the 'best' content would need to hire link builders or organizing their own link building outreach campaign begging bloggers to link to them.
Getting backlinks is easy if you have cash for Google Ads. If you really have the 'best' content you can rank it number one tomorrow on Google Ads and it will then start to pick up backlinks much faster than if you wait for it to be found on page 4 of the search results!
Getting backlinks is easy if you have a few thousand websites. I'm talking really good quality websites where you have invested 7 figures in creating great content and there are people out there who have. Their bound to have a few dozen which are related to each other so when they make a new piece of content on one they can just turn on the tap and get links from all the others.
Getting backlinks is easy if you have celebrity status. Having access to other online marketing resources such as a large social media following in order to make large numbers of people aware of the new content generates backlinks. People like to be associated with success.
Getting backlinks is skewed towards the rich. Whatever Google says you can buy your way to the top with 'good' content and in doing so block out the 'best' content.
That's not a very good way of deciding what content is 'best' - that's not going to help Google maintain its position as the worlds number one search engine.
A human on the other hand can be shown two articles and, without being told which is more popular, quickly conclude which is 'best'. Search engines want to be able to do the same and make popularity a secondary factor - yes, I don't think links will be discounted completely at any point soon. To me 'popularity' is a natural part of 'best'.
What we will see is a tipping of the balance because Google will become better at understanding relevancy and it will be harder to game it.
So what should you be doing to search engine optimize with natural language processing in mind?
Natural Language Processing is coming to Internet search and it will probably be here faster than you think. If you continue to rely on backlinks when you know your content could be improved then prepare to see your rankings slip away.
The content you produce has now got to be second to none in quality and depth as well as readability and there are plenty of ways to achieve this right under your nose.