History is littered with organizations, large and small, that bought disaster upon themselves by launching changes to their websites which seemed like a good idea at the time but had a devastating effect when placed in front of real world visitors.
A/B testing is a way of answering the question "What if ..." when it comes to website design - both in terms of presentation and process.
From a graphic design point of view it might be:
From a process point of view it might be:
Risk can be minimized by, for example, trying something out on 20% of visitors to your site while the remaining 80% see no difference at all. This means if a change turns out to be disastrous the damage done is limited.
For A/B testing smaller changes to your site Google offers a free service called Optimize that includes monitors which will tell you when enough data has been gathered for you to draw a conclusion as to what is better - version A or version B.
If you are planning to keep the same general design I always recommend small changes are tested one at a time. If you make multiple changes to a page or site it can be difficult to conclude which ones were effective. Changes that might have been positive could be hidden by the negative impacts of others and it all becomes a real mess.
For larger changes there are a variety of paid software options out there but you will need to take some time to do the careful research which will identify the best one for your specific needs.
Be careful of taking the results of someone else's A/B Testing and applying it to your website. Just because someone changed a particular color here or there and found it worked really well for them and they've made a YouTube video to tell you all about it doesn't mean it will work for you. In fact it could have the opposite effect.
Your website is not their website - to be safe do your own A/B Testing.
I'm Tim Hill, a Search Engine Optimization and Online Marketing specialist. I created this site to help others understand that SEO is not a mysterious black art!.
If your a newbie try the Getting Started in SEO page, otherwise feel free to dig around and learn more.