freecourses.digital
Lesson progress: 0 of 2 activities done
All lessons in Digital and Web Analytics Lesson 5.5 of 32

Module 1: Introduction to Digital Analytics

  1. 1.1What is digital analytics?
  2. 1.2The four types of analytics
  3. 1.3Connecting analytics to business objectives and KPIs

Module 2: Analytics Data and Measurement Planning

  1. 2.1How to build a measurement plan
  2. 2.2Key metrics for each marketing channel
  3. 2.3Data sources, collection, and data quality
  4. 2.4Segmentation and personalization

Module 3: Web Analytics Fundamentals

  1. 3.1Users, sessions, page views, and events
  2. 3.2Traffic sources, channels, and UTM parameters
  3. 3.3Landing pages and user behavior
  4. 3.4Engagement metrics and conversion metrics
  5. 3.5Reading reports, trends, and anomalies

Module 4: Analytics Tools and Platforms

  1. 4.1Google Analytics 4 fundamentals
  2. 4.2Google Search Console performance data
  3. 4.3Adobe Analytics overview
  4. 4.4Tag management and tracking implementation
  5. 4.5Connecting tools and choosing the right stack

Module 5: Conversion Funnels and Optimization

  1. 5.1What is a conversion?
  2. 5.2Conversion funnels and drop-off points
  3. 5.3How to calculate conversion rate
  4. 5.4Prioritizing conversion improvements
  5. 5.5Introduction to A/B testing

Module 6: Digital Marketing Performance Analysis

  1. 6.1Measuring SEO and paid advertising performance
  2. 6.2Social, content, and landing page performance
  3. 6.3Comparing acquisition channels
  4. 6.4Evaluating campaigns and avoiding vanity metrics

Module 7: Reporting, Visualization, and Insights

  1. 7.1Building a marketing performance report
  2. 7.2Data visualization and dashboards
  3. 7.3Comparing periods and benchmarking
  4. 7.4Turning data into actionable recommendations

Module 8: Analytics Projects and Practical Assessment

  1. 8.1Analytics case study: Isla Coffee Co.
  2. 8.2Final assessment: digital analytics case study

Introduction to A/B testing

Your roadmap for this lesson

  • How an A/B test works
  • The six steps of a sound test
  • Mistakes that ruin test results

Short answer

An A/B test shows two versions of a page to similar users at the same time and measures which converts better. Start with a hypothesis, change one main element, split traffic randomly, and run until you reach enough sample size and statistical significance. Then roll out the winner and record what you learned.

Six steps

  1. Form a hypothesis: “Showing shipping cost on the product page will raise add-to-cart rate because users fear hidden fees.”
  2. Create versions: A (control) and B (variant), changing one main element.
  3. Split traffic randomly.
  4. Run long enough: enough sample size and at least one full business cycle (usually 2 or more weeks).
  5. Check significance: commonly 95% before declaring a winner.
  6. Implement and document the winner and what you learned.

Common mistakes

  • Stopping early because one version looks ahead after two days.
  • Testing too many changes at once, so you cannot tell what worked.
  • Ignoring segments: a winner on desktop may lose on mobile.

Quick quiz

1. Why not stop a test after two days?
2. What is the control?

Your action

Key terms

A/B test
A comparison of two versions with split traffic.
Control
The original version, A.
Variant
The changed version, B.
Statistical significance
Confidence that a result is not due to chance.

Questions beginners ask

How long should a test run?

At least one full business cycle, usually two weeks or more.

What if my site has little traffic?

Test bigger changes, or use surveys and recordings instead.

Related lessons

Want someone to set up your analytics and reporting while you learn? The team at Remar Digital can help.