11 KiB

Raw Blame History

stage	group	info
none	unassigned	To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments

Beginner's guide to writing end-to-end tests

This tutorial walks you through the creation of end-to-end (e2e) tests for GitLab Community Edition and GitLab Enterprise Edition.

By the end of this tutorial, you can:

Determine whether an end-to-end test is needed.
Understand the directory structure within qa/.
Write a basic end-to-end test that validates login features.
Develop any missing page object libraries.

Before you write a test

Before you write tests, your GitLab Development Kit (GDK) must be configured to run the specs. The end-to-end tests:

Are contained within the qa/ directory.
Should be independent and idempotent.
Create resources (such as project, issue, user) on an ad-hoc basis.
Test the UI and API interfaces, and use the API to efficiently set up the UI tests.

TIP: Tip: For more information, see End-to-end testing Best Practices.

Determine if end-to-end tests are needed

Check the code coverage of a specific feature before writing end-to-end tests, for both GitLab Community Edition and GitLab Enterprise Edition projects. Does sufficient test coverage exist at the unit, feature, or integration levels? If you answered yes, then you don't need an end-to-end test.

For information about the distribution of tests per level in GitLab, see Testing Levels.

See the How to test at the correct level? section of the Testing levels document.
Review how often the feature changes. Stable features that don't change very often might not be worth covering with end-to-end tests if they are already covered in lower level tests.
Finally, discuss the proposed test with the developer(s) involved in implementing the feature and the lower-level tests.

CAUTION: Caution: Check both GitLab Community Edition and GitLab Enterprise Edition coverage projects for previously-written tests for this feature. For analyzing the code coverage, you must understand which application files implement specific features.

In this tutorial we're writing a login end-to-end test, even though it has been sufficiently covered by lower-level testing, because it's the first step for most end-to-end flows, and is easiest to understand.

Identify the DevOps stage

The GitLab QA end-to-end tests are organized by the different stages in the DevOps lifecycle. Determine where the test should be placed by stage, determine which feature the test belongs to, and then place it in a subdirectory under the stage.

If the test is Enterprise Edition only, the test is created in the features/ee directory, but follow the same DevOps lifecycle format.

Create a skeleton test

In the first part of this tutorial we are testing login, which is owned by the Manage stage. Inside qa/specs/features/browser_ui/1_manage/login, create a file basic_login_spec.rb.

The outer `context` block

See the RSpec.describe outer block

CAUTION: Deprecation notice: The outer context was deprecated in 13.2 in adherence to RSpec 4.0 specifications. Use RSpec.describe instead.

The outer `RSpec.describe` block

Specs have an outer RSpec.describe indicating the DevOps stage.

# frozen_string_literal: true

module QA
  RSpec.describe 'Manage' do

  end
end

The `describe` block

Inside of our outer RSpec.describe, describe the feature to test. In this case, Login.

# frozen_string_literal: true

module QA
  RSpec.describe 'Manage' do
    describe 'Login' do

    end
  end
end

The `it` blocks (examples)

Every test suite contains at least one it block (example). A good way to start writing end-to-end tests is to write test case descriptions as it blocks:

module QA
  RSpec.describe 'Manage' do
    describe 'Login' do
      it 'can login' do

      end

      it 'can logout' do

      end
    end
  end
end

Write the test

An important question is "What do we test?" and even more importantly, "How do we test?"

Begin by logging in.

# frozen_string_literal: true

module QA
  RSpec.describe 'Manage' do
    describe 'Login' do
      it 'can login' do
        Flow::Login.sign_in

      end

      it 'can logout' do
        Flow::Login.sign_in

      end
    end
  end
end

After running the spec, our test should login and end; then we should answer the question "What do we test?"

# frozen_string_literal: true

module QA
  RSpec.describe 'Manage' do
    describe 'Login' do
      it 'can login' do
        Flow::Login.sign_in

        Page::Main::Menu.perform do |menu|
          expect(menu).to be_signed_in
        end
      end

      it 'can logout' do
        Flow::Login.sign_in

        Page::Main::Menu.perform do |menu|
          menu.sign_out

          expect(menu).not_to be_signed_in
        end
      end
    end
  end
end

What do we test?

Can we sign in?
Can we sign out?

How do we test?

Check if the user avatar appears in the top navigation.
Check if the user avatar does not appear in the top navigation.

Behind the scenes, be_signed_in is a predicate matcher that implements checking the user avatar.

De-duplicate your code

Refactor your test to use a before block for test setup, since it's duplicating a call to sign_in.

# frozen_string_literal: true

module QA
  RSpec.describe 'Manage' do
    describe 'Login' do
      before do
        Flow::Login.sign_in
      end

      it 'can login' do
        Page::Main::Menu.perform do |menu|
          expect(menu).to be_signed_in
        end
      end

      it 'can logout' do
        Page::Main::Menu.perform do |menu|
          menu.sign_out

          expect(menu).not_to be_signed_in
        end
      end
    end
  end
end

The before block is essentially a before(:each) and is run before each example, ensuring we now log in at the beginning of each test.

Test setup using resources and page objects

Next, let's test something other than Login. Let's test Issues, which are owned by the Plan stage, so create a file in qa/specs/features/browser_ui/3_create/issues called issues_spec.rb.

# frozen_string_literal: true

module QA
  RSpec.describe 'Plan' do
    describe 'Issues' do
      let(:issue) do
        Resource::Issue.fabricate_via_api! do |issue|
          issue.title = 'My issue'
          issue.description = 'This is an issue specific to this test'
        end
      end

      before do
        Flow::Login.sign_in
        issue.visit!
      end

      it 'can close an issue' do
        Page::Project::Issue::Show.perform do |show|
          show.click_close_issue_button

          expect(show).to be_closed
        end
      end
    end
  end
end

Note the following important points:

At the start of our example, we are at the page/issue/show.rb page.
Our test fabricates only what it needs, when it needs it.
The issue is fabricated through the API to save time.
GitLab prefers let() over instance variables. See best practices.
be_closed is not implemented in page/project/issue/show.rb yet, but is implemented in the next step.

The issue is fabricated as a Resource, which is a GitLab entity you can create through the UI or API. Other examples include:

Write the page object

A Page Object is a class in our suite that represents a page within GitLab. The Login page would be one example. Since our page object for the Issue Show page already exists, add the closed? method.

module Page::Project::Issue
  class Show
    view 'app/views/projects/issues/show.html.haml' do
      element :closed_status_box
    end

    def closed?
      has_element?(:closed_status_box)
    end
  end
end

Next, define the element closed_status_box within your view, so your Page Object can see it.

-#=> app/views/projects/issues/show.html.haml
.issuable-status-box.status-box.status-box-issue-closed{ ..., data: { qa_selector: 'closed_status_box' } }

Run the spec

Before running the spec, confirm:

The GDK is installed.
The GDK is running on port 3000 locally.
No additional RSpec metadata tags have been applied.
Your working directory is qa/ within your GDK GitLab installation.

To run the spec, run the following command:

bundle exec bin/qa Test::Instance::All http://localhost:3000 -- <test_file>

Where <test_file> is:

qa/specs/features/browser_ui/1_manage/login/login_spec.rb when running the Login example.
qa/specs/features/browser_ui/2_plan/issues/issue_spec.rb when running the Issue example.

End-to-end test merge request template

When submitting a new end-to-end test, use the "New End to End Test" merge request description template for additional steps that are required prior a successful merge.

11 KiB Raw Blame History