Improve documentation for TestContext events

This commit improves the documentation for test execution events,
especially with regard to the fact that, by default, a
BeforeTestClassEvent is not published for the first test class using a
particular ApplicationContext.

This commit also introduces tests that verify the default behavior and
the ability to change the default behavior with a custom
TestExecutionListener that eagerly loads the context.

Closes gh-27757

spring-test/src/main/java/org/springframework/test/context/event/EventPublishingTestExecutionListener.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2019 the original author or authors.
+ * Copyright 2002-2022 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -17,16 +17,12 @@
 package org.springframework.test.context.event;
 
 import org.springframework.test.context.TestContext;
-import org.springframework.test.context.TestExecutionListener;
 import org.springframework.test.context.support.AbstractTestExecutionListener;
 
 /**
- * {@link org.springframework.test.context.TestExecutionListener TestExecutionListener}
+ * {@code TestExecutionListener} that publishes test execution events to the
- * that publishes test execution events to the
  * {@link org.springframework.context.ApplicationContext ApplicationContext}
- * for the currently executing test. Events are only published if the
+ * for the currently executing test.
- * {@code ApplicationContext} {@linkplain TestContext#hasApplicationContext()
- * has already been loaded}.
  *
  * <h3>Supported Events</h3>
  * <ul>
@@ -41,11 +37,33 @@ import org.springframework.test.context.support.AbstractTestExecutionListener;
  *
  * <p>These events may be consumed for various reasons, such as resetting <em>mock</em>
  * beans or tracing test execution. One advantage of consuming test events rather
- * than implementing a custom {@link TestExecutionListener} is that test events
+ * than implementing a custom {@link org.springframework.test.context.TestExecutionListener
- * may be consumed by any Spring bean registered in the test {@code ApplicationContext},
+ * TestExecutionListener} is that test events may be consumed by any Spring bean
- * and such beans may benefit directly from dependency injection and other features
+ * registered in the test {@code ApplicationContext}, and such beans may benefit
- * of the {@code ApplicationContext}. In contrast, a {@link TestExecutionListener}
+ * directly from dependency injection and other features of the {@code ApplicationContext}.
- * is not a bean in the {@code ApplicationContext}.
+ * In contrast, a {@code TestExecutionListener} is not a bean in the {@code ApplicationContext}.
+ *
+ * <p>Note that the {@code EventPublishingTestExecutionListener} is registered by
+ * default; however, it only publishes events if the {@code ApplicationContext}
+ * {@linkplain TestContext#hasApplicationContext() has already been loaded}. This
+ * prevents the {@code ApplicationContext} from being loaded unnecessarily or too
+ * early. Consequently, a {@code BeforeTestClassEvent} will not be published until
+ * after the {@code ApplicationContext} has been loaded by another
+ * {@code TestExecutionListener}. For example, with the default set of
+ * {@code TestExecutionListeners} registered, a {@code BeforeTestClassEvent} will
+ * not be published for the first test class that uses a particular test
+ * {@code ApplicationContext}, but a {@code BeforeTestClassEvent} will be published
+ * for any subsequent test class in the same test suite that uses the same test
+ * {@code ApplicationContext} since the context will already have been loaded
+ * when subsequent test classes run (as long as the context has not been removed
+ * from the {@link org.springframework.test.context.cache.ContextCache ContextCache}
+ * via {@link org.springframework.test.annotation.DirtiesContext @DirtiesContext}
+ * or the max-size eviction policy). If you wish to ensure that a
+ * {@code BeforeTestClassEvent} is published for every test class, you need to
+ * register a {@code TestExecutionListener} that loads the {@code ApplicationContext}
+ * in the {@link org.springframework.test.context.TestExecutionListener#beforeTestClass
+ * beforeTestClass} callback, and that {@code TestExecutionListener} must be registered
+ * before the {@code EventPublishingTestExecutionListener}.
  *
  * <h3>Exception Handling</h3>
  * <p>By default, if a test event listener throws an exception while consuming

spring-test/src/main/java/org/springframework/test/context/event/annotation/AfterTestClass.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2019 the original author or authors.
+ * Copyright 2002-2022 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -49,6 +49,8 @@ import static java.lang.annotation.RetentionPolicy.RUNTIME;
  * <p>The {@code EventPublishingTestExecutionListener} must be registered in order
  * for this annotation to have an effect &mdash; for example, via
  * {@link org.springframework.test.context.TestExecutionListeners @TestExecutionListeners}.
+ * Note, however, that the {@code EventPublishingTestExecutionListener} is registered
+ * by default.
  *
  * @author Frank Scheffler
  * @author Sam Brannen

spring-test/src/main/java/org/springframework/test/context/event/annotation/AfterTestExecution.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2019 the original author or authors.
+ * Copyright 2002-2022 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -49,6 +49,8 @@ import static java.lang.annotation.RetentionPolicy.RUNTIME;
  * <p>The {@code EventPublishingTestExecutionListener} must be registered in order
  * for this annotation to have an effect &mdash; for example, via
  * {@link org.springframework.test.context.TestExecutionListeners @TestExecutionListeners}.
+ * Note, however, that the {@code EventPublishingTestExecutionListener} is registered
+ * by default.
  *
  * @author Frank Scheffler
  * @author Sam Brannen

spring-test/src/main/java/org/springframework/test/context/event/annotation/AfterTestMethod.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2019 the original author or authors.
+ * Copyright 2002-2022 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -49,6 +49,8 @@ import static java.lang.annotation.RetentionPolicy.RUNTIME;
  * <p>The {@code EventPublishingTestExecutionListener} must be registered in order
  * for this annotation to have an effect &mdash; for example, via
  * {@link org.springframework.test.context.TestExecutionListeners @TestExecutionListeners}.
+ * Note, however, that the {@code EventPublishingTestExecutionListener} is registered
+ * by default.
  *
  * @author Frank Scheffler
  * @author Sam Brannen

spring-test/src/main/java/org/springframework/test/context/event/annotation/BeforeTestClass.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2019 the original author or authors.
+ * Copyright 2002-2022 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -49,6 +49,8 @@ import static java.lang.annotation.RetentionPolicy.RUNTIME;
  * <p>The {@code EventPublishingTestExecutionListener} must be registered in order
  * for this annotation to have an effect &mdash; for example, via
  * {@link org.springframework.test.context.TestExecutionListeners @TestExecutionListeners}.
+ * Note, however, that the {@code EventPublishingTestExecutionListener} is registered
+ * by default.
  *
  * @author Frank Scheffler
  * @author Sam Brannen

spring-test/src/main/java/org/springframework/test/context/event/annotation/BeforeTestExecution.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2019 the original author or authors.
+ * Copyright 2002-2022 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -49,6 +49,8 @@ import static java.lang.annotation.RetentionPolicy.RUNTIME;
  * <p>The {@code EventPublishingTestExecutionListener} must be registered in order
  * for this annotation to have an effect &mdash; for example, via
  * {@link org.springframework.test.context.TestExecutionListeners @TestExecutionListeners}.
+ * Note, however, that the {@code EventPublishingTestExecutionListener} is registered
+ * by default.
  *
  * @author Frank Scheffler
  * @author Sam Brannen

spring-test/src/main/java/org/springframework/test/context/event/annotation/BeforeTestMethod.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2019 the original author or authors.
+ * Copyright 2002-2022 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -49,6 +49,8 @@ import static java.lang.annotation.RetentionPolicy.RUNTIME;
  * <p>The {@code EventPublishingTestExecutionListener} must be registered in order
  * for this annotation to have an effect &mdash; for example, via
  * {@link org.springframework.test.context.TestExecutionListeners @TestExecutionListeners}.
+ * Note, however, that the {@code EventPublishingTestExecutionListener} is registered
+ * by default.
  *
  * @author Frank Scheffler
  * @author Sam Brannen

spring-test/src/main/java/org/springframework/test/context/event/annotation/PrepareTestInstance.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2019 the original author or authors.
+ * Copyright 2002-2022 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -49,6 +49,8 @@ import static java.lang.annotation.RetentionPolicy.RUNTIME;
  * <p>The {@code EventPublishingTestExecutionListener} must be registered in order
  * for this annotation to have an effect &mdash; for example, via
  * {@link org.springframework.test.context.TestExecutionListeners @TestExecutionListeners}.
+ * Note, however, that the {@code EventPublishingTestExecutionListener} is registered
+ * by default.
  *
  * @author Frank Scheffler
  * @author Sam Brannen

spring-test/src/main/java/org/springframework/test/context/event/annotation/package-info.java

@@ -1,4 +1,4 @@
 /**
- * Test event annotations for the <em>Spring TestContext Framework</em>.
+ * Test execution event annotations for the <em>Spring TestContext Framework</em>.
  */
 package org.springframework.test.context.event.annotation;

spring-test/src/test/java/org/springframework/test/context/event/EagerTestExecutionEventPublishingTests.java

@@ -0,0 +1,190 @@
+/*
+ * Copyright 2002-2022 the original author or authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.springframework.test.context.event;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.platform.testkit.engine.EngineTestKit;
+
+import org.springframework.context.annotation.Configuration;
+import org.springframework.core.annotation.Order;
+import org.springframework.test.context.TestContext;
+import org.springframework.test.context.TestExecutionListener;
+import org.springframework.test.context.TestExecutionListeners;
+import org.springframework.test.context.TestExecutionListeners.MergeMode;
+import org.springframework.test.context.event.annotation.AfterTestClass;
+import org.springframework.test.context.event.annotation.AfterTestExecution;
+import org.springframework.test.context.event.annotation.AfterTestMethod;
+import org.springframework.test.context.event.annotation.BeforeTestClass;
+import org.springframework.test.context.event.annotation.BeforeTestExecution;
+import org.springframework.test.context.event.annotation.BeforeTestMethod;
+import org.springframework.test.context.event.annotation.PrepareTestInstance;
+import org.springframework.test.context.junit.jupiter.SpringJUnitConfig;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;
+
+/**
+ * Tests for the {@link EventPublishingTestExecutionListener} which verify that
+ * a {@link BeforeTestClassEvent} can be eagerly published; whereas, such an
+ * event is not published by default for the first run of a test class for a
+ * specific {@code ApplicationContext}.
+ *
+ * @author Sam Brannen
+ * @since 5.3.17
+ * @see https://github.com/spring-projects/spring-framework/issues/27757
+ */
+class EagerTestExecutionEventPublishingTests {
+
+	private static final List<Class<? extends TestContextEvent>> events = new ArrayList<>();
+
+
+	@BeforeEach
+	@AfterEach
+	void resetEvents() {
+		events.clear();
+	}
+
+	@Test
+	void beforeTestClassEventIsNotPublishedByDefaultForFirstTestClass() {
+		EngineTestKit.engine("junit-jupiter")//
+				.selectors(selectClass(LazyTestCase1.class), selectClass(LazyTestCase2.class))//
+				.execute()//
+				.testEvents()//
+				.assertStatistics(stats -> stats.started(2).succeeded(2).failed(0));
+
+		assertThat(events).containsExactly(//
+			// 1st test class
+			// BeforeTestClassEvent.class -- missing for 1st test class
+			PrepareTestInstanceEvent.class, //
+			BeforeTestMethodEvent.class, //
+			BeforeTestExecutionEvent.class, //
+			AfterTestExecutionEvent.class, //
+			AfterTestMethodEvent.class, //
+			AfterTestClassEvent.class, //
+			// 2nd test class
+			BeforeTestClassEvent.class, //
+			PrepareTestInstanceEvent.class, //
+			BeforeTestMethodEvent.class, //
+			BeforeTestExecutionEvent.class, //
+			AfterTestExecutionEvent.class, //
+			AfterTestMethodEvent.class, //
+			AfterTestClassEvent.class//
+		);
+	}
+
+	@Test
+	void beforeTestClassEventIsPublishedForAllTestClassesIfCustomListenerEagerlyLoadsContext() {
+		EngineTestKit.engine("junit-jupiter")//
+				.selectors(selectClass(EagerTestCase1.class), selectClass(EagerTestCase2.class))//
+				.execute()//
+				.testEvents()//
+				.assertStatistics(stats -> stats.started(2).succeeded(2).failed(0));
+
+		assertThat(events).containsExactly(//
+			// 1st test class
+			BeforeTestClassEvent.class, //
+			PrepareTestInstanceEvent.class, //
+			BeforeTestMethodEvent.class, //
+			BeforeTestExecutionEvent.class, //
+			AfterTestExecutionEvent.class, //
+			AfterTestMethodEvent.class, //
+			AfterTestClassEvent.class, //
+			// 2nd test class
+			BeforeTestClassEvent.class, //
+			PrepareTestInstanceEvent.class, //
+			BeforeTestMethodEvent.class, //
+			BeforeTestExecutionEvent.class, //
+			AfterTestExecutionEvent.class, //
+			AfterTestMethodEvent.class, //
+			AfterTestClassEvent.class//
+		);
+	}
+
+
+	@SpringJUnitConfig(Config.class)
+	static class LazyTestCase1 {
+
+		@Test
+		void test() {
+		}
+	}
+
+	static class LazyTestCase2 extends LazyTestCase1 {
+	}
+
+	@TestExecutionListeners(listeners = EagerLoadingTestExecutionListener.class, mergeMode = MergeMode.MERGE_WITH_DEFAULTS)
+	static class EagerTestCase1 extends LazyTestCase1 {
+	}
+
+	static class EagerTestCase2 extends EagerTestCase1 {
+	}
+
+	@Configuration
+	static class Config {
+
+		@BeforeTestClass
+		public void beforeTestClass(BeforeTestClassEvent e) {
+			events.add(e.getClass());
+		}
+
+		@PrepareTestInstance
+		public void prepareTestInstance(PrepareTestInstanceEvent e) {
+			events.add(e.getClass());
+		}
+
+		@BeforeTestMethod
+		public void beforeTestMethod(BeforeTestMethodEvent e) {
+			events.add(e.getClass());
+		}
+
+		@BeforeTestExecution
+		public void beforeTestExecution(BeforeTestExecutionEvent e) {
+			events.add(e.getClass());
+		}
+
+		@AfterTestExecution
+		public void afterTestExecution(AfterTestExecutionEvent e) {
+			events.add(e.getClass());
+		}
+
+		@AfterTestMethod
+		public void afterTestMethod(AfterTestMethodEvent e) {
+			events.add(e.getClass());
+		}
+
+		@AfterTestClass
+		public void afterTestClass(AfterTestClassEvent e) {
+			events.add(e.getClass());
+		}
+
+	}
+
+	@Order(0)
+	static class EagerLoadingTestExecutionListener implements TestExecutionListener {
+
+		@Override
+		public void beforeTestClass(TestContext testContext) {
+			testContext.getApplicationContext();
+		}
+	}
+
+}

src/docs/asciidoc/testing.adoc

@@ -2672,8 +2672,6 @@ test's `ApplicationContext` can listen to the following events published by the
 * `AfterTestMethodEvent`
 * `AfterTestClassEvent`
 
-NOTE: These events are only published if the `ApplicationContext` has already been loaded.
-
 These events may be consumed for various reasons, such as resetting mock beans or tracing
 test execution. One advantage of consuming test execution events rather than implementing
 a custom `TestExecutionListener` is that test execution events may be consumed by any
@@ -2681,6 +2679,28 @@ Spring bean registered in the test `ApplicationContext`, and such beans may bene
 directly from dependency injection and other features of the `ApplicationContext`. In
 contrast, a `TestExecutionListener` is not a bean in the `ApplicationContext`.
 
+[NOTE]
+====
+The `EventPublishingTestExecutionListener` is registered by default; however, it only
+publishes events if the `ApplicationContext` has _already been loaded_. This prevents the
+`ApplicationContext` from being loaded unnecessarily or too early.
+
+Consequently, a `BeforeTestClassEvent` will not be published until after the
+`ApplicationContext` has been loaded by another `TestExecutionListener`. For example, with
+the default set of `TestExecutionListener` implementations registered, a
+`BeforeTestClassEvent` will not be published for the first test class that uses a
+particular test `ApplicationContext`, but a `BeforeTestClassEvent` _will_ be published for
+any subsequent test class in the same test suite that uses the same test
+`ApplicationContext` since the context will already have been loaded when subsequent test
+classes run (as long as the context has not been removed from the `ContextCache` via
+`@DirtiesContext` or the max-size eviction policy).
+
+If you wish to ensure that a `BeforeTestClassEvent` is always published for every test
+class, you need to register a `TestExecutionListener` that loads the `ApplicationContext`
+in the `beforeTestClass` callback, and that `TestExecutionListener` must be registered
+_before_ the `EventPublishingTestExecutionListener`.
+====
+
 In order to listen to test execution events, a Spring bean may choose to implement the
 `org.springframework.context.ApplicationListener` interface. Alternatively, listener
 methods can be annotated with `@EventListener` and configured to listen to one of the