Test Execution

The execution module runs property-based tests against routes using generated data and validates responses according to configuration.

Overview

Test execution is the final stage of the pytest-routes pipeline:

1. Route Discovery - Extract routes from your application

2. Strategy Generation - Create Hypothesis strategies for parameters

3. Test Creation - Build Hypothesis-powered test functions

4. Execution - Run tests with generated data

5. Validation - Check responses against expected criteria

Quick Start

Create and run tests programmatically:

from pytest_routes import (
    RouteTestConfig,
    RouteTestRunner,
    get_extractor,
)

# Configure testing
config = RouteTestConfig(max_examples=50)

# Extract routes
extractor = get_extractor(app)
routes = extractor.extract_routes(app)

# Create runner and execute tests
runner = RouteTestRunner(app, config)

for route in routes:
    test_func = runner.create_test(route)
    try:
        test_func()
        print(f"PASS: {route}")
    except AssertionError as e:
        print(f"FAIL: {route}")
        print(e)

Architecture

+------------------+
| RouteTestRunner  |
+--------+---------+
         |
         | creates
         v
+------------------+     +-------------------+
|  Hypothesis Test |---->| RouteTestClient   |
|  (@given)        |     | (ASGI client)     |
+------------------+     +-------------------+
         |                        |
         | on failure             | HTTP request
         v                        v
+------------------+     +-------------------+
| RouteTestFailure |     | ASGI Application  |
| (detailed error) |     +-------------------+
+------------------+

API Reference

Test Runner

RouteTestRunner

class pytest_routes.execution.runner.RouteTestRunner

Bases: object

Executes smoke tests against routes.

Parameters:

• app (Any)

• config (RouteTestConfig)

The main test runner that orchestrates test creation and execution.

Example

from pytest_routes import RouteTestRunner, RouteTestConfig

config = RouteTestConfig(
    max_examples=100,
    fail_on_5xx=True,
    verbose=True,
)

runner = RouteTestRunner(app, config)

# Create a test for a specific route
test = runner.create_test(route)
test()  # Runs 100 Hypothesis examples

# Or test all routes asynchronously
import asyncio
results = asyncio.run(runner.test_all_routes(routes))

__init__(app, config)

Initialize test runner.

Parameters:

• app (Any) – The ASGI application.

• config (RouteTestConfig) – Test configuration.

create_test(route)

Create a Hypothesis test for a route.

Parameters:

route (RouteInfo) – The route to test.

Return type:

Callable[[], None]

Returns:

A test function decorated with @given.

async test_route_async(route)

Test a single route asynchronously.

Parameters:

route (RouteInfo) – The route to test.

Return type:

dict[str, Any]

Returns:

Test result dictionary.

async test_all_routes(routes)

Test all routes.

Parameters:

routes (list[RouteInfo]) – List of routes to test.

Return type:

list[dict[str, Any]]

Returns:

List of test results.

RouteTestFailure

class pytest_routes.execution.runner.RouteTestFailure

Bases: object

Detailed information about a route test failure.

Parameters:

• route_path (str)

• method (str)

• status_code (int)

• expected_codes (list[int])

• request_path (str)

• path_params (dict[str, Any])

• query_params (dict[str, Any])

• body (Any)

• response_body (str | None)

• error_type (str)

• request_headers (dict[str, str])

• response_headers (dict[str, str])

• auth_type (str | None)

Detailed information about a test failure, including the shrunk example from Hypothesis.

Attributes

route_path

The original route pattern (e.g., /users/{user_id})

method

HTTP method that failed

status_code

Actual status code received

expected_codes

List of allowed status codes

request_path

The actual request URL with parameters filled in

path_params

Path parameter values that caused the failure (shrunk)

query_params

Query parameter values that caused the failure (shrunk)

body

Request body that caused the failure (shrunk)

response_body

Response body from the server (truncated)

error_type

Type of error: "server_error_5xx", "unexpected_status", or "validation_error"

Example Output

============================================================
ROUTE TEST FAILURE: GET /users/{user_id}
============================================================

Error Type:
  unexpected_status

Request Details:
  Method: GET
  Path: /users/0
  Status Code: 404
  Expected: [200, 201, 204, 400, 401, 403, 422]

Path Parameters (shrunk example):
  user_id: 0

Response Body (truncated):
  {"detail": "User not found"}

============================================================

route_path: str

method: str

status_code: int

expected_codes: list[int]

request_path: str

path_params: dict[str, Any]

query_params: dict[str, Any]

body: Any = None

response_body: str | None = None

error_type: str = 'unexpected_status'

request_headers: dict[str, str]

response_headers: dict[str, str]

auth_type: str | None = None

format_message()

Format a detailed error message with shrunk example.

Return type:

str

__init__(route_path, method, status_code, expected_codes, request_path, path_params=<factory>, query_params=<factory>, body=None, response_body=None, error_type='unexpected_status', request_headers=<factory>, response_headers=<factory>, auth_type=None)

Parameters:

• route_path (str)

• method (str)

• status_code (int)

• expected_codes (list[int])

• request_path (str)

• path_params (dict[str, Any])

• query_params (dict[str, Any])

• body (Any)

• response_body (str | None)

• error_type (str)

• request_headers (dict[str, str])

• response_headers (dict[str, str])

• auth_type (str | None)

Test Client

RouteTestClient

class pytest_routes.execution.client.RouteTestClient

Bases: object

Async test client for ASGI applications.

Parameters:

• app (Any)

• base_url (str)

An async ASGI test client wrapper for making requests to your application.

Example

from pytest_routes import RouteTestClient

async def test_endpoint():
    client = RouteTestClient(app)

    response = await client.request(
        method="GET",
        path="/users/123",
        params={"include_posts": "true"},
        timeout=30.0,
    )

    assert response.status_code == 200

__init__(app, base_url='http://test')

Initialize test client.

Parameters:

• app (Any) – The ASGI application.

• base_url (str) – Base URL for requests.

async request(method, path, *, params=None, json=None, headers=None, timeout=30.0)

Make an HTTP request to the ASGI app.

Parameters:

• method (str) – HTTP method.

• path (str) – Request path.

• params (dict[str, Any] | None) – Query parameters.

• json (Any) – JSON body.

• headers (dict[str, str] | None) – Request headers.

• timeout (float) – Request timeout in seconds.

Return type:

Response

Returns:

The HTTP response.

async get(path, **kwargs)

Make a GET request.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Response

async post(path, **kwargs)

Make a POST request.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Response

async put(path, **kwargs)

Make a PUT request.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Response

async patch(path, **kwargs)

Make a PATCH request.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Response

async delete(path, **kwargs)

Make a DELETE request.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Response

Understanding Test Creation

The RouteTestRunner.create_test() method creates a Hypothesis-decorated test function:

# What create_test() generates internally:

@settings(
    max_examples=config.max_examples,
    suppress_health_check=[HealthCheck.too_slow],
    deadline=None,
)
@given(
    path_params=generate_path_params(route.path_params, route.path),
    query_params=st.fixed_dictionaries({...}),
    body=generate_body(route.body_type),
)
def test_route(path_params, query_params, body):
    # Format path with generated params
    formatted_path = format_path(route.path, path_params)

    # Make request
    response = client.request(
        method=route.methods[0],
        path=formatted_path,
        params=query_params,
        json=body,
    )

    # Validate response
    validate_response(response, route)

Async Execution

For async test execution, use the async methods:

import asyncio
from pytest_routes import RouteTestRunner, RouteTestConfig, get_extractor

async def run_smoke_tests():
    config = RouteTestConfig(max_examples=25)
    runner = RouteTestRunner(app, config)
    extractor = get_extractor(app)
    routes = extractor.extract_routes(app)

    # Test single route
    result = await runner.test_route_async(routes[0])
    print(f"Route: {result['route']}, Passed: {result['passed']}")

    # Test all routes
    results = await runner.test_all_routes(routes)

    passed = sum(1 for r in results if r["passed"])
    failed = len(results) - passed
    print(f"Results: {passed} passed, {failed} failed")

asyncio.run(run_smoke_tests())

Verbose Mode

Enable verbose output for debugging:

config = RouteTestConfig(verbose=True)
runner = RouteTestRunner(app, config)

# Output during test execution:
#   -> GET /users/42
#     path_params: {'user_id': 42}
#     query_params: {'include_posts': True}
#   <- 200

Error Handling

The runner distinguishes between different error types:

Server Errors (5xx)

Caught when fail_on_5xx=True (default). These indicate bugs in your application.

Unexpected Status Codes

When the response status is not in allowed_status_codes. Configure this list based on your API’s expected behavior.

Validation Errors

When validate_responses=True and validators fail. See Response Validation for details.

config = RouteTestConfig(
    fail_on_5xx=True,                    # Fail on server errors
    allowed_status_codes=[200, 201, 204, 400, 404, 422],  # Expected codes
    validate_responses=True,             # Enable response validation
    fail_on_validation_error=True,       # Fail on validation errors
)

See Also

• Configuration - Configuration options for test execution

• Route Discovery - How routes are extracted for testing

• Strategy Generation - How test data is generated

• Response Validation - Response validation details