Icono del sitio IT&ライフハックブログ|学びと実践のためのアイデア集

A Practical Introduction to Audit Log Design in FastAPI: Design and Implementation Patterns for Safely Recording Who Did What and When

green snake

Photo by Pixabay on Pexels.com

A Practical Introduction to Audit Log Design in FastAPI: Design and Implementation Patterns for Safely Recording Who Did What and When


Summary

  • Audit logs have a different purpose from ordinary application logs. They are not only for troubleshooting, but also for preserving evidence of important actions such as permission changes, data updates, downloads, and deletions.
  • In FastAPI, it is practical to organize request-level information, the authenticated user, the tenant, the target resource, and the operation result, then record audit events from the service layer or shared helpers.
  • The most important step is deciding what to record first. If you try to record everything, operations become difficult; if you record too little, you will not be able to investigate later.
  • In real-world systems, there are two main approaches: outputting audit events as structured logs, or storing them in a dedicated audit table. Structured logs are fine at first, but dedicated storage becomes more attractive when you consider searchability and retention.
  • This article gradually organizes the basic policy for audit logs in FastAPI, event design, database storage, the division of responsibilities with middleware, multi-tenant support, and testing considerations.

Who will benefit from reading this?

Individual developers and learners

This is for people building admin panels or login features who want to check later things like, “Who deleted this?” or “When was this setting changed?”
Even a simple audit log at the beginning can greatly improve the trustworthiness of your application.

Backend engineers on small teams

This is for those who are adding user management, permission changes, CSV exports, file downloads, and similar features, and feel that ordinary logs are no longer enough for tracking.
By introducing a dedicated audit log design, handling customer inquiries and incident investigations becomes easier.

SaaS teams and startups

This is aimed at teams running multi-tenant systems or handling operations with strong admin privileges, where legal, security, or customer requirements make evidence tracking necessary.
It is especially useful when you want to consistently record who did what, in which tenant, and in what format.


Accessibility evaluation

  • This article first explains what an audit log is, then moves in order through what to record, where to store it, and how to implement it.
  • Technical terms are briefly explained when they first appear, and then used consistently afterward so the flow is easy to follow.
  • Code examples are split into short sections, each with a single responsibility.
  • The target level is roughly equivalent to AA.

1. What is an audit log?

An audit log is different from a simple debug log.
Its purpose is to make it possible to confirm later the evidence of important operations.

For example, it should help answer questions like these:

  • Who deleted this record?
  • When was a user’s role changed?
  • In which tenant did someone download a file?
  • Which settings did an administrator modify?
  • When did an important operation happen, even if it did not cause an error?

In other words, audit logs are the foundation not only for troubleshooting, but also for operations, security, and customer support.


2. The difference between ordinary logs and audit logs

These are easy to confuse, so it helps to separate them clearly from the start.

Ordinary logs

  • For development and troubleshooting
  • Record stack traces, database errors, processing times, and so on
  • Operated with levels such as DEBUG / INFO / WARNING / ERROR

Audit logs

  • For evidence and traceability
  • Successful operations are also recorded
  • Store “who did what, where, and how” in a consistent format
  • Ideally stored somewhere that is difficult to delete or tamper with

For example, a failed login may be worth logging in ordinary logs, but a “user role change” is the kind of event you usually want to preserve as an audit log as well.


3. What should you record? Start by designing the events

The first thing to do in audit logging is not implementation, but event design.
That means deciding which actions are audit-worthy.

3.1 A minimum recommended set of targets

  • Login and logout
  • Password changes
  • Permission changes and role changes
  • Create, update, and delete operations
  • Exports such as CSV or PDF
  • File downloads
  • API key issuance and revocation
  • Tenant setting changes
  • Billing plan changes

3.2 Do not record too much

Audit logs are best suited for important operations.
If you record every list view and every normal read operation, the volume can become unmanageable.

At first, it is easier to decide using criteria like these:

  • Is there likely to be accountability later?
  • Would you want to trace this during customer support?
  • Is evidence required from a security perspective?
  • Is this the kind of operation you want to record even when it succeeds?

4. Fields that should ideally be included in a single audit log entry

It becomes much easier later if you standardize the format of audit logs early.
At minimum, the following fields are useful:

  • timestamp
    When it happened
  • actor_type
    The type of actor, for example: user, service, system
  • actor_id
    The actor’s ID
  • tenant_id
    Which tenant the event occurred in
  • action
    What was done, for example: user.create, project.delete
  • resource_type
    The kind of target, for example: user, project, invoice
  • resource_id
    The target ID
  • result
    success or failure
  • request_id
    An ID for tracing the request
  • ip_address
    Source IP
  • user_agent
    Client information
  • detail
    Additional information, such as before/after values or a reason

You do not need to populate every field every time, but having them in the schema makes the system easier to handle.


5. Decide on a naming convention for event names

To make searching easier later, it is a good idea to standardize the naming of action.

For example:

  • user.login
  • user.logout
  • user.role.update
  • project.create
  • project.update
  • project.delete
  • file.download
  • api_key.revoke

A format like <resource>.<verb> or <resource>.<field>.<verb> makes events easier to scan and filter.


6. Basic policy in FastAPI: record audit logs mainly in the service layer, not only in middleware

It is safer not to try to complete audit logging using middleware alone.

Why?

Middleware can tell you that an HTTP request arrived, but it usually cannot tell you:

  • What was actually updated
  • Which specific resource ID was affected
  • Whether the event was important from a business perspective

So a practical division of responsibility looks like this:

  • Middleware
    • Collects common information such as request_id, IP address, and User-Agent
  • Service layer
    • Records the actual important events
  • Audit log storage layer
    • Persists them to the database or structured logs

7. First, create a Pydantic model

If you fix the audit event shape in code, you can use the same format everywhere.

# app/audit/schemas.py
from pydantic import BaseModel
from typing import Any, Literal
from datetime import datetime

class AuditEvent(BaseModel):
    timestamp: datetime
    actor_type: Literal["user", "service", "system"]
    actor_id: str | None = None
    tenant_id: int | None = None
    action: str
    resource_type: str | None = None
    resource_id: str | None = None
    result: Literal["success", "failure"]
    request_id: str | None = None
    ip_address: str | None = None
    user_agent: str | None = None
    detail: dict[str, Any] | None = None

You can treat this model as the shared container for audit events.


8. Minimal implementation: output to structured logs

The simplest first step is to emit audit events as structured logs instead of storing them in the database.
It is easy to introduce and works well as a starting point.

# app/audit/logger.py
import logging
from app.audit.schemas import AuditEvent

audit_logger = logging.getLogger("audit")

def write_audit_log(event: AuditEvent) -> None:
    audit_logger.info(
        "audit_event",
        extra={
            "timestamp": event.timestamp.isoformat(),
            "actor_type": event.actor_type,
            "actor_id": event.actor_id,
            "tenant_id": event.tenant_id,
            "action": event.action,
            "resource_type": event.resource_type,
            "resource_id": event.resource_id,
            "result": event.result,
            "request_id": event.request_id,
            "ip_address": event.ip_address,
            "user_agent": event.user_agent,
            "detail": event.detail,
        },
    )

With this approach, you can feed audit events directly into an existing JSON logging pipeline.
However, if you want detailed searching later, a dedicated database table is usually more convenient.


9. Create a dedicated table in the database

Once operations become more serious, having a dedicated audit log table becomes much easier to manage.

# app/audit/models.py
from sqlalchemy.orm import Mapped, mapped_column
from sqlalchemy import Integer, String, DateTime, Text, JSON
from datetime import datetime

from app.db.base import Base

class AuditLog(Base):
    __tablename__ = "audit_logs"

    id: Mapped[int] = mapped_column(Integer, primary_key=True)
    timestamp: Mapped[datetime] = mapped_column(DateTime, nullable=False)
    actor_type: Mapped[str] = mapped_column(String(20), nullable=False)
    actor_id: Mapped[str | None] = mapped_column(String(100), nullable=True)
    tenant_id: Mapped[int | None] = mapped_column(Integer, nullable=True, index=True)
    action: Mapped[str] = mapped_column(String(100), nullable=False, index=True)
    resource_type: Mapped[str | None] = mapped_column(String(100), nullable=True)
    resource_id: Mapped[str | None] = mapped_column(String(100), nullable=True)
    result: Mapped[str] = mapped_column(String(20), nullable=False)
    request_id: Mapped[str | None] = mapped_column(String(100), nullable=True, index=True)
    ip_address: Mapped[str | None] = mapped_column(String(100), nullable=True)
    user_agent: Mapped[str | None] = mapped_column(Text, nullable=True)
    detail: Mapped[dict | None] = mapped_column(JSON, nullable=True)

Design points

  • It is useful to add indexes to tenant_id, action, and request_id
  • Using JSON for detail is flexible, but values you often search on should be promoted to normal columns for efficiency

10. Create a repository for saving audit logs

# app/audit/repository.py
from sqlalchemy.orm import Session
from app.audit.models import AuditLog
from app.audit.schemas import AuditEvent

class AuditLogRepository:
    def __init__(self, db: Session):
        self.db = db

    def create(self, event: AuditEvent) -> AuditLog:
        row = AuditLog(
            timestamp=event.timestamp,
            actor_type=event.actor_type,
            actor_id=event.actor_id,
            tenant_id=event.tenant_id,
            action=event.action,
            resource_type=event.resource_type,
            resource_id=event.resource_id,
            result=event.result,
            request_id=event.request_id,
            ip_address=event.ip_address,
            user_agent=event.user_agent,
            detail=event.detail,
        )
        self.db.add(row)
        self.db.flush()
        return row

At this point, the service layer has what it needs to persist audit events into the database.


11. Gather request information: create a context helper

It is cumbersome to retrieve request_id and ip_address directly in every service method, so it helps to create a shared context object.

# app/audit/context.py
from pydantic import BaseModel

class AuditContext(BaseModel):
    request_id: str | None = None
    ip_address: str | None = None
    user_agent: str | None = None
    actor_type: str = "user"
    actor_id: str | None = None
    tenant_id: int | None = None

Build it with a FastAPI dependency.

# app/deps/audit.py
from fastapi import Depends, Request
from app.audit.context import AuditContext
from app.deps.auth import get_current_user
from app.deps.tenant import get_current_tenant_id

def get_audit_context(
    request: Request,
    current_user=Depends(get_current_user),
    tenant_id: int = Depends(get_current_tenant_id),
) -> AuditContext:
    return AuditContext(
        request_id=request.headers.get("X-Request-ID"),
        ip_address=request.client.host if request.client else None,
        user_agent=request.headers.get("User-Agent"),
        actor_type="user",
        actor_id=str(current_user.user_id),
        tenant_id=tenant_id,
    )

12. Record audit logs in the service layer

This is the core part.
Record the event when an important operation succeeds, and when needed, also when it fails.

# app/services/project_service.py
from datetime import datetime, timezone
from sqlalchemy.orm import Session

from app.audit.context import AuditContext
from app.audit.schemas import AuditEvent
from app.audit.repository import AuditLogRepository
from app.models.project import Project

class ProjectService:
    def __init__(self, db: Session):
        self.db = db
        self.audit_repo = AuditLogRepository(db)

    def create_project(self, tenant_id: int, name: str, audit_ctx: AuditContext) -> Project:
        project = Project(tenant_id=tenant_id, name=name)
        self.db.add(project)
        self.db.flush()

        event = AuditEvent(
            timestamp=datetime.now(timezone.utc),
            actor_type=audit_ctx.actor_type,
            actor_id=audit_ctx.actor_id,
            tenant_id=audit_ctx.tenant_id,
            action="project.create",
            resource_type="project",
            resource_id=str(project.id),
            result="success",
            request_id=audit_ctx.request_id,
            ip_address=audit_ctx.ip_address,
            user_agent=audit_ctx.user_agent,
            detail={"name": name},
        )
        self.audit_repo.create(event)

        return project

When you record events at a place that is meaningful in business terms, the logs become much more valuable later.


13. Should failures also be recorded?

In short, important failures are worth recording.

For example:

  • A permission change was rejected
  • Cross-tenant access was blocked
  • API key revocation failed
  • A billing plan change failed due to an external payment error

If you only record successful operations, you cannot see suspicious attempts.
On the other hand, recording every validation error as an audit event can create too much noise.

In practice, it is reasonable to divide them like this:

  • Ordinary input mistakes → ordinary logs
  • Failures related to security or permissions → audit logs
  • Failures in important business operations → audit logs

14. Recording differences: how to store what changed

For update operations, you may want to keep the “before” and “after” difference.

For example, for a role change:

  • before: member
  • after: admin

It is useful to keep a diff like this.

detail={
    "before": {"role": "member"},
    "after": {"role": "admin"},
}

However, if you store too much difference data, you risk putting personal or confidential data into the audit log.
So you should choose carefully what fields are included.

A practical policy is:

  • Never store passwords, tokens, or secrets
  • Record only the necessary fields in a diff
  • Instead of storing full records, keep IDs and key fields only

15. Audit logs in a multi-tenant system

Following the previous article, tenant_id becomes especially important in multi-tenant environments.

Minimum things to protect

  • Include tenant_id in every audit event
  • If an operation crosses tenants, record that fact in detail
  • In search screens and admin UIs, prevent logs from other tenants from being mixed in

For example, even if internal administrators are allowed to view audit logs across all tenants, ordinary tenant administrators should only see the logs belonging to their own tenant.


16. Where to store audit logs: database, log platform, or both?

There are roughly three patterns for storing audit logs.

16.1 Database only

  • Easy to search
  • Easy to build an admin UI on top of
  • But if stored in the same application DB, the table can grow very large

16.2 Log platform only

  • Fits well with existing centralized logging
  • Easy to operate
  • But strict searching and retention control depend on the platform

16.3 Database + log platform

  • The most practical in many real systems
  • Store key searchable fields in the database
  • Send detailed context to the log platform as well

If you are unsure, start with DB storage in a small system, then move to DB + log platform duplication as operations mature.


17. Protecting against deletion and tampering

If audit logs can be deleted later, their value decreases.

Practical measures include:

  • Do not create an API for deleting audit logs
  • Do not give ordinary administrators permission to update or delete them
  • As a rule, forbid UPDATE on the audit table
  • If long-term retention is needed, also send them to an external platform
  • Separate backups and archives from ordinary data

You do not need perfect tamper resistance on day one, but you do want to avoid a state where audit logs can be edited just like ordinary application data.


18. Designing an API to view audit logs

If you want to show audit logs in an admin panel, you will need a list API.

Recommended filters

  • tenant_id
  • actor_id
  • action
  • resource_type
  • resource_id
  • datetime range such as from, to

Example

@router.get("/audit-logs")
def list_audit_logs(
    tenant_id: int = Depends(get_current_tenant_id),
    action: str | None = None,
):
    ...

Here too, just like with ordinary business data, it is essential to strictly control who is allowed to see what.
Audit logs themselves can become sensitive information, so read permissions should be designed very carefully.


19. Testing points: minimum things you should protect

For audit logs, you need to verify not only that they “work,” but also that the events that should be recorded are not missing.

Minimum tests

  • An important action creates one audit log entry
  • tenant_id is recorded correctly
  • actor_id is recorded correctly
  • Other tenants’ audit logs cannot be retrieved
  • Important failure events are recorded
  • Sensitive information is not included in detail

Example test

def test_project_create_writes_audit_log(client, db_session, auth_headers):
    res = client.post("/projects", json={"name": "New Project"}, headers=auth_headers)
    assert res.status_code == 201

    rows = db_session.execute("SELECT action, result FROM audit_logs").fetchall()
    assert len(rows) == 1
    assert rows[0][0] == "project.create"
    assert rows[0][1] == "success"

20. Adoption roadmap

Individual developers and learners

  1. Decide on three important operations first
  2. Create the AuditEvent model
  3. Output to structured logs
  4. Once comfortable, move to a dedicated audit table

Engineers on small teams

  1. Create a list of audit target events
  2. Decide on naming rules for event names and fields
  3. Build a shared function for recording from the service layer
  4. Make tenant_id and request_id mandatory fields
  5. Build a list API or admin UI

SaaS teams and startups

  1. Inventory necessary events from legal, customer support, and security perspectives
  2. Design both DB storage and log-platform shipping
  3. Decide who can view audit logs and how long they are retained
  4. Strengthen cross-tenant access prevention and tamper resistance
  5. Define which audit events should also trigger alerts

Reference links


Conclusion

  • Audit logs are not just error logs. They are a mechanism for preserving evidence of important operations.
  • In FastAPI, a practical design is to collect common information in middleware, while recording the actual audit events in the service layer.
  • The first thing to decide is not the storage location, but which events to record, and in what format.
  • In multi-tenant systems, you should always include tenant_id and design viewing permissions strictly.
  • Instead of aiming for perfection from the start, it is easier to begin with important actions such as role changes, deletions, and downloads.

As a natural next topic, this flows well into things like RBAC/ABAC authorization in FastAPI or SaaS plan, billing, and feature restriction design.

Salir de la versión móvil