Skip to content

BUG: checkpoint scheduler messes with logging #4753

@stephankramer

Description

@stephankramer

Consider the following MFE:

from firedrake import *
from firedrake.adjoint import *
from checkpoint_schedules import SingleMemoryStorageSchedule

set_log_level(INFO)
continue_annotation()
tape = get_working_tape()
tape.enable_checkpointing(
    SingleMemoryStorageSchedule(),
    gc_timestep_frequency=1,
    gc_generation=2
)
trange = iter(range(10))
adj_timestepper = tape.timestepper(trange)

for i in adj_timestepper:
    info(f"Timestep {i}")

When run in serial, the info messages are duplicated:

firedrake:INFO Timestep 0
INFO:firedrake:Timestep 0
firedrake:INFO Timestep 1
INFO:firedrake:Timestep 1
firedrake:INFO Timestep 2
INFO:firedrake:Timestep 2
firedrake:INFO Timestep 3
INFO:firedrake:Timestep 3
firedrake:INFO Timestep 4
INFO:firedrake:Timestep 4
firedrake:INFO Timestep 5
INFO:firedrake:Timestep 5
firedrake:INFO Timestep 6
INFO:firedrake:Timestep 6
firedrake:INFO Timestep 7
INFO:firedrake:Timestep 7
firedrake:INFO Timestep 8
INFO:firedrake:Timestep 8
firedrake:INFO Timestep 9
INFO:firedrake:Timestep 9

whereas if you comment out the enable_checkpointing() line, it only prints the expected firedrake:INFO Timestep ... line. When run in serial this gets worse because it now also duplicates the additional lines (not prefixed with "firedrake:" for every process.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions