Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 27 additions & 23 deletions nri-observability/scripts/memory-leak-test/Main.hs
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
{-# LANGUAGE OverloadedRecordDot, OverloadedRecordUpdate #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedRecordDot #-}
{-# LANGUAGE OverloadedRecordUpdate #-}
{-# LANGUAGE OverloadedStrings #-}

module Main where


import Control.Monad (void, sequence, forM_)
import Control.Concurrent.Async (mapConcurrently_)
import Control.Concurrent (threadDelay)
import qualified Conduit
import Control.Concurrent (threadDelay)
import Control.Concurrent.Async (mapConcurrently_)
import Control.Monad (forM_, sequence, void)
import Data.List (splitAt)
import qualified Environment
import GHC.Conc (numCapabilities)
import qualified Observability
import qualified Process
import qualified Platform
import Data.List (splitAt)
import Prelude (IO, show, putStrLn, fromIntegral, pure, mapM_, div)
import GHC.Conc (numCapabilities)
import qualified Process
import Prelude (IO, div, fromIntegral, mapM_, pure, putStrLn, show)

threads :: Int
threads = fromIntegral numCapabilities
Expand All @@ -24,27 +24,31 @@ main :: IO ()
main = do
settings' <- Environment.decode Observability.decoder
putStrLn (show settings'.enabledReporters)
let ids = [1..12] |> List.map Text.fromInt
let ids = [1 .. 12] |> List.map Text.fromInt
Conduit.withAcquire (Observability.handler settings') <| \handler -> do
forM_ [1..(floor (1_000_000 / fromIntegral threads))] <| \n -> do
forM_ [1 .. (floor (1_000_000 / fromIntegral threads))] <| \n -> do
runRequests handler ids

runRequests :: Observability.Handler -> [Text] -> IO ()
runRequests handler =
mapConcurrently_ (\requestId -> do
Platform.rootTracingSpanIO
requestId
(handler.report requestId)
("Running task" ++ requestId)
( \log -> do
Task.perform log (do
Process.sleep 5
Task.succeed ())
mapConcurrently_
( \requestId -> do
Platform.rootTracingSpanIO
requestId
(handler.report requestId)
("Running task" ++ requestId)
( \log -> do
Task.perform
log
( do
Process.sleep 5
Task.succeed ()
)
)
)

chunks :: Int -> [a] -> [[a]]
chunks _ [] = []
chunks n xs =
let (ys, zs) = splitAt (fromIntegral n) xs
in ys : chunks n zs
let (ys, zs) = splitAt (fromIntegral n) xs
in ys : chunks n zs
3 changes: 2 additions & 1 deletion nri-observability/tests/Spec/Reporter/File.hs
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,8 @@ logTest name span =
Ok reEncoded ->
Text.join "\n" reEncoded
|> Expect.equalToContentsOf
( goldenResultsDir ++ "/file-reporter-"
( goldenResultsDir
++ "/file-reporter-"
++ Text.replace " " "-" name
)

Expand Down
2 changes: 1 addition & 1 deletion nri-prelude/src/Platform.hs
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ requestId = map Internal.requestId logHandler

-- | A log handler that doesn't log anything.
silentHandler :: IO Internal.LogHandler
silentHandler = Internal.mkHandler "" (Internal.Clock (pure 0)) (\_ -> pure ()) Nothing ""
silentHandler = pure Internal.nullHandler

-- | Throw a runtime exception that cannot be caught. This function, like
-- @Debug.todo@, breaks type level guarantees and should be avoided. Where
Expand Down
19 changes: 19 additions & 0 deletions nri-prelude/src/Platform/Internal.hs
Original file line number Diff line number Diff line change
Expand Up @@ -616,6 +616,25 @@ mkHandler requestId clock onFinish onFinishRoot' name' = do
finishTracingSpan = finalizeTracingSpan clock allocationCounterStartVal tracingSpanRef >> andThen onFinish
}

-- | Helper that creates a handler that does nothing. This is intended to power
-- basically @Platform.silentHandler@ and nothing else. We provide this to make
-- @Platform.silentHandler@ as efficient as possible, skipping all side effects.
Comment on lines +619 to +621
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to mention memory leaks specifically.
Engineers often disagree on the importance of efficiency, whether certain special cases are justified, etc.
But a memory leak is pretty objective. And if someone changes this code, they need to watch out for that.

Copy link
Member Author

@omnibs omnibs Sep 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I didn't mention memory leaks because I'm not really sure the leaks are caused by mkHandler. We use mkHandler all the time with other LogHandler instances. What's special about silentHandler? Remember we didn't see leakage in high-throughput apps like HQE before #134? Every request in HQE uses mkHandler.

I think it might be more of an issue of how the handler is being used, than the handler itself.

There's this old article by Edsko de Vries on Conduit and space leaks, which I don't really follow, that reinforces the suspicious there's something about our usage that makes this leak. It seems it's not exactly what Edsko describes tho, because -fno-full-laziness didn't help.

This is also why I was more keen on removing the unused silentHandler logger in our Amazonka wrapper, than fixing it to actually log things to the request's tracing span. If I knew what's really causing the leak and how to fix it, I'd rather have logs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although I could leave a word of warning wrt this whole situation. Someone might be grateful.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah exactly 👍

--
-- The underlying desire for an IO-free `silentHandler`, aside from principles,
-- is we saw space leaks carrying @TracingSpan@ and @TracingSpanDetails@ we
-- couldn't understand, which went away when we switched to this no-op handler.
nullHandler :: LogHandler
nullHandler = do
LogHandler
{ requestId = "",
startChildTracingSpan = \_ -> pure nullHandler,
startNewRoot = \_ -> pure nullHandler,
setTracingSpanDetailsIO = \_ -> pure (),
setTracingSpanSummaryIO = \_ -> pure (),
markTracingSpanFailedIO = pure (),
finishTracingSpan = \_ -> pure ()
}

-- | Set the details for a tracingSpan created using the @tracingSpan@
-- function. Like @tracingSpan@ this is intended for use in writing libraries
-- that define custom types of effects, such as database queries or http
Expand Down