Persisting wallaby sessions

[hhubert6]

Elixir and Erlang/OTP versions

Erlang/OTP 27 [erts-15.1.2] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

Elixir 1.18.3 (compiled with Erlang/OTP 27)

Operating system

macOS 15.0.1

Browser

Chrome 135

Driver

ChromeDriver

Correct Configuration

• I confirm that I have Wallaby configured correctly.

Current behavior

When interrupting tests with Ctrl-C the browser sessions seem to persist to the next test run.

It can be seen on this branch after running
mix e2e
or specifically
mix e2e test/live_debugger/live_views_dashboard_test.exs.

The test starts two applications, one of which is used to debug the other. Debugger application gets all live view processes of the other application and lists them on a page.

Code of the failing test:

  @sessions 3
  feature "user can see active live views and refresh to see more", %{
    sessions: [dev_session1, dev_session2, lvdbg_session]
  } do
    dev_session1
    |> visit(@dev_app_url)

    lvdbg_session
    |> visit("/")
    |> assert_has(title(text: "Active LiveViews"))
    |> assert_has(live_sessions(count: 1))

    dev_session2
    |> visit(@dev_app_url)

    lvdbg_session
    |> assert_has(live_sessions(count: 1))
    |> click(button("refresh"))
    |> assert_has(live_sessions(count: 2))
  end

  defp title(text: text),
    do: css("h1", text: text)

  defp live_sessions(count: count),
    do: css("#live-sessions > div", count: count)

The test aims to open one session of dev_app and after that the main page of debugger with list of all active live views. On the debugger active live views page should be seen one element which happens in running tests. The problem occurs when the test is interrupted with Ctrl-C. After that, one session seem to persist over next test runs and then test fails with:

  1) feature user can see active live views and refresh to see more (LiveDebugger.LiveViewsDashboardTest)
     test/live_debugger/live_views_dashboard_test.exs:12
     ** (Wallaby.ExpectationNotMetError) Expected to find 1 visible element that matched the css '#live-sessions > div', but 2 visible elements were found.
     
     code: |> assert_has(live_sessions(count: 1))
     stacktrace:
       test/live_debugger/live_views_dashboard_test.exs:21: (test)

From now on there is an additional session that is not created directly in the test but can be seen by the debugger.

For now the only method I found to reset this behavior is to shutdown and restart the operating system.

Steps to reproduce

• clone repository from this branch
• run mix setup
• run mix e2e and press Ctrl-C during running tests
• run mix e2e test/live_debugger/live_views_dashboard_test.exs
  (try a few times, it not always shows up)

cc @kraleppa

Expected behavior

There should not be any persisting sessions after interrupting running tests.

Test Code & HTML

  @sessions 3
  feature "user can see active live views and refresh to see more", %{
    sessions: [dev_session1, dev_session2, lvdbg_session]
  } do
    dev_session1
    |> visit(@dev_app_url)

    lvdbg_session
    |> visit("/")
    |> assert_has(title(text: "Active LiveViews"))
    |> assert_has(live_sessions(count: 1))

    dev_session2
    |> visit(@dev_app_url)

    lvdbg_session
    |> assert_has(live_sessions(count: 1))
    |> click(button("refresh"))
    |> assert_has(live_sessions(count: 2))
  end

  defp title(text: text),
    do: css("h1", text: text)

  defp live_sessions(count: count),
    do: css("#live-sessions > div", count: count)

Demonstration Project

https://github.com/software-mansion/live-debugger/tree/223-introduce-wallaby-based-testing

[mhanberg]

I'll clone and try to reproduce