codexctl install causes swupdate errors

[StarNumber12046]

The command python main.py install 3.22.4.2 (probably also an issue with the binary build of codexctl) fails on my rM2 with the following

Success: Connected to device
Version 3.22.4.2 not found. Attempting to download
[==================================================]
Downloaded version 3.22.4.2 to .//remarkable-production-memfault-image-3.22.4.2-rm2-public
Uploading .//remarkable-production-memfault-image-3.22.4.2-rm2-public image
Transferring progress100%
Done! Running swupdate (PLEASE BE PATIENT, ~5 MINUTES)
[ERROR] : SWUPDATE failed [0] ERROR : mount: /mnt: WARNING: source write-protected, mounted read-only.
[ERROR] : SWUPDATE failed [0] ERROR : Failed to locate /usr/lib/ld-linux-armhf.so.3 in /mnt
[ERROR] : SWUPDATE failed [0] ERROR : execute postinstall scripts failed
[ERROR] : SWUPDATE failed [1] Installation failed !

Traceback (most recent call last):
  File "C:\Users\StarNumber\codexctl\main.py", line 13, in <module>
    main()
    ~~~~^^
  File "C:\Users\StarNumber\codexctl\codexctl\__init__.py", line 642, in main
    man.call_func(args.command, vars(args))
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\StarNumber\codexctl\codexctl\__init__.py", line 440, in call_func
    remarkable.install_sw_update(update_file, bootloader_files=bootloader_files_for_install)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\StarNumber\codexctl\codexctl\device.py", line 764, in install_sw_update
    raise SystemError("Update failed!")
SystemError: Update failed!

The same issue arises with swupdate $SWUPDATE_ARGS -i /tmp/[name].swu, however, swupdate-from-image-file succeeds

[Eeems]

It's failing to run the post install scripts, so the update was mostly applied and all the files should be in place. Part of the post install is to use the libraries from the updated partition to run the scripts, and it's failing to find one for some reason. If you can replicate this, it would be good to check to see the state of the upgraded partition to make sure the expected files are there, and if the post install script has a bug or not.

[ilbonte]

Hi @Eeems i am having the same issue.

I was using version 3.11.3.3, updated to 3.24.0.149 from the remarkable itself and i was trying to downgrade to 3.22.4.2 to install rm-hacks-qml

> codexctl --address 10.11.99.1 --password ~/.ssh/id_ed25519 status

Success: Connected to device

Current version: 3.24.0.149
Backup version: 3.11.3.3
Old update engine: True
Beta active: Release
Version id: 20251130114409

> codexctl --address 10.11.99.1 --password ~/.ssh/id_ed25519 install 3.22.4.2
Success: Connected to device
Version 3.22.4.2 not found. Attempting to download
[==================================================]
Downloaded version 3.22.4.2 to .//remarkable-production-memfault-image-3.22.4.2-rm2-public
Uploading .//remarkable-production-memfault-image-3.22.4.2-rm2-public image
Transferring progress100%
Done! Running swupdate (PLEASE BE PATIENT, ~5 MINUTES)

[ERROR] : SWUPDATE failed [0] ERROR : mount: /mnt: WARNING: source write-protected, mounted read-only.
[ERROR] : SWUPDATE failed [0] ERROR : Failed to locate /usr/lib/ld-linux-armhf.so.3 in /mnt
[ERROR] : SWUPDATE failed [0] ERROR : execute postinstall scripts failed
[ERROR] : SWUPDATE failed [1] Installation failed !

Traceback (most recent call last):
  File "/private/var/folders/hk/1h7862r54yqczjz6s00xmtch0000gn/T/onefile_16397_1767352510_62777/main.py", line 13, in <module>
  File "/private/var/folders/hk/1h7862r54yqczjz6s00xmtch0000gn/T/onefile_16397_1767352510_62777/codexctl/__init__.py", line 642, in main
  File "/private/var/folders/hk/1h7862r54yqczjz6s00xmtch0000gn/T/onefile_16397_1767352510_62777/codexctl/__init__.py", line 440, in call_func
  File "/private/var/folders/hk/1h7862r54yqczjz6s00xmtch0000gn/T/onefile_16397_1767352510_62777/codexctl/device.py", line 764, in install_sw_update
SystemError: Update failed!

any suggestion?

UPDATE: some more info:

cat /proc/cmdline | tr ' ' '\n' | grep '^root='
root=/dev/mmcblk2p2

ls -l /dev/mmcblk2p*
brw-rw----    1 root     disk      179,   1 Jan  2 11:11 /dev/mmcblk2p1
brw-rw----    1 root     disk      179,   2 Jan  2 11:19 /dev/mmcblk2p2
brw-rw----    1 root     disk      179,   3 Jan  2 11:19 /dev/mmcblk2p3
brw-rw----    1 root     disk      179,   4 Jan  2 11:11 /dev/mmcblk2p4

[Eeems]

Could you mount /dev/mmcblk2p3 to /mnt and see if /mnt/usr/lib/ld-linux-armhf.so.3 exists?

[ilbonte]

Sure!

reMarkable: ~/ umount /mnt 2>/dev/null || true
reMarkable: ~/ mount -o ro /dev/mmcblk2p3 /mnt

reMarkable: ~/ ls -l /mnt/usr/lib/ld-linux-armhf.so.3
ls: /mnt/usr/lib/ld-linux-armhf.so.3: No such file or directory

reMarkable: ~/ ls -l /mnt/usr/lib | grep ld-linux || true
ls: /mnt/usr/lib: No such file or directory

reMarkable: ~/ find /mnt/usr/lib -maxdepth 3 -name 'ld-linux-armhf.so.3' -o -name 'ld-linux*.so*'
find: /mnt/usr/lib: No such file or directory

reMarkable: /mnt/ cd /mnt
reMarkable: /mnt/ ls
ls: ./postinst: Bad message
ls: ./proc: Bad message
ls: ./etc: Bad message
ls: ./bin: Bad message
ls: ./tmp: Bad message
ls: ./dev: Bad message
^[[Fls: ./postinst-waveform: Bad message
ls: ./var: Bad message
ls: ./mnt: Bad message
ls: ./uboot-version: Bad message
ls: ./home: Bad message
ls: ./boot: Bad message
ls: ./lost+found: Bad message
ls: ./run: Bad message
ls: ./media: Bad message
ls: ./lib: Bad message
ls: ./sbin: Bad message
ls: ./srv: Bad message
sys  usr

[Eeems]

Well that's very odd. Why would the root user not have access to anything but the sys and usr directories? What does ls /mnt/usr return?

[ilbonte]

/mnt/usr is empty 🤷
This is quite a vanilla remarkable... aside ddvk's hacks and rm-hacks I never modded anything :)

[Eeems]

This doesn't seem right, swupdate doesn't seem to have unpacked the image properly. Could you extract the update image (codexctl extract) and check the sha256sum of the update image file that it produces and /dev/mmcblk2p3 to see if they are different?

[ilbonte]

Well... I rebooted the remarkable and tried again with download + install and it worked without any issue! I should have tried it from the start 😅

[Eeems]

That's very odd, as the update process should validate the file before installing it, so it's not clear to me why downloading a second time would work.