Feat: created annotation handler #38

jimmyjam100 · 2025-12-09T14:58:10Z

Created an annotation handler that automatically emulates .trk files as precomputed annotation folder

tracts will be loaded based on this flowchart

flow chart for how to handle tracts:
format is trk -> use handler
file to be loaded ends in .trk, .tck or .tracts and format is empty or precomputed and layer is not tractsv1 -> use handler
layer is tracts or tractsv2 -> load premade precomputed annotation with tract styles
layer is tractsv1 -> load tracts as skeleton
format is precomputed and type is line annotations -> load premade precomputed annotation with tract styles
format is precomputed -> load premade precomputed format with no styles (not implemented yet)
otherwise -> assume its not tracts

balbasty · 2025-12-09T15:17:46Z

ngtools/local/handlers.py

 LOG = logging.getLogger(__name__)


+class AnnotationHandler(Handler):


I would call it TractAnnotationHandler

balbasty · 2025-12-09T15:18:16Z

ngtools/local/handlers.py

+    """Handler that returns annotation data."""
+
+    annotation_cache = {}
+    spatical_cache = {}


spatical -> spatial
(please be careful with typos, especially in code)

balbasty · 2025-12-09T15:21:23Z

ngtools/local/handlers.py

+class AnnotationHandler(Handler):
+    """Handler that returns annotation data."""
+
+    annotation_cache = {}


We need to be careful with caches.

It seems that you never clear it. It might be better to use a cache that has a size limit (you can maybe reuse or copy this one, although we might need something a bit more advanced that puts a file back on top of the queue when used).

We should also ask Scene.unload to remove files from cache when possible.

balbasty · 2025-12-09T15:21:39Z

ngtools/local/handlers.py

+    spatical_cache = {}
+    grid_densities = [1, 2, 4, 8]
+
+    def read_from_file(self, path: str) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:


Make it private: _read_from_file

balbasty · 2025-12-09T15:22:27Z

ngtools/local/handlers.py

+            self.headers["Content-type"] = "application/json"
+            self.body = json.dumps(info).encode()
+            return None
+        if self.parse_path(path) is not None:


Separate the cases with an empty line, for readability.

balbasty · 2025-12-09T15:24:24Z

ngtools/local/handlers.py

+        else:
+            return None
+
+    def get_spaticals(self, path: str) -> bytes:


typo spaticals -> spatials
Also, make private: _get_spatials

balbasty · 2025-12-09T15:26:03Z

ngtools/local/handlers.py

+        self.spatical_cache[trk_path] = spatial
+        return spatial[int(parent)][f"{a}_{b}_{c}"]
+
+    def get_info(self, path: str) -> dict:


Make private: _get_info

balbasty · 2025-12-09T15:27:57Z

ngtools/local/handlers.py

+            raise FileNotFoundError(
+                f"Tract file not found: {trk_path}")
+
+        segments, bbox, offsets, affine = self.read_from_file(trk_path)


It seems like you read the entire file every time info is queried.

It's probably worth caching it as well.

The file is cached, inside that function it caches it. unless you are saying I should cache info as well which is not a bad idea

I meant that if the info file is queried twice, the trk file is going to be read again, since caching happens later in the function, no? But yes I guess it's relatd with caching the info file or not. (I am not so worried about caching the info file itself as I am with caching the trk-dependent quantities that are computed to create the info file)

The end of self._read_from_file caches the segments, bbox, offsets, and affine before returning into annotation_cache.

balbasty · 2025-12-09T15:29:58Z

ngtools/scene.py

                        "Cannot load local files without a fileserver"
                    )
-                short_uri = fileserver + "/local/" + op.abspath(short_uri)
+                if parsed.format == "precomputed" and short_uri.endswith(".trk"):


I would actually not look for the precomputed:// hint. If the file is a trk -> use the handler.

And we might also want to include a trk:// hint for trk files that do not have the trk extension.

balbasty · 2025-12-09T15:30:30Z

ngtools/local/viewer.py

                (r"^/local/(.*)", StaticFileHandler),
                (r"^/lut/([^/]+)/(.*)", LutHandler),
                (r"^/linc/(.*)", LincHandler),
+                (r"^/ann/(.*)", AnnotationHandler),


I would use the /trk/ prefix

Also, we might want to accept a protocol prefix, like in the LutHandler, in case the trk file is remote instead of local. This allows path of the form https://some/url/to/file.trk to be mapped to {fileserver}/trk/https/some/url/to/file.trk, while local files are mapped to {fileserver}/trk/file/some/path/to/file.trk.

balbasty · 2025-12-10T18:59:00Z

I only have an issue with this option in the flowchart:

format is precomputed and type is line annotations -> load premade precomputed annotation with tract styles

I'd prefer we only use the tracts style if we are confident the file really contains tracts data. Maybe we could look for the presence of the "orientation" property?

balbasty

Almost there!

balbasty · 2025-12-10T19:00:02Z

ngtools/local/handlers.py

+import trk_to_annotation.utils as tau

 # internals
+from ngtools.datasources import _SizedRefreshCache


Can you move the cache classes to ngtools.utils, now that they are used in multiple modules?

balbasty · 2025-12-10T19:00:45Z

ngtools/local/handlers.py

+
+        try:
+            folder_listing = {"path": path,
+                              "children": ["info", "0", "1", "2"]}


Should the list of level be read from the info file?

This is for the benefit of neuroglancer to make the .trk file look like a folder. I think it was needed, but I implemented this super early when I was still testing on actual neuroglancer instead of within ngtools so it might not be needed for ngtools

I will double check

not needed. Removing now

balbasty · 2025-12-10T19:02:00Z

ngtools/local/handlers.py

+        if path in annotation_cache:
+            return annotation_cache[path]
+        if os.path.exists(path):
+            segments, bbox, offsets, affine = tap.load_from_file(path)


I feel like we don't really need this case, we can always use the filesystem opener (it work with local files).

balbasty · 2025-12-10T19:07:06Z

ngtools/scene.py

+            del spatial_cache[key]
+        for key in annotation_cache.keys():
+            del annotation_cache[key]
+


We should only remove cached values that are used in this unloaded layer I think.

Thinking more about it: maybe check whether the cached data is used in any remaining layer, and only delete it if not (in case multiple layers use the same datasource).

ah my bad. I don't know why but I assumed unload was to unload everything. That makes more sense though

balbasty · 2025-12-10T19:08:38Z

ngtools/scene.py

                    )
-                short_uri = fileserver + "/local/" + op.abspath(short_uri)
+                if parsed.format == "trk":
+                    short_uri = fileserver + "/trk/local/" + op.abspath(


Have you tried this case? I am thinking it should be /trk/file/ instead of /trk/local/ (unless fsspec understands local as an alias for file)

I have tested this, but I just had a hard coded check to ignore local protocol. I forgot that file protocol was a thing

balbasty · 2025-12-10T19:11:58Z

ngtools/local/handlers.py

+
+        return info
+
+    def _get_transform(self, path: str) -> np.ndarray:


Is this used anywhere?

yes on line 59 of this file

…ales/ngtools into james/annotation_handler

James Jay Scherick added 2 commits December 9, 2025 09:55

created annotation handler

fb13ad9

updated tractsv2 to be the default

e0b6492

balbasty reviewed Dec 9, 2025

View reviewed changes

James Jay Scherick added 2 commits December 9, 2025 16:13

Merge branch 'main' into james/annotation_handler

e8f8a0b

made requested changes

0a3f6de

balbasty reviewed Dec 10, 2025

View reviewed changes

James Jay Scherick and others added 6 commits December 10, 2025 16:05

requested changes

308779b

fixed function return indicator

b5dca2d

Merge branch 'main' into james/annotation_handler

0b3480d

added requirements

7ffcb8d

Merge branch 'james/annotation_handler' of https://github.com/neurosc…

cf1f840

…ales/ngtools into james/annotation_handler

fixed dependencies

210090b

		LOG = logging.getLogger(__name__)


		class AnnotationHandler(Handler):


		return info

		def _get_transform(self, path: str) -> np.ndarray:

Feat: created annotation handler #38

Are you sure you want to change the base?

Feat: created annotation handler #38

Uh oh!

Conversation

jimmyjam100 commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

balbasty commented Dec 10, 2025

Uh oh!

balbasty left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jimmyjam100 commented Dec 9, 2025 •

edited

Loading