Implementation of publishing vdb product #49

moonyuet · 2025-05-30T09:45:48Z

Changelog Description

This PR is to implement the publishing aspect for vdb product type. It is only supported for the vdb publish from TyFlow.

Solve #48
TODO List:

Refactor instance.data["operator"]
Refactor on the product name based on templated profile product name like we did in Houdini

Additional review information

As vdb publishes can be supported by multiple external party plugins, so some of the codes are structured to make the implementation of vdb published from other parties easier to be integrated in the future.
Make sure you included Export VDB operators in the tyflow editor, otherwises nothing shows up for publishing in the insrance container
Users need to add vdb as product type into the ayon+settings://core/tools/publish/template_name_profiles/6/product_types

Testing notes:

Create VDB (TyFlow)
Publish

moonyuet · 2025-06-03T05:38:37Z

@antirotor guess the representation trait you implemented in core addon would be super helpful for the implementation of this product type as there should be different templates used for publishing if we have some different vdb extractors (other than tyflow one) included in the future.

BigRoy · 2025-06-03T06:51:31Z

guess the representation trait you implemented in core addon would be super helpful for the implementation of this product type as there should be different templates used for publishing if we have some different vdb extractors (other than tyflow one) included in the future.

Why would it need different templates?

BigRoy · 2025-06-03T06:52:46Z

client/ayon_max/plugins/publish/collect_tyflow_vdb.py

+                # make sure the tyflow vdb extractor would not be triggered
+                # when the non-tyflow vdb workflow is adopted by the user
+                # in the future
+                "is_tyflow": True,


If that ever happens, that would be a different product type? Because this one is tyflow_vdb? It just feels like we're overcomplicating code here for a non-existing use case yet (since there isn't any other VDB extractor yet?)

If that ever happens, that would be a different product type? Because this one is tyflow_vdb? It just feels like we're overcomplicating code here for a non-existing use case yet (since there isn't any other VDB extractor yet?)

I am thinking the approach to be less complicated. It's just that the tyflow vdb export(select the target exporter node from the tyflow editor) is totally different from the general vdb export(select the target asset from the max editor).
I am thinking if it is possible to just implement the vdb product type in the creator but with quite specific creator identifier(as if what we did in Houdini for abc/bgeo pointcache).

We recently did something similar with 'render' creators in Houdini. Those are now all render product type, regardless of renderer. It's this one: ynput/ayon-houdini#214

BigRoy · 2025-06-03T06:54:10Z

client/ayon_max/plugins/create/create_tyflow.py

+    """Creator plugin for TyFlow VDB."""
+    identifier = "io.ayon.creators.max.vdb"
+    label = "VDB (TyFlow)"
+    product_type = "tyflow_vdb"


Note that a lot of VDB based loaders currently target the vdbcache product type, and hence will not allow loading this product type.

For example, these all target vdbcache product type.

ayon_maya/plugins/load/load_vdb_to_vray.py ayon_maya/plugins/load/load_vdb_to_redshift.py ayon_maya/plugins/load/load_vdb_to_arnold.py ayon_houdini/plugins/load/load_vdb.py

BigRoy · 2025-06-03T06:55:33Z

client/ayon_max/plugins/publish/collect_tyflow_vdb.py

+                "productName": f"{container_name}_{prod_name}",
+                # get the name of operator for the export
+                "operator": operator,
+                "productType": "vdb",


Pretty sure it's vdbcache in other hosts.

BigRoy · 2025-06-03T06:56:29Z

client/ayon_max/plugins/publish/collect_tyflow_vdb.py

+                "label": f"{container_name}_{prod_name}",
+                "family": "vdb",
+                "families": ["vdb"],
+                "productName": f"{container_name}_{prod_name}",


Please do not override productName directly - this should really be targeting vdbcache product type from the get go... from the creator itself so that the product naming templates apply accordingly. @iLLiCiTiT thoughts?

Please do not override productName directly - this should really be targeting vdbcache product type from the get go... from the creator itself so that the product naming templates apply accordingly. @iLLiCiTiT thoughts?

if that's the case, we need to also change the similar variable to not override productName in tycache collector.

Don't know full flow, usually when this is needed then the product name template contains some key that is kept unfilled, e.g. {some_key} then create plugin returns the key value in get_dynamic_data as unfilled.

def get_dynamic_data(self, *args, **kwargs): return { "some_key": "{some_key}", }

With that you can fill the product name during collection phase.

iLLiCiTiT · 2025-06-09T07:39:17Z

client/ayon_max/api/plugin.py

+        elif attribute == "ty_vdb_data":
+            node.modifiers[0].name = "AYON TyFlow VDB Data"
+        else:
+            node.modifiers[0].name = "OP Data"


Why "OP Data"?

It's the name of general container attributes for other product type. It would be renamed after the related PR for this issue as the backward compatibility needs to be implemented for this.

iLLiCiTiT · 2025-06-09T07:49:25Z

client/ayon_max/plugins/publish/collect_tyflow_vdb.py

+                # in the future
+                "is_tyflow": True,
+                "publish_attributes": {
+                    "ValidateTyVDBFrameRange": {


This looks like out of the scope. Is the ValidateTyVDBFrameRange plugin ever showed in publisher UI? Because it looks like it is not, and you're using the "optional" plugin only to enabled/disable validation of instances based on "has_frame_range_validator".

If that's the true, then don't use Optional plugin, just add something like families = ["vdb_frame_validation"] filter there and add the family "vdb_frame_validation" to the instance (the family vdb_frame_validation is first random name that came to my mind).

… hosts

iLLiCiTiT · 2025-06-09T09:12:11Z

client/ayon_max/plugins/publish/collect_tyflow_vdb.py

+            if attr_values.get("has_frame_range_validator",
+                               self.validate_tyvdb_frame_range):
+
+                instance.data["families"] += "vdb_frame_validation"


Suggested change

instance.data["families"] += "vdb_frame_validation"

instance.data["families"].append("vdb_frame_validation")

iLLiCiTiT · 2025-06-09T09:12:19Z

client/ayon_max/plugins/publish/validate_frame_range.py

+class ValidateTyVDBFrameRange(ValidateFrameRange):
+    label = "Validate Frame Range (TyFlow VDB)"
+    families = ["vdb_frame_validation"]
+    optional = True


Suggested change

optional = True

iLLiCiTiT · 2025-06-09T09:12:50Z

server/settings/publishers.py

        default_factory=BasicValidateModel,
        title="Validate Frame Range (TyCache)"
    )
+    ValidateTyVDBFrameRange: BasicValidateModel = SettingsField(


This should probably have only enabled in settings, the others will not affect anything.

…he-tyFlow-Export-VDB

iLLiCiTiT · 2025-07-15T09:18:53Z

client/ayon_max/plugins/create/create_tyflow.py

 class CreateTyVDB(plugin.MaxTyflowVDBCacheCreator):
    """Creator plugin for TyFlow VDB."""
-    identifier = "io.ayon.creators.max.vdb"
+    identifier = "io.ayon.creators.max.vdbcache"


This change means all existing instances won't be collected.

iLLiCiTiT · 2025-07-15T09:19:21Z

client/ayon_max/plugins/create/create_tyflow.py

    icon = "gear"
+
+    def get_publish_families(self):
+        return ["vdbcache", "tyflow_vdb"]


Shouldn't that "tyflow_vdb" be added during collection phase based on create plugin identifier instead?

implement vdb product for publishing with tyflow

694c178

moonyuet requested a review from LiborBatek May 30, 2025 09:45

moonyuet self-assigned this May 30, 2025

moonyuet added type: enhancement Improvement of existing functionality or minor addition sponsored This is directly sponsored by a client or community member labels May 30, 2025

moonyuet linked an issue May 30, 2025 that may be closed by this pull request

AY-7855_Exporting VDBs with the tyFlow "Export VDB" #48

Open

moonyuet added 2 commits May 31, 2025 11:59

fix the error encountered during creating and publishing instances

daaf39e

make sure everything published successfully

b2248ff

moonyuet marked this pull request as ready for review May 31, 2025 04:34

moonyuet requested a review from antirotor June 3, 2025 05:37

BigRoy reviewed Jun 3, 2025

View reviewed changes

BigRoy requested a review from iLLiCiTiT June 3, 2025 06:56

iLLiCiTiT reviewed Jun 9, 2025

View reviewed changes

make sure vdb product align with the product type vdbcache from other…

f9f0f7f

… hosts

iLLiCiTiT reviewed Jun 9, 2025

View reviewed changes

edit the setting for validate vdb frame range

1db24c2

moonyuet marked this pull request as draft June 30, 2025 10:05

moonyuet and others added 2 commits June 30, 2025 18:05

Merge branch 'develop' into enhancement/AY-7855_Exporting-VDBs-with-t…

11539f9

…he-tyFlow-Export-VDB

work in progress to implment the published families

ef7b299

iLLiCiTiT reviewed Jul 15, 2025

View reviewed changes

wip vdb publish for tyflow

2fa72be

	instance.data["families"] += "vdb_frame_validation"
	instance.data["families"].append("vdb_frame_validation")

Implementation of publishing vdb product #49

Are you sure you want to change the base?

Implementation of publishing vdb product #49

Conversation

moonyuet commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog Description

Additional review information

Testing notes:

Uh oh!

moonyuet commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BigRoy commented Jun 3, 2025

Uh oh!

BigRoy Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iLLiCiTiT Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

moonyuet commented May 30, 2025 •

edited

Loading

moonyuet commented Jun 3, 2025 •

edited

Loading

BigRoy Jun 3, 2025 •

edited

Loading

iLLiCiTiT Jun 9, 2025 •

edited

Loading