$ref overwrites `nullable` property

[bushkov]

If a property is defined via a $ref and it's nullable set to True, then this nullable will be ignored.

If you take an example from the TODO app, make a title nullable:

    Item:
      type: object
      required:
        - title
      properties:
        title:
          type: string
          maxLength: 40
          nullable: true

and modify ITEMS in app.py to contain an item with null property:

ITEMS = [
    Item(title="Buy milk"),
    Item(title="Buy eggs"),
    Item(title="Make pankaces!"),
    Item(title=None),
]

Then everything works fine.

However, if we define title via a $ref:

    Title:
      type: string
      maxLength: 40
      nullable: true

    Item:
      type: object
      required:
        - title
      properties:
        title:
          $ref: '#/components/schemas/Title'

Then we get the following error:

Value [{'title': 'Buy milk'}, {'title': 'Buy eggs'}, {'title': 'Make pankaces!'}, {'title': None}] not valid for schema of type SchemaType.ARRAY: (<ValidationError: 'None for not nullable'>,)

Note that we started observing this error only since yesterday, before it was working as expected.

[zupo]

Thanks for the bug report!

Note that we started observing this error only since yesterday, before it was working as expected.

What changed since yesterday? Did some dependency get upgraded? This looks like it might be a bug in https://github.com/p1c2u/openapi-core (that pyramid_openapi3 depends on).

[bushkov]

What changed since yesterday?

That's the mystery for us, since no packages that we explicitly use have been updated since the last couple of days. Might be one of the implicit dependencies of those packages.`

On our side https://github.com/p1c2u/openapi-core is frozen to version 0.13.8, because one of the dependencies requires version of openapi-core to be < 0.14. So at least openapi-core remained the same on our side.

[bushkov]

It seems the offending implicit dependency package that causes this issue is jsonschema. We started having this issue once it was updated from 3.2.0 to 4.0.1. If we downgrade jsonschema back to 3.2.0, everything works fine.

[bushkov]

I have noticed something which might be related to this issue. We have the following warning from jsonschema when using pyramid_openapi3:

jsonschema/validators.py:190: DeprecationWarning: Passing a schema to Validator.iter_errors is deprecated and will be removed in a future release. Call validator.evolve(schema=new_schema).iter_errors(...) instead

And in openapi-schema-validator, iter_errors overwrites nullable parameter setting it to False.

[zupo]

Hey @bushkov, did you manage to resolve this, or is the issue still there?

[bushkov]

Hey @bushkov, did you manage to resolve this, or is the issue still there?

No, the issue is still there.

[zupo]

The underlying problem seems to have been addressed in openapi-schema-validator recently: python-openapi/openapi-schema-validator#22.

[bushkov]

@zupo Any update on this issue? I still get ValidationError: 'None for not nullable' for the above example.

[zupo]

As per python-openapi/openapi-schema-validator#22, have you tried using openapi-schema-validator version 0.3.0?

[bushkov]

As per python-openapi/openapi-schema-validator#22, have you tried using openapi-schema-validator version 0.3.0?

Yes, I still get the error even with openapi-schema-validator version 0.3.0.

[zupo]

That is indeed strange. I'll try to take a look in the not too distant future. Any help debugging this is greatly appreciated. If you have any additional info, please share. Or even a failing tests that showcases exactly how the thing fails.

[bushkov]

Any help debugging this is greatly appreciated. If you have any additional info, please share. Or even a failing tests that showcases exactly how the thing fails.

You can use the example with the modified TODO app which I mentioned in my first comment of this issue. If you modify the TODO app following the instructions in the comment and then execute the tests from the TODO app directory, the tests will fail with:

webtest.app.AppError: Bad response: 500 Internal Server Error (not 200)
b'[{"exception":"ValidationError","message":"None for not nullable"}]'

[wilsonjsq]

@zupo any update for this issue? We;re also running into the same problem as this example.

[zupo]

The entire nullable story seems to have been rewritten in OpenAPI v3.1. This library does not yet support v3.1, but work is in progress: #206