Request for Clarification/Refinement: V1.1 "Original State" Data Storage Principle

[ajayojha]

First, thank you for the continued efforts in maintaining and evolving the ASVS — it’s an incredibly valuable resource for the security and development community.

While reviewing section V1.1: Encoding and Sanitization Architecture, I came across the following statement:

They also aim to ensure that whenever data is stored, it remains in its original state and is not stored in an encoded or escaped form (e.g., HTML encoding), to prevent double encoding issues.

The above statement assumes that:

• Output encoding is always done correctly (which often isn't true in large, evolving systems),
• There's never a reason to store sanitized/encoded data (which isn't always the case).

Practical risks of strict raw-data-only storage

• Storing unsafe data can lead to persistent vulnerabilities, especially if output encoding is missed.
• Repetitive UI rendering tasks increase the risk of inconsistent encoding, as developers may forget or incorrectly apply output encoding.
• Repeated encoding at each render point can be a performance burden, especially for high-volume or dynamic applications.
• In real-world usage, WYSIWYG editors are not the only sources of potentially unsafe input — assuming so can leave gaps in coverage.

Blanket rule to “store raw and never encoded” is risky. Security should balance between defense-in-depth and clear encoding boundaries.

I would be happy know OWASP ASVS team on the above concern.

[elarlang]

The requirement was made to address exactly those anti-patterns listed here.

Encoding data into the storage and not keeping the original form is ruining the integrity of the data.

Storing unsafe data can lead to persistent vulnerabilities, especially if output encoding is missed.

The problem to solve is to have the correct output encoding. Proposal is an insecure anti-pattern.

Repetitive UI rendering tasks increase the risk of inconsistent encoding, as developers may forget or incorrectly apply output encoding.

The problem to solve here is to use the correct caching mechanism. To ruin the original data to solve the caching problem from one output format is clearly an incorrect solution and an anti-pattern.

Repeated encoding at each render point can be a performance burden, especially for high-volume or dynamic applications.

See previous comment.

In real-world usage, WYSIWYG editors are not the only sources of potentially unsafe input — assuming so can leave gaps in coverage.

Output encoding must be applied per functionality needs - is there a need to show it as text, or is there a need to let the browser render it.

Blanket rule to “store raw and never encoded” is risky.

Disagree. It is vice-versa.

[ajayojha]

On caching as a mitigation for missed output encoding:
With respect, caching isn’t always a viable or efficient solution to mitigate encoding omissions at output time. In one of our enterprise applications, over 10,000 property are rendered across multiple paginated views. Using caching here introduces several engineering trade-offs:

• Increased code complexity for keeping cache and database in sync.
• Difficulty maintaining data consistency across relational joins.
• High infrastructure overhead for large data volumes.
• Reduced scalability for smaller organizations with limited resources.

Introducing an elaborate caching mechanism just to enforce output encoding increases the maintenance burden and risks masking other architectural issues.

Overall concern and suggestion:
While I fully support the principle of context-aware output encoding, the reality for many small to medium-sized teams — without dedicated security personnel — is that absolute rule like “should not be stored in an encoded or escaped form” can backfire. Developers may interpret it as discouragement from taking any early-stage defensive actions, even when certain data is always destined for the same HTML output context.

To better support teams across different maturity levels, I’d like to suggest refining the guidance. We could suggest:

• Reinforce storing raw input as a recommended best practice.
• But also describe situations where pre-encoded storage may be appropriate.
• Share some usecases, so development teams can make informed decisions tailored to their application’s complexity and exposure.

If this direction aligns with the ASVS team’s goals, I’d be happy to draft a pull request proposing revised wording with use cases, subject to further feedback.

[elarlang]

If this direction aligns with the ASVS team’s goals, I’d be happy to draft a pull request proposing revised wording with use cases, subject to further feedback.

Sorry, this direction does not align with the goals the mentioned requirement or section defines.

For example, it is not possible to encode data for HTML before saving it to the database, as:

• It ruins the integrity of the data - e.g., it was not the value that the user entered
• It assumes that the only output encoding is HTML, but if it is to be used it in JSON, CSV, displayed as just text, or whatever other format, it is already in the incorrect format

[ajayojha]

Okay, No problem :).
Thanks for your response. I accept that the OWASP ASVS team cannot change the wording of V1.1, even though it doesn’t clearly cover the point.

I completely agree with ASVS V1.1 for plain text data: always store it as is.

My main point is that ASVS V1.1's statement, "data is not stored in an encoded or escaped form," seems to contradict ASVS V1.3's guidance that "sanitization becomes necessary" for content where encoding isn't possible (like HTML that needs to keep its formatting). This ambiguity makes it tough for architects/developers in real projects.

For user-generated HTML content (e.g., from a blog editor):

• Data Integrity: When HTML sanitization happens, the data is intentionally changed by removing malicious parts (like <script> tags). This is a necessary security transformation. For user-generated HTML, the "integrity" we aim to preserve isn't the raw, potentially unsafe input; it's the safe, usable HTML. We successfully apply this principle in our own enterprise applications when dealing with potentially malicious user-provided HTML content.

My suggestion is for ASVS V1.1 to be clearer (A clear preference for raw storage): while it's crucial for plain text, it should explicitly clarify that for user-generated HTML content, HTML sanitization (as per V1.3) is a required step before storage. This helps bridge the gap between these two important ASVS principles.

The goal is not to replace encoding-at-output, but to empower teams to make context-aware decisions without introducing risk through oversimplified rules.

I understand this might not align with ASVS at the moment, but if the team revisits this section in future versions,I hope this practical perspective might help inform potential refinements or clarifications. I’d be happy to contribute or support the effort whenever needed.

[ajayojha]

The problem to solve here is to use the correct caching mechanism.

I assume that The V1.1 rule against storing encoded data applies to the persistent System of Record (the database), while the advice to "use caching" refers to storing rendered/encoded output in a volatile performance layer.

This highlights what I believe is a critical clarity gap in the standard: the document itself does not make this architectural distinction explicit.

As a result, a development team could misinterpret V1.1's use of the word "stored" as a blanket rule that applies to any form of storage, including caches. This could lead them to avoid a necessary and valid performance optimization pattern.

Furthermore, there is a conflict with the practices of major, security-conscious platforms have made a deliberate architectural choice to perform sanitization and rendering on write, storing the resulting safe HTML in their primary database. They do this as a fundamental part of their defense-in-depth and performance strategy.

Given this, I would suggest the ASVS team consider (if align) that a future version of the standard should:

• Reinforce "storing raw" as the default and recommended best practice.
• Explicitly clarify that this rule applies to the System of Record (database), not necessarily volatile caches.