Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement predefined field constraints #246

Merged
merged 25 commits into from
Sep 17, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0d5bfe9
Implement shared field rules
jchadwick-buf Sep 3, 2024
8657cd2
Fix bad formatting
jchadwick-buf Sep 4, 2024
bf72afb
Add shared rule tests for each type
jchadwick-buf Sep 4, 2024
9030e64
Add tests for 'rule' constant
jchadwick-buf Sep 4, 2024
0236de1
Add tests using shared rules from proto3
jchadwick-buf Sep 4, 2024
d72b7d7
Add docs for shared field rules
jchadwick-buf Sep 4, 2024
4ef8796
Further documentation improvements
jchadwick-buf Sep 4, 2024
13aee7a
Merge branch 'main' of https://github.com/bufbuild/protovalidate into…
jchadwick-buf Sep 4, 2024
09dcec8
Fix signed/unsigned confusion in shared rules test
jchadwick-buf Sep 6, 2024
23b00c8
Merge branch 'main' of https://github.com/bufbuild/protovalidate into…
jchadwick-buf Sep 6, 2024
74e3f64
Documentation improvements
jchadwick-buf Sep 6, 2024
bb9336f
Add test cases covering mixed shared and custom constraints
jchadwick-buf Sep 6, 2024
006e5d5
Add test with standard, shared and custom constraints on field
jchadwick-buf Sep 6, 2024
ddde592
Merge branch 'main' of https://github.com/bufbuild/protovalidate into…
jchadwick-buf Sep 9, 2024
628a280
Add workaround for bufbuild/buf#3306
jchadwick-buf Sep 9, 2024
40d8d14
Fix bad formatting
jchadwick-buf Sep 9, 2024
1696bea
Merge everything into validate.proto + proto2 conversion
jchadwick-buf Sep 10, 2024
e22dab3
Re-add bufbuild/buf#3306 workaround for shared_rules_proto3.proto only
jchadwick-buf Sep 10, 2024
d2d22f3
generate
jchadwick-buf Sep 10, 2024
4fd7a36
shared_field: Fix type, better comment hopefully
jchadwick-buf Sep 10, 2024
95e8a98
Shared -> predefined, fix docs, gen, etc.
jchadwick-buf Sep 11, 2024
b1890fc
updates
jchadwick-buf Sep 16, 2024
eef5323
generate
jchadwick-buf Sep 16, 2024
c1862ea
oops, no need for a helper here
jchadwick-buf Sep 17, 2024
444deab
Merge branch 'main' of https://github.com/bufbuild/protovalidate into…
jchadwick-buf Sep 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@ library.
This guide will help you understand when and how to use these standard
constraints effectively.

- [Predefined Constraints](predefined-constraints.md): This section
discusses how to extend `protovalidate` with custom reusable rules that
behave similarly to the standard constraints. This can be useful in order
to share similar custom validation logic across multiple fields or
messages.

- [Errors](errors.md): This section explains the error system in `protovalidate`
and provides guidance on how to handle them effectively.

Expand All @@ -38,7 +44,7 @@ designed to help you implement new language support and assist in migrating your
existing projects to `protovalidate`.

- [Conformance](conformance.md): This document is dedicated to explaining the
Conformance tool. Learn how to use this tool to ensure all implementations
Conformance tool. Learn how to use this tool to ensure all implementations
align with `protovalidate`'s rules and constraints effectively.

- [Migrate](migrate.md): If you're planning to migrate your existing project to
Expand Down
71 changes: 71 additions & 0 deletions docs/predefined-constraints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Predefined Constraints

Custom constraints in `protovalidate` afford a lot of power, but can become
cumbersome and repetitive when the same kind of custom constraints are needed
across multiple fields or messages. To this end, `protovalidate` provides a
mechanism for creating reusable constraints that can be applied on multiple
fields.

Predefined constraints require Protobuf extensions, which are not available in
proto3. Either proto2 or at least Protobuf 2023 Edition must be used to define
predefined constraints. Predefined constraints defined in proto2 or Protobuf
2023 Edition or later can be imported and utilized in proto3 files.

## Predefined Field Constraints

To create a predefined field constraint, extend one of the standard rules
messages. For example, to define a new rule for `float` fields, extend
`buf.validate.FloatRules`, as follows:

```proto
import "buf/validate/validate.proto";

extend buf.validate.FloatRules {
float abs_range = 80048952 [(buf.validate.predefined).cel = {
id: "float.abs_range"
expression: "this >= -rule && this <= rule"
message: "float value is out of range"
}];
}
```

> [!TIP]
> Constraints can refer to their own value with the `rule` constant. Rules apply
> when they are set, so a boolean constraint in the form of `is_...` should
> always check to ensure that `rule` is `true`.

> [!WARNING]
> Be mindful that extension numbers must not conflict with any other extension
> to the same message across _all_ Protobuf files in a given process. This
> restriction also applies to users that consume Protobuf files indirectly as
> dependencies. The same extension number may be re-used across different kinds
> of constraint, e.g. `1000` in `FloatRules` is distinct from `1000` in
> `Int32Rules`.
>
> Extension numbers may be from 1000 to 536870911, inclusive. Values from 1000
> to 99999 are reserved for [Protobuf Global Extension Registry][1] entries, and
> values from 100000 to 536870911 are reserved for integers that are not
> explicitly assigned. It is discouraged to use the latter range for rules that
> are defined in public schemas due to the risk of conflicts. If using a
> randomly-generated integer as a tag number, please use an appropriate source
> of randomness. [This link to random.org][2] can be used if you wish to do
> this, but be aware that it is not recommended.

[1]: https://github.com/protocolbuffers/protobuf/blob/main/docs/options.md "Protobuf Global Extension Registry"
[2]: https://www.random.org/integers/?num=1&min=100000&max=536870911&format=html&col=1&base=10 "RANDOM.ORG - Integer Generator"

Similarly to the standard constraints, a rule will take effect when it is set on
the options of a field. Here is how one might use a predefined constraint:

```proto
message MyMessage {
float normal_value = 1 [(buf.validate.field).float.(abs_range) = 1.0];
}
```

> [!TIP]
> Extensions are always qualified by the package they are defined in. In this
> example, we assume that `abs_range` is defined in the same package it is used
> in, so no further qualification is needed. In other cases, you will need to
> qualify the package name of the extension, e.g.
> `(buf.validate.field).float.(foo.bar.abs_range)`
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ proto_library(
"messages.proto",
"numbers.proto",
"oneofs.proto",
"predefined_rules_proto2.proto",
"predefined_rules_proto3.proto",
"predefined_rules_proto_editions.proto",
"repeated.proto",
"required_field_proto2.proto",
"required_field_proto3.proto",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,288 @@
// Copyright 2023 Buf Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto2";

package buf.validate.conformance.cases;

import "buf/validate/validate.proto";
import "google/protobuf/duration.proto";
import "google/protobuf/timestamp.proto";

extend buf.validate.FloatRules {
optional float float_abs_range_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "float.abs_range.proto2"
expression: "this >= -rule && this <= rule"
message: "float value is out of range"
}];
}

extend buf.validate.DoubleRules {
optional double double_abs_range_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "double.abs_range.proto2"
expression: "this >= -rule && this <= rule"
message: "double value is out of range"
}];
}

extend buf.validate.Int32Rules {
optional bool int32_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "int32.even.proto2"
expression: "this % 2 == 0"
message: "int32 value is not even"
}];
}

extend buf.validate.Int64Rules {
optional bool int64_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "int64.even.proto2"
expression: "this % 2 == 0"
message: "int64 value is not even"
}];
}

extend buf.validate.UInt32Rules {
optional bool uint32_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "uint32.even.proto2"
expression: "this % 2u == 0u"
message: "uint32 value is not even"
}];
}

extend buf.validate.UInt64Rules {
optional bool uint64_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "uint64.even.proto2"
expression: "this % 2u == 0u"
message: "uint64 value is not even"
}];
}

extend buf.validate.SInt32Rules {
optional bool sint32_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "sint32.even.proto2"
expression: "this % 2 == 0"
message: "sint32 value is not even"
}];
}

extend buf.validate.SInt64Rules {
optional bool sint64_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "sint64.even.proto2"
expression: "this % 2 == 0"
message: "sint64 value is not even"
}];
}

extend buf.validate.Fixed32Rules {
optional bool fixed32_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "fixed32.even.proto2"
expression: "this % 2u == 0u"
message: "fixed32 value is not even"
}];
}

extend buf.validate.Fixed64Rules {
optional bool fixed64_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "fixed64.even.proto2"
expression: "this % 2u == 0u"
message: "fixed64 value is not even"
}];
}

extend buf.validate.SFixed32Rules {
optional bool sfixed32_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "sfixed32.even.proto2"
expression: "this % 2 == 0"
message: "sfixed32 value is not even"
}];
}

extend buf.validate.SFixed64Rules {
optional bool sfixed64_even_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "sfixed64.even.proto2"
expression: "this % 2 == 0"
message: "sfixed64 value is not even"
}];
}

extend buf.validate.BoolRules {
optional bool bool_false_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "bool.false.proto2"
expression: "this == false"
message: "bool value is not false"
}];
}

extend buf.validate.StringRules {
optional bool string_valid_path_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "string.valid_path.proto2"
expression: "!this.matches('^([^/.][^/]?|[^/][^/.]|[^/]{3,})(/([^/.][^/]?|[^/][^/.]|[^/]{3,}))*$') ? 'not a valid path: `%s`'.format([this]) : ''"
}];
}

extend buf.validate.BytesRules {
optional bool bytes_valid_path_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "bytes.valid_path.proto2"
expression: "!string(this).matches('^([^/.][^/]?|[^/][^/.]|[^/]{3,})(/([^/.][^/]?|[^/][^/.]|[^/]{3,}))*$') ? 'not a valid path: `%s`'.format([this]) : ''"
}];
}

extend buf.validate.EnumRules {
optional bool enum_non_zero_proto2 = 1161 [(buf.validate.predefined).cel = {
id: "enum.non_zero.proto2"
expression: "int(this) != 0"
message: "enum value is not non-zero"
}];
}

extend buf.validate.RepeatedRules {
optional bool repeated_at_least_five_proto2 = 1161 [(predefined).cel = {
id: "repeated.at_least_five.proto2"
expression: "uint(this.size()) >= 5u"
message: "repeated field must have at least five values"
}];
}

extend buf.validate.DurationRules {
optional bool duration_too_long_proto2 = 1161 [(predefined).cel = {
id: "duration.too_long.proto2"
expression: "this <= duration('10s')"
message: "duration can't be longer than 10 seconds"
}];
}

extend buf.validate.TimestampRules {
optional bool timestamp_in_range_proto2 = 1161 [(predefined).cel = {
id: "timestamp.time_range.proto2"
expression: "int(this) >= 1049587200 && int(this) <= 1080432000"
message: "timestamp out of range"
}];
}

message PredefinedFloatRuleProto2 {
optional float val = 1 [(buf.validate.field).float.(float_abs_range_proto2) = 1.0];
}

message PredefinedDoubleRuleProto2 {
optional double val = 1 [(buf.validate.field).double.(double_abs_range_proto2) = 1.0];
}

message PredefinedInt32RuleProto2 {
optional int32 val = 1 [(buf.validate.field).int32.(int32_even_proto2) = true];
}

message PredefinedInt64RuleProto2 {
optional int64 val = 1 [(buf.validate.field).int64.(int64_even_proto2) = true];
}

message PredefinedUInt32RuleProto2 {
optional uint32 val = 1 [(buf.validate.field).uint32.(uint32_even_proto2) = true];
}

message PredefinedUInt64RuleProto2 {
optional uint64 val = 1 [(buf.validate.field).uint64.(uint64_even_proto2) = true];
}

message PredefinedSInt32RuleProto2 {
optional sint32 val = 1 [(buf.validate.field).sint32.(sint32_even_proto2) = true];
}

message PredefinedSInt64RuleProto2 {
optional sint64 val = 1 [(buf.validate.field).sint64.(sint64_even_proto2) = true];
}

message PredefinedFixed32RuleProto2 {
optional fixed32 val = 1 [(buf.validate.field).fixed32.(fixed32_even_proto2) = true];
}

message PredefinedFixed64RuleProto2 {
optional fixed64 val = 1 [(buf.validate.field).fixed64.(fixed64_even_proto2) = true];
}

message PredefinedSFixed32RuleProto2 {
optional sfixed32 val = 1 [(buf.validate.field).sfixed32.(sfixed32_even_proto2) = true];
}

message PredefinedSFixed64RuleProto2 {
optional sfixed64 val = 1 [(buf.validate.field).sfixed64.(sfixed64_even_proto2) = true];
}

message PredefinedBoolRuleProto2 {
optional bool val = 1 [(buf.validate.field).bool.(bool_false_proto2) = true];
}

message PredefinedStringRuleProto2 {
optional string val = 1 [(buf.validate.field).string.(string_valid_path_proto2) = true];
}

message PredefinedBytesRuleProto2 {
optional bytes val = 1 [(buf.validate.field).bytes.(bytes_valid_path_proto2) = true];
}

message PredefinedEnumRuleProto2 {
enum EnumProto2 {
ENUM_PROTO2_ZERO_UNSPECIFIED = 0;
ENUM_PROTO2_ONE = 1;
}
optional EnumProto2 val = 1 [(buf.validate.field).enum.(enum_non_zero_proto2) = true];
}

message PredefinedRepeatedRuleProto2 {
repeated uint64 val = 1 [(buf.validate.field).repeated.(repeated_at_least_five_proto2) = true];
}

message PredefinedDurationRuleProto2 {
optional google.protobuf.Duration val = 1 [(buf.validate.field).duration.(duration_too_long_proto2) = true];
}

message PredefinedTimestampRuleProto2 {
optional google.protobuf.Timestamp val = 1 [(buf.validate.field).timestamp.(timestamp_in_range_proto2) = true];
}

message PredefinedAndCustomRuleProto2 {
optional int32 a = 1 [
(field).cel = {
id: "predefined_and_custom_rule_scalar_proto2"
expression: "this > 24 ? '' : 'a must be greater than 24'"
},
(field).int32.(int32_even_proto2) = true
];

optional Nested b = 2 [(field).cel = {
id: "predefined_and_custom_rule_embedded_proto2"
message: "b.c must be a multiple of 3"
expression: "this.c % 3 == 0"
}];

message Nested {
optional int32 c = 1 [
(field).cel = {
id: "predefined_and_custom_rule_nested_proto2"
expression: "this > 0 ? '' : 'c must be positive'"
},
(field).int32.(int32_even_proto2) = true
];
}
}

message StandardPredefinedAndCustomRuleProto2 {
optional int32 a = 1 [
(field).int32.lt = 28,
(field).int32.(int32_even_proto2) = true,
(field).cel = {
id: "standard_predefined_and_custom_rule_scalar_proto2"
expression: "this > 24 ? '' : 'a must be greater than 24'"
}
];
}
Loading
Loading