[TypeSpec Language] object, tuple values and the value world

[timotheeguerin]

Values in TypeSpec

Problem

Currently we have a very limited set of types in TypeSpec that are used as values(String, Number, Boolean) as well as a misuse of other types(Model and Tuple) in order to fill that gap

For example we might see this in the OpenAPI library:

model Info {
  title?: string;
  version?: string;
  contact?: Contact;
}
model Contact { name?: string; email?: string }

extern dec info(target, data: Info);

which is then used like this

@info({
  title: "My API",
  version: "1.0"
})

however previous signature allows other types to be passed as well, for example a union which is in this case not at all what we want

@info({
  title: "My API",
  version: "1.0"
} | {
  title: "My API alt",
  version: "2.0"
})

Current state

We currently have a way to request a value instead of a type using valueof keyword. This will automatically cast the type to the JS value in a decorator
valueof is limited to string, numeric and boolean scalars today

extern dec doc(target, value: valueof string);

// Here the value param was marshalled to a JS string
export function $doc(context, target, value: string) {}

This introduce the concept of values in the language however as it was limited to string, numeric and boolean scalars the existing String, Number and Boolean types were still used to represent values depending on the context. Those places are:

• Decorator arguments
• Template arguments
• Alias
• Property defaults

Approved design

A follow up part of the valueof proposal was to add object literals #{} and tuple literals #[] to fill the gap of the missing types.
The design allowed property values to be types as well as values.

For example

op myOp(): void;

@foo(#{
  title: "My API",
  operation: myOp,
})

Changes proposed

Distinction of values and types

As shown before the current state reuse the same types for values and types. However with the introduction of object and tuple literal we are now having some new entities that are only meant to be used as values.

Entity Name	Type	Value
Namespace	✅
Model	✅
ModelProperty	✅
Union	✅
UnionVariant	✅
Interface	✅
Operation	✅
Scalar	✅
Tuple	✅
Enum	✅
EnumMember	✅	✅
StringLiteral	✅	✅
NumberLiteral	✅	✅
BooleanLiteral	✅	✅
ObjectLiteral		✅
TupleLiteral		✅
---- intrinsic ---	---	---
null	✅	✅
unknown	✅

Object and Tuple literals only accept other values

In order to make sure that types and value stay separated we need to make sure that object and tuple literals only accept other values as their properties/values.
This mean this example would produce an error

op myOp(): void;

@foo(#{
  title: "My API",
  operation: myOp,
})

If this is a behavior we want to add in the future we could add a special template type that explicitly define a reference to a type.

@foo(#{
  title: "My API",
  operation: Ref<myOp>,
})

Contexts

There is 3 context that can exists in TypeSpec:

• Type only This is when an expression can only be a Type.
  • Model property type
  • Array element type
  • Tuple values
  • Operation parameters
  • Operation return type
  • Union variant type with some exceptions when used as a decorator or template parameter constraint.
• Value only This is when an expression can only be a Value.
  • Default values
• Type and Value Constaints This is when an expression can be a type or a valueof
  • Decorator parameters
  • Template parameters
• Type and Value This is when an expression can be a type or a value.
  • Aliases
  • Decorator arguments
  • Template arguments

What does this mean?

1. You cannot assign a value to something not marked with valueof

{} =>  {} // ok
{name: string} | {} =>  {} // ok
#{} => {} // error

2. You cannot a type to something marked with valueof

{} => valueof {} // error
#{} => valueof {} // ok

3. If you want to accept both you have to be explicit in allowing x | valueof y (e.g. unknown | valueof unknown to allow any type or any value, or {} | valueof {} to any type assignable to {} or any object value)
4. Property default should only accept values
5. Decorators will only marshall values (if possible) to JS types.

Value entities

const statements

In order to separate values from types we need to have a new equivalent to alias(meant for types) for values. This is where const comes in.

Syntax:

const identifier[: Type] = Value;

Example:

// Implicit type
const myValue = "Hello, World!";
// Explicit type
const myValue: string = "Hello, World!";
const myValue: Abc = #{name: "John", age: 30};

Evaluation:

const like alias are evaluated in the reference order not in the declaration order.

This means for example the following is valid:

const a = b;
const b = "foo";

This is needed as we are merging namespace, passing values to decorators which are not evaluated in the declaration order and this would cause all kinda of issues.

Scalar constructors

For primitives scalars(numeric, string, boolean) we can infer the type from a literal in most cases but for other scalars we can't know how to instantiate them, or sometimes it is ambiguous which primitive scalar a literal should be.

Primitive scalar constructors

This only apply to scalars that extends numeric, string or boolean(Not sure we need this one)
As for those scalars are the base of every other values we do have to treat them specially. This mean they have a default constructor that will take their value.

• string for string scalars
• Numeric for numeric scalars
• boolean for boolean scalars

const a = int16(123);
const b = string("abc");
const c = myFormattedString("abc");

Scalars named constructors

For other scalars as we cannot define a syntax to create every possible scalar we can instead allow scalars to define constructor in their body which can then be used to instantiate that scalar with the given values.
The constructor can be defined with the init (or new open for either) keyword.

Those constructors do not actually instantiate the scalar with those values behind the scene they more keep reference of the parameters given to it so an emitter can convert that constructor into their coresponding language scalar initialization.

For example the utcDateTime scalar could look something like that

scalar utcDateTime {
  init fromISO(value: string);
  init fromDate(value: {year: int32, month: int8, day: int8, hour: int8, minute: int8, second: int8, millisecond: int16});
  init now();
}

And could then be used like this

model User {
  createdAt: utcDateTime =  utcDateTime.now();
  expire: utcDateTime =  utcDateTime.fromDate({year: 2100});
  cleanup: utcDateTime =  utcDateTime.fromISO("2025-01-01T01:01:01");
}

Type inference

Scalars

Whenever a value type is declared it is resolved against the constraint to see what type it is.

Single type case

const a: int8 = 123; // a is now a ScalarValue with type int8
model Foo {
  prop?: int32 = 123; // prop default is now a ScalarValue with type int32
}

Multiple type non-ambiguous case

const a: int8 | string = 123; // a is now a ScalarValue with type int8, it cannot be a string

Multiple type ambiguous case

When a value is assigned to multiple type and it is not clear which type it should be, it is a compile error.

const a: int8 | int32 = 123; // it is ambiguous if 123 should be an int8 or int32 as it could be both. This is a compile error.

// instead you have to be explicit in those cases
const a: int8 | int32 = int8(123);

Implicit type

If the target has no type then we just keep it as a XLiteralValue and keep the precise type. It might get assigned a scalar type next time it reach a constraint.

const a = 123; // a is now a NumericLiteralValue

Models

As models are structural in the type there is never a defined model assigned to an object literal. It takes the format of whatever is its current assigned type.

For example given the following models

model Info {
  title?: string;
  version?: int32;
}

model InfoWithExtra {
  ...Info;
  extra?: string
}

// Type of implicit is the exact type `{a: "foo", b: 123}`
const implicit = #{
  a: "foo",
  b: 123
}

const info: Info = implicit; // Now the type of info is just `Info` there is no reference back to the original type.
const infoWithExtra: InfoWithExtra = info; // Now the type of info is just `InfoWithExtra` there is no reference back to the Info type.

Scalars in models

In the same way as scalar works, scalars in models will need to resolve what type they are. Either by being explicit or implicitly resolving from the property scalar type. This also mean that if a property has ambiguous scalars then it will be an error

const a: {foo: int32}  = #{a: 123} // a.foo is an int32
const a: {foo: int32 | int64}  = #{a: 123} // error we can't know which numeric to use

Internals

type Value =
  | ScalarValue
  | NumericValue
  | StringValue
  | BooleanValue
  | ObjectValue
  | ArrayValue
  | EnumMemberValue
  | NullValue;

interface ValueBase {
  valueKind: string;
  type: Type; // Every value has a type. That type could be something completely different(much wider type)
}

interface ObjectValue extends ValueBase {
  valueKind: "ObjectValue";
  type: Model | Union;
  properties: Map<string, ObjectValuePropertyDescriptor>;
}

interface ObjectValuePropertyDescriptor {
  valueKind: "ObjectProperty";
  name: string;
  value: Value;
}

interface ArrayValue extends BaseValue {
  valueKind: "ArrayValue";
  length: number;
  values: Value[];
}

interface ScalarValue extends BaseValue {
  valueKind: "ScalarValue";
  scalar: Scalar; // We need to keep a reference of what scalar this is.
  value: { name: string; args: Value[] }; // e.g. for utcDateTime(2020,12,01)
}
interface NumericValue extends BaseValue {
  valueKind: "NumericValue";
  scalar: Scalar | undefined;
  value: Numeric;
}
interface StringValue extends BaseValue {
  valueKind: "StringValue";
  scalar: Scalar | undefined;
  value: string;
}
interface BooleanValue extends BaseValue {
  valueKind: "BooleanValue";
  scalar: Scalar | undefined;
  value: boolean;
}

interface Numeric {
  (value: string): Numeric;
  /** Return the value as number. @throws if the value is not representable as number */
  asNumber(): number;
  asBigInt(): BigInt; // ?
  asString(): string;
  equals(value: Numeric): boolean;
  gt(value: Numeric): boolean;
  lt(value: Numeric): boolean;
  gte(value: Numeric): boolean;
  lte(value: Numeric): boolean;
}

interface EnumMemberValue extends BaseValue {
  valueKind: "EnumMemberValue";
  value: EnumMember;
}

interface NullValue extends BaseValue {
  valueKind: "NullValue";
}

Things not included in this proposal

Accessing object or tuple literal properties

Things like this are not part of this proposal as it would involve designing optionality in values which doesn't seem necessary at this time and add more complexity to this already large proposal.

const a = #{name: "Abc"}
const b = a.name;

How does this affect users

Library authors

There is only 3 locations where values can get exposed to a library author

1. Decorator arguments: For this one we already receive the JS primitive values when using valueof this would now also make it that we receive the JS object/array in the case of object and tuple values and null for NullValue. For EnumMemberValue user would still receive it as that type as there is no equivalent.
2. Property default: This one is tricky as currently you would get the types that can also be values but here ideally the type would be the Value type defined above but this would be a breaking change. We can however add a new defaultValue: Value property and deprecate the existing default property(which wouldn't be set in the case of object and array values but keep working as before in the other cases).
3. Template arguments: This one is not really something that emitters would usually dig into as templates are already instantiated. So in this case it would just mean that there is additional types that you might receive which is just breaking in the TypeScript compilation but not in the JS code - Like adding any new type to the compiler.

Spec authors

For spec authors the breaking changes are minimal, the only issue is that we used to allow tuples to be used as default values. However now those should be done with tuple literals.

To make this change non breaking we can allow with a deprecation warning tuple and models to be automatically casted to array and object values. For object this is not covering anything we supported before however allowing that would allow our libraries to migrate to using valueof which would then show a deprecated warning to the user which can be automatically resolved by a codefix.

[timotheeguerin]

Design meeting feedback

Need a way to resolve the type in the case of a union

const a: Info | string = #{};

function findTypeForValue(value: ArrayValue): ModelArrayType | Tuple;
function findTypeForValue(value: ObjectValue): Model;
function findTypeForValue(value: ScalarValue): Scalar;
function findTypeForValue(value: ScalarValue): Scalar;
function findTypeForValue(value: EnumMemberValue): Enum;
function findTypeForValue(value: Value): Type;

[timotheeguerin]

Value entities

const statements

In order to separate values from types we need to have a new equivalent to alias(meant for types) for values. This is where const comes in.

Syntax:

const identifier[: Type] = Value;

Example:

// Implicit type
const myValue = "Hello, World!";
// Explicit type
const myValue: string = "Hello, World!";
const myValue: Abc = #{name: "John", age: 30};

Evaluation:

const like alias are evaluated in the reference order not in the declaration order.

This means for example the following is valid:

const a = b;
const b = "foo";

This is needed as we are merging namespace, passing values to decorators which are not evaluated in the declaration order and this would cause all kinda of issues.

Scalar constructors

For primitives scalars(numeric, string, boolean) we can infer the type from a literal in most cases but for other scalars we can't know how to instantiate them, or sometimes it is ambiguous which primitive scalar a literal should be.

Primitive scalar constructors

This only apply to scalars that extends numeric, string or boolean(Not sure we need this one)
As for those scalars are the base of every other values we do have to treat them specially. This mean they have a default constructor that will take their value.

• string for string scalars
• Numeric for numeric scalars
• boolean for boolean scalars

const a = int16(123);
const b = string("abc");
const c = myFormattedString("abc");

Scalars named constructors

For other scalars as we cannot define a syntax to create every possible scalar we can instead allow scalars to define constructor in their body which can then be used to instantiate that scalar with the given values.
The constructor can be defined with the init (or new open for either) keyword.

Those constructors do not actually instantiate the scalar with those values behind the scene they more keep reference of the parameters given to it so an emitter can convert that constructor into their coresponding language scalar initialization.

For example the utcDateTime scalar could look something like that

scalar utcDateTime {
  init fromISO(value: string);
  init fromDate(value: {year: int32, month: int8, day: int8, hour: int8, minute: int8, second: int8, millisecond: int16});
  init now();
}

And could then be used like this

model User {
  createdAt: utcDateTime =  utcDateTime.now();
  expire: utcDateTime =  utcDateTime.fromDate({year: 2100});
  cleanup: utcDateTime =  utcDateTime.fromISO("2025-01-01T01:01:01");
}

Type inference

Scalars

Whenever a value type is declared it is resolved against the constraint to see what type it is.

Single type case

const a: int8 = 123; // a is now a ScalarValue with type int8
model Foo {
  prop?: int32 = 123; // prop default is now a ScalarValue with type int32
}

Multiple type non-ambiguous case

const a: int8 | string = 123; // a is now a ScalarValue with type int8, it cannot be a string

Multiple type ambiguous case

When a value is assigned to multiple type and it is not clear which type it should be, it is a compile error.

const a: int8 | int32 = 123; // it is ambiguous if 123 should be an int8 or int32 as it could be both. This is a compile error.

// instead you have to be explicit in those cases
const a: int8 | int32 = int8(123);

Implicit type

If the target has no type then we just keep it as a XLiteralValue and keep the precise type. It might get assigned a scalar type next time it reach a constraint.

const a = 123; // a is now a NumericLiteralValue

Models

As models are structural in the type there is never a defined model assigned to an object literal. It takes the format of whatever is its current assigned type.

For example given the following models

model Info {
  title?: string;
  version?: int32;
}

model InfoWithExtra {
  ...Info;
  extra?: string
}

// Type of implicit is the exact type `{a: "foo", b: 123}`
const implicit = #{
  a: "foo",
  b: 123
}

const info: Info = implicit; // Now the type of info is just `Info` there is no reference back to the original type.
const infoWithExtra: InfoWithExtra = info; // Now the type of info is just `InfoWithExtra` there is no reference back to the Info type.

Scalars in models

In the same way as scalar works, scalars in models will need to resolve what type they are. Either by being explicit or implicitly resolving from the property scalar type. This also mean that if a property has ambiguous scalars then it will be an error

const a: {foo: int32}  = #{a: 123} // a.foo is an int32
const a: {foo: int32 | int64}  = #{a: 123} // error we can't know which numeric to use

Internals

type Value =
  | ScalarValue
  | NumericValue
  | StringValue
  | BooleanValue
  | ObjectValue
  | ArrayValue
  | EnumMemberValue
  | NullValue;

interface ValueBase {
  valueKind: string;
  type: Type; // Every value has a type. That type could be something completely different(much wider type)
}

interface ObjectValue extends ValueBase {
  valueKind: "ObjectValue";
  type: Model | Union;
  properties: Map<string, ObjectValuePropertyDescriptor>;
}

interface ObjectValuePropertyDescriptor {
  valueKind: "ObjectProperty";
  name: string;
  value: Value;
}

interface ArrayValue extends BaseValue {
  valueKind: "ArrayValue";
  length: number;
  values: Value[];
}

interface ScalarValue extends BaseValue {
  valueKind: "ScalarValue";
  scalar: Scalar; // We need to keep a reference of what scalar this is.
  value: { name: string; args: Value[] }; // e.g. for utcDateTime(2020,12,01)
}
interface NumericValue extends BaseValue {
  valueKind: "NumericValue";
  scalar: Scalar | undefined;
  value: Numeric;
}
interface StringValue extends BaseValue {
  valueKind: "StringValue";
  scalar: Scalar | undefined;
  value: string;
}
interface BooleanValue extends BaseValue {
  valueKind: "BooleanValue";
  scalar: Scalar | undefined;
  value: boolean;
}

interface Numeric {
  (value: string): Numeric;
  /** Return the value as number. @throws if the value is not representable as number */
  asNumber(): number;
  add(value: Numeric): Numeric;
  substract(value: Numeric): Numeric;
  multiply(value: Numeric): Numeric;
  divide(value: Numeric): Numeric;
  equals(value: Numeric): boolean;
}

interface EnumMemberValue extends BaseValue {
  valueKind: "EnumMemberValue";
  value: EnumMember;
}

interface NullValue extends BaseValue {
  valueKind: "NullValue";
}

Things not included in this proposal

Accessing object or tuple literal properties

Things like this are not part of this proposal as it would involve designing optionality in values which doesn't seem necessary at this time and add more complexity to this already large proposal.

const a = #{name: "Abc"}
const b = a.name;

[markcowl]

@timotheeguerin Please add in a Story Points estimate

[markcowl]

This change enables three important features for the language

• Enable representing examples in OpenAPI3 using Value types of the corresponding TypeSpec types.
• Enable default values for a wide variety of types, rather than the small set of scalar literals that are currently supported
• Enable type checking in decorators and templates for values of any type. This will vastly simplify decorator and template definition in libraries.

Describe the breaking change
Default values in specs will now be of type 'Value' rather than of type 'Type'. Emitters that process values will receive a deprecation warning if they continue to use the 'default' property to access property defaults, rather than the new 'defaultValue' property

Back-compat design to avoid immediate breaking
Continue to support converting values to literal types in the 'default' property for backward compatibility. Deprecated the property to encourage library and emitter developers to move to supporting the new 'Value' type.

Impact
This change affecting:

• ARM
• Data-plane
• Existing specs:
• New specs
• Emitters : No DPG Emitter is affected
• SDK

Targeted Sprint
Sprint: June 2024

Additional information
Add any other information about the change including fixes for here.