Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add extern_protocol! macro and ProtocolType trait #250

Merged
merged 9 commits into from
Nov 23, 2022

Conversation

madsmtm
Copy link
Owner

@madsmtm madsmtm commented Aug 29, 2022

Unsure about how to handle this.

The need arose because MTLBuffer is a protocol, not an actual class (it can't be instantiated by itself), but is useful to handle it as-if it's a class that you're holding instances to.

See https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/WorkingwithProtocols/WorkingwithProtocols.html

@madsmtm madsmtm added the A-objc2 Affects the `objc2`, `objc2-exception-helper` and/or `objc2-encode` crates label Aug 29, 2022
@madsmtm madsmtm added this to the objc2 v0.3 milestone Aug 29, 2022
@madsmtm madsmtm force-pushed the extern-protocol-macro branch 3 times, most recently from 2566684 to 03cb28b Compare September 2, 2022 16:22
@madsmtm
Copy link
Owner Author

madsmtm commented Sep 4, 2022

Need to figure out how we reconcile protocols and traits - for example, NSCopying is useful as a trait, while MTLDevice is useful as a type.
Ideally we'd have one thing called NSCopying, if not, we could perhaps have NSCopying and NSCopyingTrait, with a link between these somehow?

The last usecase of protocols would be like NSWindowDelegate, which is useful inside declare_class!, but that is even less thought-out than the other two.

@madsmtm
Copy link
Owner Author

madsmtm commented Sep 9, 2022

Note also that there is not really such a thing as a "super" protocol; a protocol can have any number of parent/super protocols (similar to a trait having zero or more supertraits).

@madsmtm
Copy link
Owner Author

madsmtm commented Sep 19, 2022

Idea:

unsafe trait ProtocolType {
    // May not always return an protocol for _reasons_???
    pub fn protocol() -> Option<&'static Protocol>;
}
extern_protocol!(
    #[derive(Debug)]
    pub struct NSCopying;

    unsafe impl ProtocolType for NSCopying {}

    pub trait NSCopyingProtocol {
        #[sel_id(copy)]
        fn copy(&self) -> Id<Self, Shared>;
    }
);

// Becomes

pub struct NSCopying {
    inner: Object,
}

impl NSCopyingProtocol for NSCopying {}

pub trait NSCopyingProtocol {
    fn copy(&self) -> Id<Self, Shared> {
        ...
    }
}

@madsmtm
Copy link
Owner Author

madsmtm commented Sep 21, 2022

Need to actually figure out the use-cases for protocols.

So far I have:

  • Many classes implement, want to call a method on the protocol (e.g. NSCopying)
    • Maybe interesting to be able to use as a bound?
  • Ability to interact with protocol as a normal message-able object (e.g. as done in Metal)

But there's bound to be more than this!

@madsmtm madsmtm force-pushed the extern-protocol-macro branch 3 times, most recently from 3265e07 to 904650e Compare September 23, 2022 14:50
@madsmtm
Copy link
Owner Author

madsmtm commented Oct 4, 2022

@ericmarkmartin
Copy link

ericmarkmartin commented Oct 30, 2022

I have a use case---I'm trying to write bindings for the AuthenticationServices framework and there's the ASAuthorizationProvider protocol.

Tbh I'm not entirely sure why this is a protocol rather than a superclass, but in the framework it's used as the return type for ASAuthorizationRequest's provider property.

I guess probably all I'd need out of some protocol binding macro is appropriate From and TryFrom implementations between the "protocol object" (not sure if that's the write term) and the implementing types.

@madsmtm
Copy link
Owner Author

madsmtm commented Oct 31, 2022

Thanks for the use-case, I think it's very similar to what Metal does.

Also, further ideas:

// in objc2
unsafe trait ConformsTo<P: ProtocolType>: Message {
    fn as_protocol(&self) -> &P {
        ...
    }
    fn as_protocol_mut(&mut self) -> &mut P {
        ...
    }
}
// + Id::into_protocol

// Usage
extern_protocol!(
    #[derive(Debug)]
    pub struct ASAuthorizationProvider;

    unsafe impl ProtocolType for ASAuthorizationProvider {
        // No methods
        // The useful part of declaring the methods in here is that we'd be able to
        // do extra some tricks when using `declare_class!`.
    }
);

// The set of protocols that the type implements
unsafe impl ConformsTo<NSObject> for ASAuthorizationProvider {}

// When defining these types, we can state which protocols they implement
unsafe impl ConformsTo<ASAuthorizationProvider> for ASAuthorizationAppleIDProvider {}
unsafe impl ConformsTo<ASAuthorizationProvider> for ASAuthorizationPasswordProvider {}
unsafe impl ConformsTo<ASAuthorizationProvider> for ASAuthorizationPlatformPublicKeyCredentialProvider {}
unsafe impl ConformsTo<ASAuthorizationProvider> for ASAuthorizationSecurityKeyPublicKeyCredentialProvider {}
unsafe impl ConformsTo<ASAuthorizationProvider> for ASAuthorizationSingleSignOnProvider {}

// Maybe with a macro so that we can `impl AsRef<ASAuthorizationProvider> for X`?

fn main() {
    let obj = ASAuthorizationAppleIDProvider::new();
    let proto: &ASAuthorizationProvider = obj.as_protocol();
    // Do something with the protocol
}

madsmtm added a commit that referenced this pull request Nov 3, 2022
We can't parse it yet though, see #250
@madsmtm
Copy link
Owner Author

madsmtm commented Nov 3, 2022

Note: Some protocols are "virtual", see the objc_non_runtime_protocol attribute, so as part of this it would make sense to store information about the protocol's methods in a way that we could then check that the protocol is properly implemented in declare_class! (which we already do, but only for non-virtual protocols).

EDIT: Usage of this property is very rare, I confused it with something else.

@madsmtm madsmtm mentioned this pull request Nov 3, 2022
8 tasks
@madsmtm
Copy link
Owner Author

madsmtm commented Nov 11, 2022

I took a look at a lot of protocols, as a summary I've tried to categorize the different usage of protocols as follows:

  1. The user can implement it to change the behaviour of some class (aka. delegates, or delegates in disguise). If not a delegate, these usually have existing classes that implement them.
  2. Capture common behaviour on a few classes, and the user wants to call those methods.
  3. Has no methods, meant as a "marker" protocol - may be interesting to use as a generic bound, also interesting to use as a concrete type.

A. Uses instancetype in return type to refer to the implementing class.
B. Protocols that really require some sort of associated type because of arg/return type in their methods.
C. The protocol is an "informal" protocol in the sense that types don't actually directly implement it, it is mostly there to make it look like they do (?)

Protocols in Foundation:

NSPortDelegate, NSURLDownloadDelegate, and anything else that ends with `Delegate`: 1
NSDecimalNumberBehaviors: 1
NSExtensionRequestHandling: 1
NSFilePresenter: 1
NSItemProviderWriting: 1
NSItemProviderReading: 1, A
NSCoding: 1, 2, A
NSLocking: 2
NSSecureCoding: 1
NSCopying: 2, B
NSMutableCopying: 2, B
NSFastEnumeration: 2, B
NSDiscardableContent: 1, (2)
NSProgressReporting: 1, (2)
NSURLAuthenticationChallengeSender: 1
NSURLHandleClient: 1
NSURLProtocolClient: 2
NSXPCProxyCreating: 2

Protocols in CoreData:

NSFetchedResultsSectionInfo: 2
NSFetchRequestResult: 3

Protocols in AppKit:

NSWindowDelegate, other delegates: 1
NSAccessibilityGroup: 3
NSAccessibility...: 1, 2
NSAlignmentFeedbackToken: 3
NSAnimatablePropertyContainer: 1, 2, A
NSAppearanceCustomization: 2
NSServicesMenuRequestor: 2, C
NSCollectionViewElement: 1
... many more!

Metal should mostly be case 2 as well (I at least know it doesn't do case A).


The current design with creating concrete types for each protocol, and allowing bounds to use ConformsTo, seems to work for case 1, 2, 3 and C, but falls a bit short in case A and B.

I think I'll go with defining custom traits for a few of those important cases like NSCopying, NSMutableCopying, NSFastEnumeration and NSCoding, and then NSItemProviderReading and NSAnimatablePropertyContainer will probably just have to live with it being a bit suboptimal (translating instancetype to Self is still sound in those cases, just not as precise as it could be).

@madsmtm
Copy link
Owner Author

madsmtm commented Nov 11, 2022

Wow, can't believe I didn't think of this before: Protocols can have class methods!

They're quite rare though, the only instances in Foundation + AppKit are the following (so understandable that I forgot this):

+[NSSecureCoding supportsSecureCoding] // property
+[NSItemProviderWriting writableTypeIdentifiersForItemProvider] // property
+[NSItemProviderWriting itemProviderVisibilityForRepresentationWithTypeIdentifier:]
+[NSItemProviderReading readableTypeIdentifiersForItemProvider] // property
+[NSItemProviderReading objectWithItemProviderData:typeIdentifier:error:]
+[NSAnimatablePropertyContainer defaultAnimationForKey:]
+[NSPasteboardReading readableTypesForPasteboard:]
+[NSPasteboardReading readingOptionsForType:pasteboard:]
+[NSWindowRestoration restoreWindowWithIdentifier:state:completionHandler:]

Question is: Is that few enough that we'll just shrug it off and say "we won't try to handle this", or do we need to figure out a way to handle this?

@madsmtm
Copy link
Owner Author

madsmtm commented Nov 15, 2022

We need some way to specify to the macro "this protocol inherits from NSObject", so that the __inner field can be NSObject, and #[derive(Debug, PartialEq, Eq, Hash)] works.

But maybe the macro can just extract those derives and call let obj: &NSObject = self.as_protocol();?

@madsmtm
Copy link
Owner Author

madsmtm commented Nov 15, 2022

Alternatively: We could allow specifying a parent type for the protocols that have a direct parent (such as MTLBuffer) - this would allow Deref impls to those as well, which would be beneficial for usability.

@madsmtm
Copy link
Owner Author

madsmtm commented Nov 16, 2022

I had an idea for how we might even further improve things, but in the interest of speeding this PR up, I moved that to #291.

Missing parts of this is basically just getting declare_class! to work with ConformsTo.

@madsmtm madsmtm marked this pull request as ready for review November 23, 2022 08:49
@madsmtm madsmtm added the enhancement New feature or request label Nov 23, 2022
@madsmtm madsmtm merged commit bf28404 into master Nov 23, 2022
@madsmtm madsmtm deleted the extern-protocol-macro branch November 23, 2022 12:00
madsmtm added a commit that referenced this pull request Nov 24, 2022
We can't parse it yet though, see #250
madsmtm added a commit that referenced this pull request Dec 8, 2022
We can't parse it yet though, see #250
madsmtm added a commit that referenced this pull request Dec 21, 2022
See full history in 1c4c875.

Add initial header translation

Closer to usable

Mostly works with NSCursor

Mostly works with NSAlert.h

Refactor a bit

AppKit is now parse-able

handle reserved keywords

Handle protocols somewhat

Handle the few remaining entity kinds

Works with Foundation

Cleanup

Refactor

Refactor Method to (almost) be PartialEq

Parse more things

Parse NSConsumed

Verify memory management

More work

Fix reserved keywords

Refactor statements

Add initial availability

Prepare RustType

Split RustType up in parse and ToToken part

Add icrate

Add config files

Temporarily disable protocol generation

Generate files

Add initial generated files for Foundation

Skip "index" header

Add basic imports

Allow skipping methods

Properly emit `unsafe`

Make classes public

Rename objc feature flag

Improve imports somewhat

Further import improvements

Handle basic typedefs

Improve generics handling

Improve pointers to objects

Refactor RustType::TypeDef

Mostly support generics

Refactor config setup

Small fixes

Support nested generics

Comment out a bit of debug logging

Emit all files

Parse sized integer types

Parse typedefs that map to other typedefs

Appease clippy

Add `const`-parsing for RustType::Id

Parse Objective-C in/out/inout/bycopy/byref/oneway qualifiers

Fix `id` being emitted when it actually specifies a protocol

Make AppKit work again

Parse all qualifiers, in particular lifetime qualifiers

More consistent ObjCObjectPointer parsing

Validate some lifetime attributes

Fix out parameters (except for NSError)

Assuming we find a good solution to #277

Refactor Stmt objc declaration parsing

Clean up how return types work

Refactor property parsing

Fixes their order to be the same as in the source file

Add support for functions taking NSError as an out parameter

Assuming we do #276

Change icrate directory layout

Refactor slightly

Refactor file handling to allow for multiple frameworks simultaneously

Put method output inside an extern_methods! call

We'll want this no matter what, since it'll allow us to extend things with availability attributes in the future.

Use extern_methods! functionality

To cut down on the amount of code, which should make things easier to review and understand.

This uses features which are not actually yet done, see #244.

Not happy with the formatting either, but not sure how to fix that?

Manually fix the formatting of attribute macros in extern_methods!

Add AppKit bindings

Speed things up by optionally formatting at the end instead

Prepare for parsing more than one SDK

Specify a minimum deployment target

Document SDK situation

Parse headers on iOS as well

Refactor stmt parsing a bit

Remove Stmt::FileImport and Stmt::ItemImport

These are not nearly enough to make imports work well anyhow, so I'll rip it out and find a better solution

Do preprocessing step explicitly as the first thing

Refactor so that file writing is done using plain Display

Allows us to vastly improve the speed, as well as allowing us to make the output much prettier wrt. newlines and such in the future (proc_macro2 / quote output is not really meant to be consumed by human eyes)

Improve whitespace in generated files and add warning header

Don't crash on unions

Add initial enum parsing

Add initial enum expr parsing

Add very simple enum output

Fix duplicate enum check

Improve enum expr parsing

This should make it easier for things to work on 32-bit platforms

Add static variable parsing

Add a bit of WIP code

Add function parsing

Fix generic struct generation

Make &Class as return type static

Trim unnecessary parentheses

Fix generics default parameter

Remove methods that are both instance and class methods

For now, until we can solve this more generally

Skip protocols that are also classes

Improve imports setups

Bump recursion limit

Add MacTypes.h type translation

Fix int64_t type translation

Make statics public

Fix init methods

Make __inner_extern_class allowing trailing comma in generics

Attempt to improve Rust's parsing speed of icrate

Custom NSObject

TMP import

Remove NSProxy

Temporarily remove out parameter setup

Add struct support

Add partial support for "related result types"

Refactor typedef parsing a bit

Output remaining typedefs

Fix Option<Sel> and *mut bool

Fix almost all remaining type errors in Foundation

Skip statics whoose value we cannot find

Fix anonymous enum types

Fix AppKit duplicate methods

Add CoreData

Properly fix imports

Add `abstract` keyword

Put enum and static declarations behind a macro

Add proper IncompleteArray parsing

Refactor type parsing

Make NSError** handling happen in all the places that it does with Swift

Refactor Ty a bit more

Make Display for RustType always sound

Add support for function pointers

Add support for block pointers

Add extern functions

Emit protocol information

We can't parse it yet though, see #250

Make CoreData compile

Make AppKit compile

Add support for the AuthenticationServices framework

Do clang < v13 workarounds without modifying sources

Refactor Foundation fixes
madsmtm added a commit that referenced this pull request Dec 23, 2022
See full history in 1c4c875.

Add initial header translation

Closer to usable

Mostly works with NSCursor

Mostly works with NSAlert.h

Refactor a bit

AppKit is now parse-able

handle reserved keywords

Handle protocols somewhat

Handle the few remaining entity kinds

Works with Foundation

Cleanup

Refactor

Refactor Method to (almost) be PartialEq

Parse more things

Parse NSConsumed

Verify memory management

More work

Fix reserved keywords

Refactor statements

Add initial availability

Prepare RustType

Split RustType up in parse and ToToken part

Add icrate

Add config files

Temporarily disable protocol generation

Generate files

Add initial generated files for Foundation

Skip "index" header

Add basic imports

Allow skipping methods

Properly emit `unsafe`

Make classes public

Rename objc feature flag

Improve imports somewhat

Further import improvements

Handle basic typedefs

Improve generics handling

Improve pointers to objects

Refactor RustType::TypeDef

Mostly support generics

Refactor config setup

Small fixes

Support nested generics

Comment out a bit of debug logging

Emit all files

Parse sized integer types

Parse typedefs that map to other typedefs

Appease clippy

Add `const`-parsing for RustType::Id

Parse Objective-C in/out/inout/bycopy/byref/oneway qualifiers

Fix `id` being emitted when it actually specifies a protocol

Make AppKit work again

Parse all qualifiers, in particular lifetime qualifiers

More consistent ObjCObjectPointer parsing

Validate some lifetime attributes

Fix out parameters (except for NSError)

Assuming we find a good solution to #277

Refactor Stmt objc declaration parsing

Clean up how return types work

Refactor property parsing

Fixes their order to be the same as in the source file

Add support for functions taking NSError as an out parameter

Assuming we do #276

Change icrate directory layout

Refactor slightly

Refactor file handling to allow for multiple frameworks simultaneously

Put method output inside an extern_methods! call

We'll want this no matter what, since it'll allow us to extend things with availability attributes in the future.

Use extern_methods! functionality

To cut down on the amount of code, which should make things easier to review and understand.

This uses features which are not actually yet done, see #244.

Not happy with the formatting either, but not sure how to fix that?

Manually fix the formatting of attribute macros in extern_methods!

Add AppKit bindings

Speed things up by optionally formatting at the end instead

Prepare for parsing more than one SDK

Specify a minimum deployment target

Document SDK situation

Parse headers on iOS as well

Refactor stmt parsing a bit

Remove Stmt::FileImport and Stmt::ItemImport

These are not nearly enough to make imports work well anyhow, so I'll rip it out and find a better solution

Do preprocessing step explicitly as the first thing

Refactor so that file writing is done using plain Display

Allows us to vastly improve the speed, as well as allowing us to make the output much prettier wrt. newlines and such in the future (proc_macro2 / quote output is not really meant to be consumed by human eyes)

Improve whitespace in generated files and add warning header

Don't crash on unions

Add initial enum parsing

Add initial enum expr parsing

Add very simple enum output

Fix duplicate enum check

Improve enum expr parsing

This should make it easier for things to work on 32-bit platforms

Add static variable parsing

Add a bit of WIP code

Add function parsing

Fix generic struct generation

Make &Class as return type static

Trim unnecessary parentheses

Fix generics default parameter

Remove methods that are both instance and class methods

For now, until we can solve this more generally

Skip protocols that are also classes

Improve imports setups

Bump recursion limit

Add MacTypes.h type translation

Fix int64_t type translation

Make statics public

Fix init methods

Make __inner_extern_class allowing trailing comma in generics

Attempt to improve Rust's parsing speed of icrate

Custom NSObject

TMP import

Remove NSProxy

Temporarily remove out parameter setup

Add struct support

Add partial support for "related result types"

Refactor typedef parsing a bit

Output remaining typedefs

Fix Option<Sel> and *mut bool

Fix almost all remaining type errors in Foundation

Skip statics whoose value we cannot find

Fix anonymous enum types

Fix AppKit duplicate methods

Add CoreData

Properly fix imports

Add `abstract` keyword

Put enum and static declarations behind a macro

Add proper IncompleteArray parsing

Refactor type parsing

Make NSError** handling happen in all the places that it does with Swift

Refactor Ty a bit more

Make Display for RustType always sound

Add support for function pointers

Add support for block pointers

Add extern functions

Emit protocol information

We can't parse it yet though, see #250

Make CoreData compile

Make AppKit compile

Add support for the AuthenticationServices framework

Do clang < v13 workarounds without modifying sources

Refactor Foundation fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-objc2 Affects the `objc2`, `objc2-exception-helper` and/or `objc2-encode` crates enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants