Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tree/Recursive struct support in thrift #84

Closed
wants to merge 6 commits into from

Conversation

djwatson
Copy link

Patches to implement tree/list/co-recursive structures in thrift, a la protobufs.

@jfarrell
Copy link
Contributor

Thanks @djwatson created THRIFT-2421 to track it and will review

Dave Watson added 6 commits March 26, 2014 08:53
A common complaint is that you can't express trees or other recursive structures in thrift easily - unlike protobufs. This diff loosens up the parser to allow using structs before they are defined (and uses typedef as a forward declaration).
This diff is actually enough to make recursive types work for some dyamic languages (I tried php, works out of the box!)

Other languages will need forward declarations, or ways to box types, to make this work (i.e. C++ needs both forward decls and a way to express structs as pointers)
For cpp added a cpp.ref annotation.  Users will have to make some recursive types use references to avoid a struct containing itself.  Also had to explicitly check for null references when reading/writing, so we know to set the reference to null.

Merge from fb's branch of facebook/fbthrift@9b36d51

Fb uses C++11 extensively, this merge changes the type to a raw pointer and adds deletes where necessary.  This is *very* similar to how protobufs does it.
Hand indent to make code easier to read
Move some funcs to .cpp file to avoid compilation errors using forward declares
@jfarrell jfarrell closed this in e0e8316 Apr 9, 2014
@snikulov
Copy link
Contributor

snikulov commented Apr 9, 2014

This change breaks c++ generation.

After this change in compiler/cpp/src/generate/t_cpp_generator.cc

  •  indent() << "virtual ~" << tstruct->get_name() << "() throw() {}" << endl << endl;
    
  •  indent() << "virtual ~" << tstruct->get_name() << "() throw();" << endl;
    

linker complains about missed destructor.

@djwatson
Copy link
Author

djwatson commented Apr 9, 2014

The destructor is defined in the .cpp file now:

// Destructor

  • if (tstruct->annotations_.find("final") == tstruct->annotations_.end()) {
  • out <<
  •  endl <<
    
  •  indent() << tstruct->get_name() << "::~" << tstruct->get_name() << "() throw() {" << endl;
    
  • indent_up();
  • for (m_iter = members.begin(); m_iter != members.end(); ++m_iter) {
  •  if (is_reference(*m_iter)) {
    
  • out << indent() <<
  • "delete " << (*m_iter)->get_name() << ";" << endl;
  •  }
    
  • }

Is your _types.cpp linked in? Or other repro?

@ben-craig
Copy link
Contributor

There's a log of a failure in Thrift's nightly build.

https://builds.apache.org/job/Thrift/1123/console

tutorial/shared.thrift and tutorial/tutorial.thrift end up causing problems. The relevant Makefile.am appears to be building in the _types.cpp. I have not looked at the generated source. My current suspicion is that the "final" attribute check is causing problems, but that is currently just a hunch.

@henrique
Copy link
Contributor

henrique commented Apr 9, 2014

Hi Dave,
This destructor is only for structs, isn't it?
The ld error is for *_presult and *_pargs

@snikulov
Copy link
Contributor

snikulov commented Apr 9, 2014

@djwatson Yes, my _types.cpp is linked in.
But as @henrique mentioned _presult and _pargs destructor's are not generated and thus linker gives error.

@djwatson
Copy link
Author

djwatson commented Apr 9, 2014

Thanks guys, fix: #98

allengeorge pushed a commit to allengeorge/thrift that referenced this pull request Jan 1, 2017
Client: cpp
Patch:  Dave Watson

Github Pull Request: This closes apache#84
----
commit b6134ce
Date:   2014-03-20T18:12:04Z

    Recursive structs support in parser

    A common complaint is that you can't express trees or other recursive structures in thrift easily - unlike protobufs. This diff loosens up the parser to allow using structs before they are defined (and uses typedef as a forward declaration).
    This diff is actually enough to make recursive types work for some dyamic languages (I tried php, works out of the box!)

    Other languages will need forward declarations, or ways to box types, to make this work (i.e. C++ needs both forward decls and a way to express structs as pointers)
@ringerc
Copy link

ringerc commented Jan 31, 2018

I see a bunch of mutual recursive structure support in the tests, but nothing for an optional single self-referential member, like

struct RecSelf {
   1: i16 item
   2: optional RecSelf 
 }

Thift does not appear to generate sensible code for this. The example I am working with is from the opentracing jaeger toolset, namely the IDL

struct Downstream {
    1: required string serviceName
    2: required string serverRole
    3: required string host
    4: required string port
    5: required Transport transport
    6: optional Downstream downstream
}

This works fine in Java, Go, etc. But in C++ it produces the nonsensical code

class Downstream {
 public:
 
  /* blah elided blah */

  virtual ~Downstream() throw();
  std::string serviceName;
  std::string serverRole;
  std::string host;
  std::string port;
  Transport::type transport;
  Downstream downstream;

  _Downstream__isset __isset;

  /* blah elided blah */
};

in which the class is a by-value member of its self.

I think the ideal solution here would be to adopt something like std::optional (c++17 proposal) as a thrift::optional and use that instead of the __isset stuff for optional members. But backwards compatibility could be a problem.

@ringerc
Copy link

ringerc commented Jan 31, 2018

@ringerc
Copy link

ringerc commented Feb 1, 2018

Anyone else facing that issue: workaround is to declare a reference:

struct RecSelf {
   1: i16 item
   2: optional RecSelf &self
 }

Note the &.

Jens-G pushed a commit to Jens-G/thrift that referenced this pull request Apr 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants