Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Relay][VM][Interpreter] Enable first-class constructors in VM and interpreter via eta expansion #4218

Merged
merged 9 commits into from
Nov 15, 2019

Conversation

weberlo
Copy link
Contributor

@weberlo weberlo commented Oct 30, 2019

Previously, the VM and interpreter did not support constructors as first-class objects. For example, the following code snippet would not compile:

def @main[A]() -> (fn(A, List[A]) -> List[A]) {
  Cons
}

. The eta expansion pass in this PR remedies this problem by wrapping the constructor in a function, like so:

def @main[A]() -> (fn(A, List[A]) -> List[A]) {
  fn (%x: A, %xs: List[A]) -> List[A] {
    Cons(%x, %xs)  
  }
}

. There was an existing eta expansion pass, but it was not recursive and only operated on global variables. The new pass is configurable, so you can choose to expand constructors, global variables, or both.

Also included are the following minor changes:

  • Fix LambdaLift pass so it closes over free type variables.
  • Fix LambdaLift pass so it reuses lifted functions that are alpha equal.
  • Fix pretty printing of constructors to be context-sensitive (e.g., print Cons(A, List[A]) in a type definition but only print Cons when in an expr).
  • Rename Module::HasDef to Module::ContainGlobalTypeVar to match the analogous Module::ContainGlobalVar method.
  • Change prelude definitions to use first-class constructors where applicable.

CC @jroesch @MarisaKirisame @icemelon9

Copy link
Contributor

@MarisaKirisame MarisaKirisame left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@weberlo weberlo changed the title [Relay][VM] Enable VM constructor compilation via eta expansion [Relay][VM] Enable first-class constructors in VM and interpreter via eta expansion Nov 4, 2019
@weberlo weberlo changed the title [Relay][VM] Enable first-class constructors in VM and interpreter via eta expansion [Relay][VM][Interpreter] Enable first-class constructors in VM and interpreter via eta expansion Nov 5, 2019
@weberlo
Copy link
Contributor Author

weberlo commented Nov 11, 2019

@jroesch @icemelon9 CI is green. Take a look when you get a chance.

Copy link
Member

@icemelon icemelon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, just one minor comment

src/relay/pass/eta_expand.cc Outdated Show resolved Hide resolved
@weberlo
Copy link
Contributor Author

weberlo commented Nov 13, 2019

It looks like the tests are timing out. I think this addition to the interpreter is the cause. Might need to roll back that change and the change to the prelude, but keep the EtaExpand functionality for the VM. If we want first-class constructors in the interpreter, we'll need to directly support it there.

@weberlo
Copy link
Contributor Author

weberlo commented Nov 14, 2019

nvm. there seem to be CI timeouts repo-wide (https://discuss.tvm.ai/t/remove-unused-function-in-relay-module/4753)

@weberlo
Copy link
Contributor Author

weberlo commented Nov 15, 2019

@icemelon9 CI is green again

@icemelon icemelon merged commit 2c5c4da into apache:master Nov 15, 2019
@icemelon
Copy link
Member

Thanks @weberlo @MarisaKirisame. This is now merged.

zxy844288792 pushed a commit to zxy844288792/tvm that referenced this pull request Nov 15, 2019
…terpreter via eta expansion (apache#4218)

* Fix constructor pretty printing

* Make Module::HasDef name consistent with API

* Add VM constructor compilation via eta expansion

* Lint

* Fix CI

* Fix failing test

* Address comment

* Retrigger CI

* Retrigger CI
zxy844288792 pushed a commit to zxy844288792/tvm that referenced this pull request Nov 15, 2019
…terpreter via eta expansion (apache#4218)

* Fix constructor pretty printing

* Make Module::HasDef name consistent with API

* Add VM constructor compilation via eta expansion

* Lint

* Fix CI

* Fix failing test

* Address comment

* Retrigger CI

* Retrigger CI
kevinthesun pushed a commit to neo-ai/tvm that referenced this pull request Nov 25, 2019
* [TOPI][OP] Support Faster-RCNN Proposal OP on CPU (apache#4297)

* Support Proposal operator on CPU.

* PyLint space issue

* PyLint space issue

* Pylint singleton-comparison issue

* [QNN][Legalize] Specialize for Platforms without any fast Int8 arithmetic units. (apache#4307)

* fix error when memory_id is VTA_MEM_ID_OUT (apache#4330)

* [CI][DOCKER] Add ONNX runtime dep (apache#4314)

* [DOCKER] Add ONNX runtime dep

* Improve ci script

* [QNN] Quantize - Fixing the sequence of lowering. (apache#4316)

* [QNN] Use Int16 upcast in Fallback Conv2D. Fix test names. (apache#4329)

* [doc][fix] fix sphinx parsing for pass infra tutorial (apache#4337)

* change ci image version (apache#4313)

* [Codegen] remove fp16 function override for cuda  (apache#4331)

* add volatile override back

* [codegen] remove fp16 function override for cuda

* [CI] Set workspace to be per executor (apache#4336)

* [Build][Windows] Fix Windows build by including cctype (apache#4319)

* Fix build

* dummy change to retrigger CI

* dummy change to retrigger ci

* dummy change to retrigger ci

* Enable hipModuleGetGlobal() (apache#4321)

* [Relay][Pass] Add pass to remove unused functions in relay module (apache#4334)

* [Relay][Pass] Add pass to remove unused functions in relay module

* Add tests

* Fix lint

* Fix visit order

* Add pass argument

* Fix

* Add support for quant. mul operator in tflite frontend (apache#4283)

A test for qnn_mul has to be added when the qnn elemwise tests (apache#4282) get merged.

* Add topi.nn.fifo_buffer to TVM doc (apache#4343)

* Solve custom model of prelu (apache#4326)

* Deprecate NNVM warning msg (apache#4333)

* [Contrib] Add MKL DNN option (apache#4323)

* [Contrib] Add MKL DNN

* update

* update

* [Relay][Frontend][TF] Fix transpose when axes is not a param (apache#4327)

* [Relay][Frontend][TF] Use _infer_value_simulated when axes is not a const to Transpose

* uncomment tests

* dummy change to retrigger ci

* [RUNTIME] Add device query for AMD GcnArch (apache#4341)

* add gcnArch query

* kGcnArch query for cuda is a no-op

* [Test][Relay][Pass] Add test case for lambda lift (apache#4317)

* [Relay][Frontend][ONNX] operator support: DepthToSpace, SpaceToDepth (apache#4271)

* imp module is deprecated (apache#4275)

* [VTA] Bug fix for padded load with large inputs (apache#4293)

* bug fix for padded load with large inputs

* Update TensorLoad.scala

* Update test_vta_insn.py

* fix inconsistent tag name (apache#4134)

* [CodeGen] Add build config option disable_assert to control whether to generate assert (apache#4340)

* Bump up CUDA log version in tophub.py (apache#4347)

* Add check to ensure input file was successfully opened in NNVM deploy code demo (apache#4315)

* [COMMUNITY] Add DISCLAIMER, KEYS for ASF release (apache#4345)

* [COMMUNITY] Add DISCLAIMER, KEYS for ASF release

* Add file name spec

* [Relay][VM][Interpreter] Enable first-class constructors in VM and interpreter via eta expansion (apache#4218)

* Fix constructor pretty printing

* Make Module::HasDef name consistent with API

* Add VM constructor compilation via eta expansion

* Lint

* Fix CI

* Fix failing test

* Address comment

* Retrigger CI

* Retrigger CI

* Update dmlc_tvm_commit_id.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants