Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: remove deploy instruction from venom #3703

Merged
Show file tree
Hide file tree
Changes from 51 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
65553b0
Add method to add instruction to IRBasicBlock
harkal Dec 7, 2023
29d5447
Refactor add_instruction_no_return and
harkal Dec 8, 2023
5f8a9b3
refactor to use new methods in bb
harkal Dec 8, 2023
cc0ce9d
refactor exit_to
harkal Dec 8, 2023
6ec4ecf
finish phase out direct ctx instruction adding
harkal Dec 8, 2023
24c328e
Remove append_instruction method from IRFunction
harkal Dec 8, 2023
7e09a3f
refactor basic block instruction appending
harkal Dec 8, 2023
86d8fbe
Refactor basic block instruction appending names
harkal Dec 8, 2023
53f2322
left out commit
harkal Dec 8, 2023
f03986f
"naming things" refactor
harkal Dec 8, 2023
1ae9bc1
Refactor multi-entry block test cases for venom
harkal Dec 9, 2023
020c105
Refactor test_duplicate_operands
harkal Dec 9, 2023
7555caf
Make Venom log instruction
harkal Dec 9, 2023
ad10832
Refactor-out instruction output
harkal Dec 9, 2023
87ee7bb
Update tests after output property removal
harkal Dec 9, 2023
ff998ef
Automakit IRLiteral inferance
harkal Dec 10, 2023
7df1e76
Pass jump targets as metadata to the jump IRNode
harkal Dec 11, 2023
f416d15
Eliminate special cfg for O(1) dispatcher
harkal Dec 11, 2023
32d3fce
Add test for multi-entry block with dynamic jump
harkal Dec 11, 2023
bb6165f
Refactor basic block splitting logic in
harkal Dec 11, 2023
023dd05
Add replace_label_operants() method
harkal Dec 11, 2023
b27902e
finalize split basicblock insertions
harkal Dec 11, 2023
5ffa6e7
fix import order in normalization pass
harkal Dec 11, 2023
f12f0dc
Cleanup IRLiteral convertions
harkal Dec 12, 2023
e9d7030
Merge branch 'master' into feature/jump_table
harkal Dec 13, 2023
8836a4c
add small comment
harkal Dec 13, 2023
dde51e2
djump instruction
harkal Dec 17, 2023
7c75da5
Move experimental_codegen in settings
harkal Dec 17, 2023
bbd1e04
Merge branch 'master' into feature/jump_table
harkal Dec 17, 2023
65baf5b
Disable ir_dict output type for venom
harkal Dec 18, 2023
9b45270
Merge branch 'fix/expcodegen_after_modules_merge' into feature/jump_t…
harkal Dec 18, 2023
7d961c6
experimental codegen flag passing
harkal Dec 18, 2023
937d5cc
rename "djump" to "mjump"
charles-cooper Dec 19, 2023
20d28e6
add mjmp to venom
charles-cooper Dec 19, 2023
9bf661c
rename mjump->djump, mjmp->djmp
charles-cooper Dec 19, 2023
c9dd075
update a comment
charles-cooper Dec 19, 2023
d7f3ac1
Default experimental codegen setting is None
harkal Dec 20, 2023
690b9f9
Add entry_points to IRFunction class
harkal Dec 20, 2023
051fe80
Fix _append_return_for_stack_operand's arguments
harkal Dec 20, 2023
ec00dfa
Add and remove entry points in IRFunction class
harkal Dec 20, 2023
b119518
Update global label and add runtime entry point
harkal Dec 20, 2023
60558a4
Refactor Venom code generation and add postample instructions
harkal Dec 20, 2023
d2c74d4
Add ctor_mem_size and immutables_len to IRFunction
harkal Dec 20, 2023
19bc412
Disable deploy logic in ir_node_to_venom.py
harkal Dec 20, 2023
56bfd9c
Refactor generate_assembly_experimental function signature
harkal Dec 20, 2023
9b90ac4
Refactor VenomCompiler class to support multiple contexts
harkal Dec 20, 2023
610beaf
Fix generate_assembly_experimental arguments in test_duplicate_operands
harkal Dec 20, 2023
f4516ea
Remove "deploy" opcode from basicblock.py and venom_to_assembly.py
harkal Dec 20, 2023
a471864
Remove deprecated code for deploy handling "hack"
harkal Dec 20, 2023
698ec0f
Update type annotations in venom module
harkal Dec 20, 2023
25e764c
Inline postable insertion
harkal Dec 20, 2023
6cc9401
Fix deploy_code initialization in convert_ir_basicblock function
harkal Dec 20, 2023
6ab1199
sad :(
harkal Dec 20, 2023
0dc88f0
Merge branch 'master' into feature/venom_deploy_instruction_removal
harkal Dec 21, 2023
d075ff4
remove duplicate test
harkal Dec 21, 2023
59abf8d
Refactor to handle None values in glue code not in VenomCompiler
harkal Dec 23, 2023
dcf14c5
goosfraba
harkal Dec 23, 2023
9c5b80d
API cleanup
charles-cooper Jan 2, 2024
6fa7dac
Merge branch 'feature/venom_deploy_instruction_removal' of github.com…
charles-cooper Jan 2, 2024
8db8e29
whitespace
charles-cooper Jan 2, 2024
7ac92d8
rename some variables with edit distance == 1
charles-cooper Jan 2, 2024
88f1fa3
add some variables
charles-cooper Jan 2, 2024
8f93bf1
assert message
charles-cooper Jan 2, 2024
fd78df9
fix minor bugs
charles-cooper Jan 2, 2024
fe0e154
fix cfg normalization code
charles-cooper Jan 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tests/unit/compiler/venom/test_duplicate_operands.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ def test_duplicate_operands():
bb.append_instruction("mul", sum, op)
bb.append_instruction("stop")

asm = generate_assembly_experimental(ctx, OptimizationLevel.CODESIZE)
asm = generate_assembly_experimental((None, ctx), OptimizationLevel.CODESIZE)

assert asm == ["PUSH1", 10, "DUP1", "DUP1", "DUP1", "ADD", "MUL", "STOP", "REVERT"]
45 changes: 43 additions & 2 deletions tests/unit/compiler/venom/test_multi_entry_block.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ def test_multi_entry_block_1():
finish_bb = ctx.get_basic_block(finish_label.value)
cfg_in = list(finish_bb.cfg_in.keys())
assert cfg_in[0].label.value == "target", "Should contain target"
assert cfg_in[1].label.value == "finish_split_global", "Should contain finish_split_global"
assert cfg_in[1].label.value == "finish_split___global", "Should contain finish_split___global"
assert cfg_in[2].label.value == "finish_split_block_1", "Should contain finish_split_block_1"


Expand Down Expand Up @@ -93,5 +93,46 @@ def test_multi_entry_block_2():
finish_bb = ctx.get_basic_block(finish_label.value)
cfg_in = list(finish_bb.cfg_in.keys())
assert cfg_in[0].label.value == "target", "Should contain target"
assert cfg_in[1].label.value == "finish_split_global", "Should contain finish_split_global"
assert cfg_in[1].label.value == "finish_split___global", "Should contain finish_split___global"
assert cfg_in[2].label.value == "finish_split_block_1", "Should contain finish_split_block_1"


def test_multi_entry_block_with_dynamic_jump():
ctx = IRFunction()

finish_label = IRLabel("finish")
target_label = IRLabel("target")
block_1_label = IRLabel("block_1", ctx)

bb = ctx.get_basic_block()
op = bb.append_instruction("store", 10)
acc = bb.append_instruction("add", op, op)
bb.append_instruction("djmp", acc, finish_label, block_1_label)

block_1 = IRBasicBlock(block_1_label, ctx)
ctx.append_basic_block(block_1)
acc = block_1.append_instruction("add", acc, op)
op = block_1.append_instruction("store", 10)
block_1.append_instruction("mstore", acc, op)
block_1.append_instruction("jnz", acc, finish_label, target_label)

target_bb = IRBasicBlock(target_label, ctx)
ctx.append_basic_block(target_bb)
target_bb.append_instruction("mul", acc, acc)
target_bb.append_instruction("jmp", finish_label)

finish_bb = IRBasicBlock(finish_label, ctx)
ctx.append_basic_block(finish_bb)
finish_bb.append_instruction("stop")

calculate_cfg(ctx)
assert not ctx.normalized, "CFG should not be normalized"

NormalizationPass.run_pass(ctx)
assert ctx.normalized, "CFG should be normalized"

finish_bb = ctx.get_basic_block(finish_label.value)
cfg_in = list(finish_bb.cfg_in.keys())
assert cfg_in[0].label.value == "target", "Should contain target"
assert cfg_in[1].label.value == "finish_split___global", "Should contain finish_split___global"
assert cfg_in[2].label.value == "finish_split_block_1", "Should contain finish_split_block_1"
7 changes: 4 additions & 3 deletions vyper/cli/vyper_compile.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ def _parse_args(argv):
"--experimental-codegen",
help="The compiler use the new IR codegen. This is an experimental feature.",
action="store_true",
dest="experimental_codegen",
)

args = parser.parse_args(argv)
Expand Down Expand Up @@ -184,6 +185,9 @@ def _parse_args(argv):
if args.evm_version:
settings.evm_version = args.evm_version

if args.experimental_codegen:
settings.experimental_codegen = args.experimental_codegen

if args.verbose:
print(f"cli specified: `{settings}`", file=sys.stderr)

Expand All @@ -195,7 +199,6 @@ def _parse_args(argv):
settings,
args.storage_layout,
args.no_bytecode_metadata,
args.experimental_codegen,
)

if args.output_path:
Expand Down Expand Up @@ -233,7 +236,6 @@ def compile_files(
settings: Optional[Settings] = None,
storage_layout_paths: list[str] = None,
no_bytecode_metadata: bool = False,
experimental_codegen: bool = False,
) -> dict:
paths = paths or []

Expand Down Expand Up @@ -287,7 +289,6 @@ def compile_files(
storage_layout_override=storage_layout_override,
show_gas_estimates=show_gas_estimates,
no_bytecode_metadata=no_bytecode_metadata,
experimental_codegen=experimental_codegen,
)

ret[file_path] = output
Expand Down
2 changes: 2 additions & 0 deletions vyper/codegen/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -892,6 +892,8 @@ def make_setter(left, right):
_opt_level = OptimizationLevel.GAS


# FIXME: this is to get around the fact that we don't have a
# proper context object in the IR generation phase.
@contextlib.contextmanager
def anchor_opt_level(new_level: OptimizationLevel) -> Generator:
"""
Expand Down
18 changes: 10 additions & 8 deletions vyper/codegen/module.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,21 +311,23 @@ def _selector_section_sparse(external_functions, module_ctx):

ret.append(["codecopy", dst, bucket_hdr_location, SZ_BUCKET_HEADER])

jumpdest = IRnode.from_list(["mload", 0])
# don't particularly like using `jump` here since it can cause
# issues for other backends, consider changing `goto` to allow
# dynamic jumps, or adding some kind of jumptable instruction
ret.append(["jump", jumpdest])
jump_targets = []

jumptable_data = ["data", "selector_buckets"]
for i in range(n_buckets):
if i in buckets:
bucket_label = f"selector_bucket_{i}"
jumptable_data.append(["symbol", bucket_label])
jump_targets.append(bucket_label)
else:
# empty bucket
jumptable_data.append(["symbol", "fallback"])
jump_targets.append("fallback")

jumptable_data = ["data", "selector_buckets"]
jumptable_data.extend(["symbol", label] for label in jump_targets)

jumpdest = IRnode.from_list(["mload", 0])

jump_instr = IRnode.from_list(["djump", jumpdest, *jump_targets])
ret.append(jump_instr)
ret.append(jumptable_data)

for bucket_id, bucket in buckets.items():
Expand Down
2 changes: 0 additions & 2 deletions vyper/compiler/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@ def compile_from_file_input(
no_bytecode_metadata: bool = False,
show_gas_estimates: bool = False,
exc_handler: Optional[Callable] = None,
experimental_codegen: bool = False,
) -> dict:
"""
Main entry point into the compiler.
Expand Down Expand Up @@ -107,7 +106,6 @@ def compile_from_file_input(
storage_layout_override,
show_gas_estimates,
no_bytecode_metadata,
experimental_codegen,
)

ret = {}
Expand Down
3 changes: 3 additions & 0 deletions vyper/compiler/output.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,9 @@ def build_ir_runtime_output(compiler_data: CompilerData) -> IRnode:


def _ir_to_dict(ir_node):
# Currently only supported with IRnode and not VenomIR
if not isinstance(ir_node, IRnode):
return
args = ir_node.args
if len(args) > 0 or ir_node.value == "seq":
return {ir_node.value: [_ir_to_dict(x) for x in args]}
Expand Down
35 changes: 24 additions & 11 deletions vyper/compiler/phases.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,6 @@ def __init__(
storage_layout: StorageLayout = None,
show_gas_estimates: bool = False,
no_bytecode_metadata: bool = False,
experimental_codegen: bool = False,
) -> None:
"""
Initialization method.
Expand All @@ -76,11 +75,9 @@ def __init__(
Show gas estimates for abi and ir output modes
no_bytecode_metadata: bool, optional
Do not add metadata to bytecode. Defaults to False
experimental_codegen: bool, optional
Use experimental codegen. Defaults to False
"""
# to force experimental codegen, uncomment:
# experimental_codegen = True
# settings.experimental_codegen = True

if isinstance(file_input, str):
file_input = FileInput(
Expand All @@ -93,7 +90,6 @@ def __init__(
self.storage_layout_override = storage_layout
self.show_gas_estimates = show_gas_estimates
self.no_bytecode_metadata = no_bytecode_metadata
self.experimental_codegen = experimental_codegen
self.settings = settings or Settings()
self.input_bundle = input_bundle or FilesystemInputBundle([Path(".")])

Expand Down Expand Up @@ -142,6 +138,18 @@ def _generate_ast(self):
)
self.settings.optimize = settings.optimize

if settings.experimental_codegen is not None:
if (
self.settings.experimental_codegen is not None
and self.settings.experimental_codegen != settings.experimental_codegen
):
raise StructureException(
f"compiler options indicate experimental codegen mode "
f"{self.settings.experimental_codegen}, "
f"but source pragma indicates {settings.experimental_codegen}."
)
self.settings.experimental_codegen = settings.experimental_codegen

# ensure defaults
if self.settings.optimize is None:
self.settings.optimize = OptimizationLevel.default()
Expand Down Expand Up @@ -184,9 +192,12 @@ def global_ctx(self) -> ModuleT:
@cached_property
def _ir_output(self):
# fetch both deployment and runtime IR
nodes = generate_ir_nodes(self.global_ctx, self.settings.optimize)
if self.experimental_codegen:
return [generate_ir(nodes[0]), generate_ir(nodes[1])]
nodes = generate_ir_nodes(
self.global_ctx, self.settings.optimize, self.settings.experimental_codegen
)
if self.settings.experimental_codegen:
ir, ir_runtime = generate_ir(nodes[0])
return [(ir, ir_runtime), (None, ir_runtime)]
else:
return nodes

Expand All @@ -211,7 +222,7 @@ def function_signatures(self) -> dict[str, ContractFunctionT]:

@cached_property
def assembly(self) -> list:
if self.experimental_codegen:
if self.settings.experimental_codegen:
return generate_assembly_experimental(
self.ir_nodes, self.settings.optimize # type: ignore
)
Expand All @@ -220,7 +231,7 @@ def assembly(self) -> list:

@cached_property
def assembly_runtime(self) -> list:
if self.experimental_codegen:
if self.settings.experimental_codegen:
return generate_assembly_experimental(
self.ir_runtime, self.settings.optimize # type: ignore
)
Expand Down Expand Up @@ -294,7 +305,9 @@ def generate_folded_ast(
return vyper_module_folded, symbol_tables


def generate_ir_nodes(global_ctx: ModuleT, optimize: OptimizationLevel) -> tuple[IRnode, IRnode]:
def generate_ir_nodes(
global_ctx: ModuleT, optimize: OptimizationLevel, experimental_codegen: bool
) -> tuple[IRnode, IRnode]:
"""
Generate the intermediate representation (IR) from the contextualized AST.

Expand Down
1 change: 1 addition & 0 deletions vyper/compiler/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ class Settings:
compiler_version: Optional[str] = None
optimize: Optional[OptimizationLevel] = None
evm_version: Optional[str] = None
experimental_codegen: Optional[bool] = None


_DEBUG = False
Expand Down
7 changes: 7 additions & 0 deletions vyper/ir/compile_ir.py
Original file line number Diff line number Diff line change
Expand Up @@ -702,6 +702,13 @@ def _height_of(witharg):
o.extend(_compile_to_assembly(c, withargs, existing_labels, break_dest, height + i))
o.extend(["_sym_" + code.args[0].value, "JUMP"])
return o
elif code.value == "djump":
o = []
# "djump" compiles to a raw EVM jump instruction
jump_target = code.args[0]
o.extend(_compile_to_assembly(jump_target, withargs, existing_labels, break_dest, height))
o.append("JUMP")
return o
# push a literal symbol
elif code.value == "symbol":
return ["_sym_" + code.args[0].value]
Expand Down
1 change: 1 addition & 0 deletions vyper/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,7 @@ class SizeLimits:
"with",
"label",
"goto",
"djump", # "dynamic jump", i.e. constrained, multi-destination jump
"~extcode",
"~selfcode",
"~calldata",
Expand Down
25 changes: 16 additions & 9 deletions vyper/venom/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# maybe rename this `main.py` or `venom.py`
# (can have an `__init__.py` which exposes the API).

from typing import Optional
from typing import Any, Optional

from vyper.codegen.ir_node import IRnode
from vyper.compiler.settings import OptimizationLevel
Expand All @@ -19,17 +19,14 @@


def generate_assembly_experimental(
ctx: IRFunction, optimize: Optional[OptimizationLevel] = None
ctxs: tuple[IRFunction, IRFunction], optimize: Optional[OptimizationLevel] = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should either be a list of IRFunctions or specify by names -- runtime_ir and deploy_ir.

) -> list[str]:
compiler = VenomCompiler(ctx)
compiler = VenomCompiler(list(ctxs))
return compiler.generate_evm(optimize is OptimizationLevel.NONE)


def generate_ir(ir: IRnode, optimize: Optional[OptimizationLevel] = None) -> IRFunction:
# Convert "old" IR to "new" IR
ctx = convert_ir_basicblock(ir)

# Run passes on "new" IR
def _run_passes(ctx: IRFunction, optimize: Optional[OptimizationLevel] = None) -> None:
# Run passes on Venom IR
# TODO: Add support for optimization levels
while True:
changes = 0
Expand All @@ -53,4 +50,14 @@
if changes == 0:
break

return ctx

def generate_ir(
ir: IRnode, optimize: Optional[OptimizationLevel] = None
) -> tuple[IRFunction, IRFunction]:
# Convert "old" IR to "new" IR
ctx, ctx_runtime = convert_ir_basicblock(ir)

_run_passes(ctx, optimize)
Fixed Show fixed Hide fixed
_run_passes(ctx_runtime, optimize)
Fixed Show fixed Hide fixed

return ctx, ctx_runtime
30 changes: 0 additions & 30 deletions vyper/venom/analysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,36 +19,6 @@ def calculate_cfg(ctx: IRFunction) -> None:
bb.cfg_out = OrderedSet()
bb.out_vars = OrderedSet()

# TODO: This is a hack to support the old IR format where `deploy` is
# an instruction. in the future we should have two entry points, one
# for the initcode and one for the runtime code.
deploy_bb = None
after_deploy_bb = None
for i, bb in enumerate(ctx.basic_blocks):
if bb.instructions[0].opcode == "deploy":
deploy_bb = bb
after_deploy_bb = ctx.basic_blocks[i + 1]
break

if deploy_bb is not None:
assert after_deploy_bb is not None, "No block after deploy block"
entry_block = after_deploy_bb
has_constructor = ctx.basic_blocks[0].instructions[0].opcode != "deploy"
if has_constructor:
deploy_bb.add_cfg_in(ctx.basic_blocks[0])
entry_block.add_cfg_in(deploy_bb)
else:
entry_block = ctx.basic_blocks[0]

# TODO: Special case for the jump table of selector buckets and fallback.
# this will be cleaner when we introduce an "indirect jump" instruction
# for the selector table (which includes all possible targets). it will
# also clean up the code for normalization because it will not have to
# handle this case specially.
for bb in ctx.basic_blocks:
if "selector_bucket_" in bb.label.value or bb.label.value == "fallback":
bb.add_cfg_in(entry_block)

for bb in ctx.basic_blocks:
assert len(bb.instructions) > 0, "Basic block should not be empty"
last_inst = bb.instructions[-1]
Expand Down
Loading
Loading