Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: remove deploy instruction from venom #3703

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
65553b0
Add method to add instruction to IRBasicBlock
harkal Dec 7, 2023
29d5447
Refactor add_instruction_no_return and
harkal Dec 8, 2023
5f8a9b3
refactor to use new methods in bb
harkal Dec 8, 2023
cc0ce9d
refactor exit_to
harkal Dec 8, 2023
6ec4ecf
finish phase out direct ctx instruction adding
harkal Dec 8, 2023
24c328e
Remove append_instruction method from IRFunction
harkal Dec 8, 2023
7e09a3f
refactor basic block instruction appending
harkal Dec 8, 2023
86d8fbe
Refactor basic block instruction appending names
harkal Dec 8, 2023
53f2322
left out commit
harkal Dec 8, 2023
f03986f
"naming things" refactor
harkal Dec 8, 2023
1ae9bc1
Refactor multi-entry block test cases for venom
harkal Dec 9, 2023
020c105
Refactor test_duplicate_operands
harkal Dec 9, 2023
7555caf
Make Venom log instruction
harkal Dec 9, 2023
ad10832
Refactor-out instruction output
harkal Dec 9, 2023
87ee7bb
Update tests after output property removal
harkal Dec 9, 2023
ff998ef
Automakit IRLiteral inferance
harkal Dec 10, 2023
7df1e76
Pass jump targets as metadata to the jump IRNode
harkal Dec 11, 2023
f416d15
Eliminate special cfg for O(1) dispatcher
harkal Dec 11, 2023
32d3fce
Add test for multi-entry block with dynamic jump
harkal Dec 11, 2023
bb6165f
Refactor basic block splitting logic in
harkal Dec 11, 2023
023dd05
Add replace_label_operants() method
harkal Dec 11, 2023
b27902e
finalize split basicblock insertions
harkal Dec 11, 2023
5ffa6e7
fix import order in normalization pass
harkal Dec 11, 2023
f12f0dc
Cleanup IRLiteral convertions
harkal Dec 12, 2023
e9d7030
Merge branch 'master' into feature/jump_table
harkal Dec 13, 2023
8836a4c
add small comment
harkal Dec 13, 2023
dde51e2
djump instruction
harkal Dec 17, 2023
7c75da5
Move experimental_codegen in settings
harkal Dec 17, 2023
bbd1e04
Merge branch 'master' into feature/jump_table
harkal Dec 17, 2023
65baf5b
Disable ir_dict output type for venom
harkal Dec 18, 2023
9b45270
Merge branch 'fix/expcodegen_after_modules_merge' into feature/jump_t…
harkal Dec 18, 2023
7d961c6
experimental codegen flag passing
harkal Dec 18, 2023
937d5cc
rename "djump" to "mjump"
charles-cooper Dec 19, 2023
20d28e6
add mjmp to venom
charles-cooper Dec 19, 2023
9bf661c
rename mjump->djump, mjmp->djmp
charles-cooper Dec 19, 2023
c9dd075
update a comment
charles-cooper Dec 19, 2023
d7f3ac1
Default experimental codegen setting is None
harkal Dec 20, 2023
690b9f9
Add entry_points to IRFunction class
harkal Dec 20, 2023
051fe80
Fix _append_return_for_stack_operand's arguments
harkal Dec 20, 2023
ec00dfa
Add and remove entry points in IRFunction class
harkal Dec 20, 2023
b119518
Update global label and add runtime entry point
harkal Dec 20, 2023
60558a4
Refactor Venom code generation and add postample instructions
harkal Dec 20, 2023
d2c74d4
Add ctor_mem_size and immutables_len to IRFunction
harkal Dec 20, 2023
19bc412
Disable deploy logic in ir_node_to_venom.py
harkal Dec 20, 2023
56bfd9c
Refactor generate_assembly_experimental function signature
harkal Dec 20, 2023
9b90ac4
Refactor VenomCompiler class to support multiple contexts
harkal Dec 20, 2023
610beaf
Fix generate_assembly_experimental arguments in test_duplicate_operands
harkal Dec 20, 2023
f4516ea
Remove "deploy" opcode from basicblock.py and venom_to_assembly.py
harkal Dec 20, 2023
a471864
Remove deprecated code for deploy handling "hack"
harkal Dec 20, 2023
698ec0f
Update type annotations in venom module
harkal Dec 20, 2023
25e764c
Inline postable insertion
harkal Dec 20, 2023
6cc9401
Fix deploy_code initialization in convert_ir_basicblock function
harkal Dec 20, 2023
6ab1199
sad :(
harkal Dec 20, 2023
0dc88f0
Merge branch 'master' into feature/venom_deploy_instruction_removal
harkal Dec 21, 2023
d075ff4
remove duplicate test
harkal Dec 21, 2023
59abf8d
Refactor to handle None values in glue code not in VenomCompiler
harkal Dec 23, 2023
dcf14c5
goosfraba
harkal Dec 23, 2023
9c5b80d
API cleanup
charles-cooper Jan 2, 2024
6fa7dac
Merge branch 'feature/venom_deploy_instruction_removal' of github.com…
charles-cooper Jan 2, 2024
8db8e29
whitespace
charles-cooper Jan 2, 2024
7ac92d8
rename some variables with edit distance == 1
charles-cooper Jan 2, 2024
88f1fa3
add some variables
charles-cooper Jan 2, 2024
8f93bf1
assert message
charles-cooper Jan 2, 2024
fd78df9
fix minor bugs
charles-cooper Jan 2, 2024
fe0e154
fix cfg normalization code
charles-cooper Jan 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions tests/unit/compiler/venom/test_duplicate_operands.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ def test_duplicate_operands():
ctx = IRFunction()
bb = ctx.get_basic_block()
op = bb.append_instruction("store", 10)
sum = bb.append_instruction("add", op, op)
bb.append_instruction("mul", sum, op)
sum_ = bb.append_instruction("add", op, op)
bb.append_instruction("mul", sum_, op)
bb.append_instruction("stop")

asm = generate_assembly_experimental(ctx, OptimizationLevel.CODESIZE)
asm = generate_assembly_experimental(ctx, optimize=OptimizationLevel.CODESIZE)

assert asm == ["PUSH1", 10, "DUP1", "DUP1", "DUP1", "ADD", "MUL", "STOP", "REVERT"]
6 changes: 3 additions & 3 deletions tests/unit/compiler/venom/test_multi_entry_block.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ def test_multi_entry_block_1():
finish_bb = ctx.get_basic_block(finish_label.value)
cfg_in = list(finish_bb.cfg_in.keys())
assert cfg_in[0].label.value == "target", "Should contain target"
assert cfg_in[1].label.value == "finish_split_global", "Should contain finish_split_global"
assert cfg_in[1].label.value == "finish_split___global", "Should contain finish_split___global"
assert cfg_in[2].label.value == "finish_split_block_1", "Should contain finish_split_block_1"


Expand Down Expand Up @@ -93,7 +93,7 @@ def test_multi_entry_block_2():
finish_bb = ctx.get_basic_block(finish_label.value)
cfg_in = list(finish_bb.cfg_in.keys())
assert cfg_in[0].label.value == "target", "Should contain target"
assert cfg_in[1].label.value == "finish_split_global", "Should contain finish_split_global"
assert cfg_in[1].label.value == "finish_split___global", "Should contain finish_split___global"
assert cfg_in[2].label.value == "finish_split_block_1", "Should contain finish_split_block_1"


Expand Down Expand Up @@ -134,5 +134,5 @@ def test_multi_entry_block_with_dynamic_jump():
finish_bb = ctx.get_basic_block(finish_label.value)
cfg_in = list(finish_bb.cfg_in.keys())
assert cfg_in[0].label.value == "target", "Should contain target"
assert cfg_in[1].label.value == "finish_split_global", "Should contain finish_split_global"
assert cfg_in[1].label.value == "finish_split___global", "Should contain finish_split___global"
assert cfg_in[2].label.value == "finish_split_block_1", "Should contain finish_split_block_1"
20 changes: 11 additions & 9 deletions vyper/compiler/phases.py
Original file line number Diff line number Diff line change
Expand Up @@ -181,13 +181,9 @@ def global_ctx(self) -> ModuleT:
@cached_property
def _ir_output(self):
# fetch both deployment and runtime IR
nodes = generate_ir_nodes(
return generate_ir_nodes(
self.global_ctx, self.settings.optimize, self.settings.experimental_codegen
)
if self.settings.experimental_codegen:
return [generate_ir(nodes[0]), generate_ir(nodes[1])]
else:
return nodes

@property
def ir_nodes(self) -> IRnode:
Expand All @@ -208,21 +204,27 @@ def function_signatures(self) -> dict[str, ContractFunctionT]:
fs = self.vyper_module_folded.get_children(vy_ast.FunctionDef)
return {f.name: f._metadata["func_type"] for f in fs}

@cached_property
def venom_functions(self):
return generate_ir(self.ir_nodes, self.settings.optimize)

@cached_property
def assembly(self) -> list:
if self.settings.experimental_codegen:
deploy_code, runtime_code = self.venom_functions
assert self.settings.optimize is not None # mypy hint
return generate_assembly_experimental(
self.ir_nodes, self.settings.optimize # type: ignore
runtime_code, deploy_code=deploy_code, optimize=self.settings.optimize
)
else:
return generate_assembly(self.ir_nodes, self.settings.optimize)

@cached_property
def assembly_runtime(self) -> list:
if self.settings.experimental_codegen:
return generate_assembly_experimental(
self.ir_runtime, self.settings.optimize # type: ignore
)
_, runtime_code = self.venom_functions
assert self.settings.optimize is not None # mypy hint
return generate_assembly_experimental(runtime_code, optimize=self.settings.optimize)
else:
return generate_assembly(self.ir_runtime, self.settings.optimize)

Expand Down
33 changes: 24 additions & 9 deletions vyper/venom/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# maybe rename this `main.py` or `venom.py`
# (can have an `__init__.py` which exposes the API).

from typing import Optional
from typing import Any, Optional

from vyper.codegen.ir_node import IRnode
from vyper.compiler.settings import OptimizationLevel
Expand All @@ -17,19 +17,26 @@
from vyper.venom.passes.dft import DFTPass
from vyper.venom.venom_to_assembly import VenomCompiler

DEFAULT_OPT_LEVEL = OptimizationLevel.default()


def generate_assembly_experimental(
ctx: IRFunction, optimize: Optional[OptimizationLevel] = None
runtime_code: IRFunction,
deploy_code: Optional[IRFunction] = None,
optimize: OptimizationLevel = DEFAULT_OPT_LEVEL,
) -> list[str]:
compiler = VenomCompiler(ctx)
return compiler.generate_evm(optimize is OptimizationLevel.NONE)
# note: VenomCompiler is sensitive to the order of these!
if deploy_code is not None:
functions = [deploy_code, runtime_code]
else:
functions = [runtime_code]

compiler = VenomCompiler(functions)
return compiler.generate_evm(optimize == OptimizationLevel.NONE)

def generate_ir(ir: IRnode, optimize: Optional[OptimizationLevel] = None) -> IRFunction:
# Convert "old" IR to "new" IR
ctx = convert_ir_basicblock(ir)

# Run passes on "new" IR
def _run_passes(ctx: IRFunction, optimize: OptimizationLevel) -> None:
# Run passes on Venom IR
# TODO: Add support for optimization levels
while True:
changes = 0
Expand All @@ -53,4 +60,12 @@ def generate_ir(ir: IRnode, optimize: Optional[OptimizationLevel] = None) -> IRF
if changes == 0:
break

return ctx

def generate_ir(ir: IRnode, optimize: OptimizationLevel) -> tuple[IRFunction, IRFunction]:
# Convert "old" IR to "new" IR
ctx, ctx_runtime = convert_ir_basicblock(ir)

_run_passes(ctx, optimize)
Fixed Show fixed Hide fixed
_run_passes(ctx_runtime, optimize)
Fixed Show fixed Hide fixed

return ctx, ctx_runtime
21 changes: 0 additions & 21 deletions vyper/venom/analysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,27 +19,6 @@ def calculate_cfg(ctx: IRFunction) -> None:
bb.cfg_out = OrderedSet()
bb.out_vars = OrderedSet()

# TODO: This is a hack to support the old IR format where `deploy` is
# an instruction. in the future we should have two entry points, one
# for the initcode and one for the runtime code.
deploy_bb = None
after_deploy_bb = None
for i, bb in enumerate(ctx.basic_blocks):
if bb.instructions[0].opcode == "deploy":
deploy_bb = bb
after_deploy_bb = ctx.basic_blocks[i + 1]
break

if deploy_bb is not None:
assert after_deploy_bb is not None, "No block after deploy block"
entry_block = after_deploy_bb
has_constructor = ctx.basic_blocks[0].instructions[0].opcode != "deploy"
if has_constructor:
deploy_bb.add_cfg_in(ctx.basic_blocks[0])
entry_block.add_cfg_in(deploy_bb)
else:
entry_block = ctx.basic_blocks[0]

for bb in ctx.basic_blocks:
assert len(bb.instructions) > 0, "Basic block should not be empty"
last_inst = bb.instructions[-1]
Expand Down
9 changes: 3 additions & 6 deletions vyper/venom/basicblock.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from vyper.utils import OrderedSet

# instructions which can terminate a basic block
BB_TERMINATORS = frozenset(["jmp", "djmp", "jnz", "ret", "return", "revert", "deploy", "stop"])
BB_TERMINATORS = frozenset(["jmp", "djmp", "jnz", "ret", "return", "revert", "stop"])

VOLATILE_INSTRUCTIONS = frozenset(
[
Expand Down Expand Up @@ -33,7 +33,6 @@

NO_OUTPUT_INSTRUCTIONS = frozenset(
[
"deploy",
"mstore",
"sstore",
"dstore",
Expand All @@ -56,9 +55,7 @@
]
)

CFG_ALTERING_INSTRUCTIONS = frozenset(
["jmp", "djmp", "jnz", "call", "staticcall", "invoke", "deploy"]
)
CFG_ALTERING_INSTRUCTIONS = frozenset(["jmp", "djmp", "jnz", "call", "staticcall", "invoke"])

if TYPE_CHECKING:
from vyper.venom.function import IRFunction
Expand Down Expand Up @@ -273,7 +270,7 @@ def _ir_operand_from_value(val: Any) -> IROperand:
if isinstance(val, IROperand):
return val

assert isinstance(val, int)
assert isinstance(val, int), val
return IRLiteral(val)


Expand Down
33 changes: 23 additions & 10 deletions vyper/venom/function.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
MemType,
)

GLOBAL_LABEL = IRLabel("global")
GLOBAL_LABEL = IRLabel("__global")


class IRFunction:
Expand All @@ -18,7 +18,10 @@ class IRFunction:
"""

name: IRLabel # symbol name
entry_points: list[IRLabel] # entry points
args: list
ctor_mem_size: Optional[int]
immutables_len: Optional[int]
basic_blocks: list[IRBasicBlock]
data_segment: list[IRInstruction]
last_label: int
Expand All @@ -28,14 +31,30 @@ def __init__(self, name: IRLabel = None) -> None:
if name is None:
name = GLOBAL_LABEL
self.name = name
self.entry_points = []
self.args = []
self.ctor_mem_size = None
self.immutables_len = None
self.basic_blocks = []
self.data_segment = []
self.last_label = 0
self.last_variable = 0

self.add_entry_point(name)
self.append_basic_block(IRBasicBlock(name, self))

def add_entry_point(self, label: IRLabel) -> None:
"""
Add entry point.
"""
self.entry_points.append(label)

def remove_entry_point(self, label: IRLabel) -> None:
"""
Remove entry point.
"""
self.entry_points.remove(label)

def append_basic_block(self, bb: IRBasicBlock) -> IRBasicBlock:
"""
Append basic block to function.
Expand Down Expand Up @@ -91,7 +110,7 @@ def remove_unreachable_blocks(self) -> int:
removed = 0
new_basic_blocks = []
for bb in self.basic_blocks:
if not bb.is_reachable and bb.label.value != "global":
if not bb.is_reachable and bb.label not in self.entry_points:
removed += 1
else:
new_basic_blocks.append(bb)
Expand Down Expand Up @@ -119,16 +138,10 @@ def normalized(self) -> bool:
if len(bb.cfg_in) <= 1:
continue

# Check if there is a conditional jump at the end
# Check if there is a branching jump at the end
# of one of the predecessors
#
# TODO: this check could be:
# `if len(in_bb.cfg_out) > 1: return False`
# but the cfg is currently not calculated "correctly" for
# the special deploy instruction.
for in_bb in bb.cfg_in:
jump_inst = in_bb.instructions[-1]
if jump_inst.opcode in ("jnz", "djmp"):
if len(in_bb.cfg_out) > 1:
return False

# The function is normalized
Expand Down
53 changes: 29 additions & 24 deletions vyper/venom/ir_node_to_venom.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,19 +87,35 @@ def _get_symbols_common(a: dict, b: dict) -> dict:
return ret


def convert_ir_basicblock(ir: IRnode) -> IRFunction:
global_function = IRFunction()
_convert_ir_basicblock(global_function, ir, {}, OrderedSet(), {})
def _findIRnode(ir: IRnode, value: str) -> Optional[IRnode]:
if ir.value == value:
return ir
for arg in ir.args:
if isinstance(arg, IRnode):
ret = _findIRnode(arg, value)
if ret is not None:
return ret
return None


def convert_ir_basicblock(ir: IRnode) -> tuple[IRFunction, IRFunction]:
deploy_ir = _findIRnode(ir, "deploy")
assert deploy_ir is not None

deploy_venom = IRFunction()
_convert_ir_basicblock(deploy_venom, ir, {}, OrderedSet(), {})
deploy_venom.get_basic_block().append_instruction("stop")

for i, bb in enumerate(global_function.basic_blocks):
if not bb.is_terminated and i < len(global_function.basic_blocks) - 1:
bb.append_instruction("jmp", global_function.basic_blocks[i + 1].label)
runtime_ir = deploy_ir.args[1]
runtime_venom = IRFunction()
_convert_ir_basicblock(runtime_venom, runtime_ir, {}, OrderedSet(), {})

revert_bb = IRBasicBlock(IRLabel("__revert"), global_function)
revert_bb = global_function.append_basic_block(revert_bb)
revert_bb.append_instruction("revert", 0, 0)
# Connect unterminated blocks to the next with a jump
for i, bb in enumerate(runtime_venom.basic_blocks):
if not bb.is_terminated and i < len(runtime_venom.basic_blocks) - 1:
bb.append_instruction("jmp", runtime_venom.basic_blocks[i + 1].label)

return global_function
return deploy_venom, runtime_venom


def _convert_binary_op(
Expand Down Expand Up @@ -279,20 +295,9 @@ def _convert_ir_basicblock(ctx, ir, symbols, variables, allocated_variables):
elif ir.value in ["pass", "stop", "return"]:
pass
elif ir.value == "deploy":
memsize = ir.args[0].value
ir_runtime = ir.args[1]
padding = ir.args[2].value
assert isinstance(memsize, int), "non-int memsize"
assert isinstance(padding, int), "non-int padding"

runtimeLabel = ctx.get_next_label()

ctx.get_basic_block().append_instruction("deploy", memsize, runtimeLabel, padding)

bb = IRBasicBlock(runtimeLabel, ctx)
ctx.append_basic_block(bb)

_convert_ir_basicblock(ctx, ir_runtime, symbols, variables, allocated_variables)
ctx.ctor_mem_size = ir.args[0].value
ctx.immutables_len = ir.args[2].value
return None
elif ir.value == "seq":
func_t = ir.passthrough_metadata.get("func_t", None)
if ir.is_self_call:
Expand Down
8 changes: 3 additions & 5 deletions vyper/venom/passes/normalization.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,11 @@ class NormalizationPass(IRPass):
changes = 0

def _split_basic_block(self, bb: IRBasicBlock) -> None:
# Iterate over the predecessors of the basic block
# Iterate over the predecessors to this basic block
for in_bb in list(bb.cfg_in):
jump_inst = in_bb.instructions[-1]
assert bb in in_bb.cfg_out

# Handle branching
if jump_inst.opcode in ("jnz", "djmp"):
# Handle branching in the predecessor bb
if len(in_bb.cfg_out) > 1:
self._insert_split_basicblock(bb, in_bb)
self.changes += 1
break
Expand Down
Loading
Loading