Extension / Script load and execution order system

[w-e-w]

please help and test if there's any flaws to the system

---

Description

Currently webui loads extensions based on its installation path
initially this is fine but as the amount of extension and the complexity grows this becomes increasingly troublesome

Here I implemented a system that allows a extension developer to specify

1. the load order each entry_script.py
2. the Script class initialization order of each scripts.Script class
3. the function execution order of almost every function in each scripts.Script class

this should solve the issue of an extension depending on another extensions being loaded first

example: some extensions been required control net to be loaded first

and opens up the possibility of executing callback functions in special sequence

example: a1>b1>c1>c2>b2>a2

Usage explanation

scrip_file.py Load order

in the route to directory of an extension create a webui-extension-properties.json with the following structure

{
    "load_order": {
        "script_file.py": 55555,
        "script_file.py": 66666,
        "script_file.py": 77777
    }
}

script_file_name.py corresponds to each entry script script_file.py under the scripts directory the requires a custom load order

Function execution order and Script class initialization order

both of these can be configured using the same method
in a custom scripts.Script class after defining a callback function add a order attribute the function method

from modules import scripts
class ExampleScript(scripts.Script):
    def title(self):
        return "tExampleScript"
    title.order = 87654
    def ui(self, is_img2img):
        pass
    ui.order = 76543
    def show(self, is_img2img):
        return scripts.AlwaysVisible
    show.order = 65432
    def before_process(self, p, *args):
        pass
    before_process.order = 54321
    def postprocess_batch(self, p, *args, **kwargs):
        pass
    postprocess_batch.order = 43210

Function execution order
callback function from all Scripts will be sorted based on their order attribute

details
show.order also acts as the Script class initialization order
ui.order has ability to rearrange the order of gr.Accordion of a always on script
title.order serves as the order of how a script is stored order in webui but in most cases the store order shouldn't matter should be irrelevant, but it can be used to control the order of how selectable strips are listed
other orders like the callbacks of alwayson_scripts such as before_process.order postprocess_batch are normal and no special purpose other than defining the

these are all the current function that have `order attribute`

[
    'ui',
    'setup',
    'before_process',
    'process',
    'before_process_batch',
    'after_extra_networks_activate',
    'process_batch',
    'before_hr',
    'postprocess',
    'postprocess_batch',
    'postprocess_batch_list',
    'postprocess_image',
    'before_component',
    'after_component',
]

there are certain functions that I did not add the attribute
either because I believe they do not need it or I'm not sure the function of
the run function used by selectable Scripts does not need to be ordered
I'm not familiar with on_before_component and on_after_component and as it seem to serve different purpose I did not add order to it

---

all orders can override be user in config.json

currently the setting is only accessible by directly editing the config file file
and adding the dict key script_order_override
restructure of the values is as follows

"script_order_override": {
    "scripts\\xyz_grid.py": 12345,
    "extensions\\example_extension\\scripts\\script_0.py": 23456
    "scripts\\xyz_grid.py>Script>ui": 34567
    "extensions\\example_extension\\scripts\\script_2.py>ScriptClassName>before_process_batch": 45678
}

instructor is simple identifying string as key order number as value
the first two entries in this example are Scriptfile.py Load order entries
the last two are entries for Script function > character is used as a separator
for exact detail reference the object_id in scripts.get_function_execution_order and scripts.get_script_load_order

note the \\ is due to json encoding, the real string is \

• the user is able to override any configuration by the script itself the user defined configuration has the top priority and will override any predefined order by an extension

currently there is no user interface to configure order override
this can be implemented at the later date if this system is accepted

not sure if I would be the one to implemented as I'm not good with UI

defalt order when order is not specified

applies to current extensions
a default value will be assigned automatically by web UI if order is not specified
the order is given based on the type source of the extension / script

internal webui scripts: 10000
built-in scripts: 30000
built-in extensions: 50000
extension: 70000
if for some reason it did not match any of the four categorie a order of 1000000 is assigned

included bug fix

there's some strange behavior on of the original (current) script loading order

stable-diffusion-webui/modules/scripts.py

Lines 368 to 374 in 5e80d9e

	def orderby(basedir):
	# 1st webui, 2nd extensions-builtin, 3rd extensions
	priority = {os.path.join(paths.script_path, "extensions-builtin"):1, paths.script_path:0}
	for key in priority:
	if basedir.startswith(key):
	return priority[key]
	return 9999

base on comment message
64b7e83
to described behavior is 1st webui, 2nd extensions-builtin, 3rd extensions
but in reality this code produces
1st built-in-scripts + webui, 2nd extensions, 3rd extensions-builtin
in this PR it is corrected too the order described above in defalt order when order is not specified

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests

[w-e-w]

base on things from #13944

assuming that I didn't misunderstand wfjs's system

I think wfjs's system may do a better job of loading extensions in the correct order then my system

it might still be necessary for edge cases to allow a similar system to my for manual control
maybe my system can act as a fallback?

but as wfjs's system only addresses the the loading extensions
while my system there's two parts loading extensions extension execution
I believe the best thing to do is to combine loading extensions for wfjs's system, and use my system for extension executing

---

different from loading extensions, which is more of a pass / fail situation
extension execution (call back function execution order) depends more on the functionality of a particular function and the desired result by the user

for example two different extensions provides two distinct image post-processing procedure, apply them in either order are valid, so the only determining factor is to let the user decide what they wish to achieve